Alternate Run Modes:MPI and OpenMP Parallelization

In GSSHA, there are three parallelization modes supported. When running a simulation, a control mode and a parallelization mode are both selected; the default for most runs is a single-threaded (serial), run once mode. Apart from the single-thread or serial mode, GSSHA^TM can be run in OpenMP or MPI mode.

OpenMP

OpenMP is a parallelization paradigm that assumes you are running on a machine with multiple logical processors and a shared pool of memory. When running, OpenMP splits up some portions of the code to run in parallel on multiple threads, while others are run on a single thread or processing unit.

The following is a list of the portions of the code that have been parallelized using OpenMP.

Overland boundary conditions
Rain gage nearest neighbor algorithm (for Theissen polygon interpolation)
Rain gage inverse distance algorithm (for IDW interpolation)
Green and Ampt infiltration algorithm
Multi-layer Green and Ampt infiltration algorithm
Green and Ampt w/ Soil Moisture Redistribution algorithm
Evapotranspiration algorithm
NSM stream kinetics
NSM overland kinetics
NSM soil-overland interaction
Explicit, ADE, and ADE-PC overland flow algorithms
Some parts of the SCE routine

There are some significant portions of the code that have not been parallelized with OpenMP yet, such as groundwater, Richard's infiltration, and constituent transport.

The speed-up from using OpenMP will depend on various factors such as:

How big is the simulation (how many cells)
What portion of work is done by the parallelized codes v. the non-parallelized codes
How many threads (logical processors) are you using (versus how many actual processing units you have)

The one factor you have control over for OpenMP, if you are running in single-run mode, is the number of threads you want to dedicate to a particular process. For batch, SCE, and monte carlo methods it sets the thread count to the total number of logical processors you have running. Note that if you have a CPU with hyperthreading then each processor on the CPU appears as two logical processors. Thus a dual-core CPU with hyperthreading turned on would appear as four logical processors. There is some improvement from hyperthreading, especially if a lot of writing to output files is being done.

To set the thread count for single-run modes, use the following project file card.

NUM_THREADS ##

where ## is the number of threads to set; must be greater than or equal to 1. If you run a version of GSSHA^TM that was not compiled with OpenMP, the NUM_THREADS project card will result in a warning that it was not compiled with OpenMP; GSSHA^TM will then continue to run in single-thread mode.

MPI

Unlike OpenMP, MPI mode currently is only useful for running multiple versions of a single simulation. This is best suited for monte carlo runs on supercomputers. To run in MPI mode, you must use an external program that handles the setup and communication processes, such as MPICH. Typically these program require that you specify the GSSHA^TM command line as part of the command line for the MPI program. The GSSHA^TM command line for running the various control modes is specified as part of the control mode description in the following sections.