Software Installation and Compilation

---

System Requirement

This library is written in Fortran. A Fortran 95 compatible compiler is the minimum requirement. In addition, the library code also relies on allocatable enhancement features (defined in ISO TR 15581, and officially part of Fortran 2003) and Cray pointers (for the shared-memory implementation only). These features are almost universally supported by modern Fortran compilers.

For FFT computations, the library can interface with many external FFT libraries which, of course, need to be supplied. Some advanced software options may rely on other 3rd-party libraries.

Using the Binary Package

A binary package may be available on your system for testing purpose. Suppose a binary package is installed at $DECOMP2D_DIR and a Fortran compiler is defined by $F90, to compile an application test.f90 using the library, do:

      $F90 -I$DECOMP2D_DIR/include test.f90 -L$DECOMP2D_DIR/lib -l2decomp_fft -o test.exe

Please note that when creating real-world applications using 2DECOMP&FFT, a pre-built binary release is unlikely to suit your needs. This is because the library has many software options that are either hardware or application dependent. Users are always recommended to build the library from source.

Build the Library from Source Using GNU Make

To build the library from source, unpack the source package and follow the instructions here. There are Makefiles located in the $DECOMP2D_DIR/src directory. Normally there is no need to modify the main Makefile. Create a Makefile.inc instead from which software options and platform dependent information can be set. A couple of sample Makefile.inc files prepared for popular hardware are provided as templates.

Software Options

One may choose to turn on/off certain software features in $DECOMP2D_DIR/src/Makefile.inc by appending flags to the OPTIONS variable. The available options are:

• -DDOUBLE_PREC - Use this flag to build the library in double-precision mode. Single precision is the default. Click here for further instructions on switching between single precision and double precision in applications
• -DEVEN - All communication code in this library is written to support general uneven data distributions. By default MPI_ALLTOALLV is used for communications. Turn on this flag to use MPI_ALLTOALL routines instead if:
  • The global data can be evenly distributed across processors in all three pencil orientations. For a (nx, ny, nz) global data set on a (p_row, p_col) processor grid, nx & ny have to be multiples of p_row and ny & nz have to be multiples of p_col.
• -DSHM - Use this flag to turn on the System V shared-memory version of communication code. An optional package has to be downloaded and installed first. Check the Shared-memory Programming page for technical details.
• -DOVERWRITE - This flag indicates that it is safe to overwrite the input data of the FFTs. Turning on this flag will reduce the memory footprint of several FFT implementations and may improve the overall performance. More details about memory usage can be found here.
• -DOCC - OCC stands for Overlap of Communications and Computations. A set of API to facilitate this in applications may be switch on by setting the -DOCC flag. An external library is required for this to function and instructions are available here.

If the FFT part of the library is going to be used, users can also specify a suitable FFT engine by setting the FFT variable. 2DECOMP&FFT can interface with the following FFT libraries: FFTW (version 3.x), ACML, ESSL, MKL, FFTPACK, FFTE and even an experimental GPU engine using CUFFT. Please note that not all options are available on all systems. For example you won't find an MKL installation on a AMD Opteron-based supercomputer. For details, check the FFT page.

System Dependent Information

Users are also responsible of providing a set of system dependent information for their specific hardware platforms, including compiler flags and external library location. The following variables need to be set:

• For a compiler suite
  • F90 - Fortran compiler command line (this often needs to be the name of the MPI wrapper application - e.g. mpif90 rather than gfortran).
  • CPPFLAG - The Fortran compiler flag to turn on preprocessing.
  • OPTIM - The optimisation or debugging flags for the Fortran compiler.
  • CRAYPTR - The flag to turn on Cray pointer support.
  • CC - C compiler command line.
  • CFLGAS - C compiler optimisation or debugging flags.
  The last three variables are optional - they are only required if the shared-memory code is in use.
• For an external FFT library
  • INC - The include path pointing to the header files of the external library, if applicable.
  • LIBFFT - Extra parameters, such as the library path, used by compilers at link stage.
  Both may be left blank on certain platforms.

Several sample Makefile.inc are distributed with the package.

FreeIPC for Shared-memory Code

To use the shared-memory implementation, download the FreeIPC package, extract it into the $DECOMP2D_DIR/src directory, turn on the -DSHM flag in $DECOMP2D_DIR/src/Makefile.inc and build 2DECOMP&FFT as normal. FreeIPC, provided by Ian Bush of NAG, allows easy access to System V Inter-Process Communication API from Fortran applications. This package has to be downloaded separately because it uses a different software license.