2 Module: ufileformat

This module is used for binary output and input of large 1D, 2D or 3D matrices. The GHER and a subset of the NetCDF format is currently supported. The matrix can contain exclusion points (“holes”). Matrices A where the elements are a linear combination of the indices can also be efficiently represented:

A(i,j,k) = {a}_{0} + {a}_{1}i + {a}_{2}j + {a}_{3}k
(1)

Only the coefficient {a}_{0}, {a}_{1}, {a}_{2} and {a}_{3} are stored. These file are called degenerated. For example, the longitude and latitude of each grid point can often be expressed in this way.

For the GHER format, each file represent a real matrix. If the file names ends with .gz, then the file is uncompressed (with gunzip) in the user’s temporary directory defined by the environment variable $TMPDIR (or by default in /tmp). Simple Fortran 90-style extraction can also by performed with the module ufileformat. A coma separated list of indices or ranges of indices in parenthesis can be appended to the file name, if only a subsection of the matrix should be loaded.

For example if the file toto.TEM is a 10 x 10 x 10 matrix, the “file”:

toto.TEM(:,:,6)
is 10x10x1 matrix containing all elements with the 3rd indices equal to 6.
toto.TEM(:,end,:)
is 10x1x10 matrix containing all elements with the 2nd indices equal to 10.
toto.TEM(1:,:end,1:end)
is 10x10x10 matrix equal to the original matrix

But no arithmetic with the indices (for example toto.TEM(:,end-1,:)) are allowed. If data extraction is used with degenerated matrices, the four coefficient are changed accordingly to the subsection chosen.

Data extraction and automatic decompression can only be used for loading data.

A variable in a NetCDF file can be loaded by specifying a “file name” of the following form:

NetCDF_filename#NetCDF_variable

If the NetCDF file name end with .gz, then the file is uncompressed as with the GHER file format. The data extraction follows also the same rules as above. For example, the following is a valid file name for loading a matrix.

file.nc.gz#temp(:,:,1)

The file file.nc.gz is first decompressed, then the slice with the 3rd indices equal to 1 of the variable temp is returned to the calling program.

The special value for missing data is stored in the variables attribute missing_data. In the case of degenerated file, the attribute shape must be present, containing the shape of the matrix. The actual value of the variable contains the coefficients {a}_{i}.

2.1 Order of the dimensions

The reported order of the dimensions depends on the tool that you are using to query and access a file. Two types of ordering schemes exists:

column-major order
: used by Fortran programs such as OAK
row-major order
: used by C programs such as ncdump

The order of the dimensions for NetCDF follows the recommendation of the CF-convention. If you query your NetCDF files with ncdump, the order of the dimensions should be time, depth, latitude, longitude. For a Fortran program reading this file the dimensions with automatically be longitude, latitude, depth and time since Fortran uses the column-major order (as opposed to ncdump). For Fortran binary files, the order of the dimensions is also longitude, latitude, depth and time.