Index Module Index Search Page

qnd.ncf module

QnD netCDF3 interface.

qnd.ncf.opennc(filename, mode='r', auto=1, **kwargs)[source]

Open netCDF-3 file returning a QnD QGroup.

A netCDF-3 file differs from other self-describing binary file formats because no data addresses can be known until every variable to be stored is declared. Therefore, when writing a netCDF-3 file, you must declare every variable before you can begin writing anything.

The qnd API is somewhat at odds with this semantics because it encourages you to declare and write each variable in a single step. The native netCDF-3 API forces you to declare everything, then call an enddef method to complete all variable declarations and permit you to begin writing data. The qnd.ncf backend uses the first call to the ordinary qnd flush method to emulate the netCDF-3 enddef mode switch – thus nothing will be written to the file until the first call to flush. To minimize the difference between ncf and other qnd backends, if you do use the usual qnd declare-and-write idiom, the ncf backend will save the variable value in memory until the first flush call, which will trigger the actual writing of all such saved values.

Note that closing the file flushes it, so that is also a viable way to finish a netCDF-3 file. Furthermore, when you overwrite any record variable in recording mode, ncf will implicitly flush the file, since no new variables can be declared after that.

Note that you use the standard QnD API, a copy of every variable you write to the file until you begin the second record will be kept in memory, which could potentially be a problem. If you wish to declare all variables before writing anything, so that your code is aligned with the netCDF API, do something like this:

f = opennc("myfile??.nc", "w")  # wildcards expand to 00, 01, 02, ...
# declare non-record variables from in-memory arrays
f.nrvar1 = nrvar1.dtype, nrvar1.shape
f.nrvar2 = nrvar2.dtype, nrvar2.shape
# declare record variables from in-memory arrays
f.recording(1)
f.rvar1 = rvar1.dtype, rvar1.shape
f.rvar2 = rvar2.dtype, rvar2.shape
# flushing the file is equivalent to netCDF ENDDEF mode switch
f.flush()
# now write the current values of all the variables
f.nrvar1 = nrvar1
f.nrvar2 = nrvar2
# writing the record variables writes their values for first record
f.rvar1 = rvar1
f.rvar2 = rvar2
# change values of record variables and write the second record
f.rvar1 = rvar1
f.rvar2 = rvar2
# when you've written all records, close the file
f.close()
Parameters:
  • filename (str) – Name of file to open. See notes below for file family.
  • mode (str) – One of ‘r’ (default, read-only), ‘r+’ (read-write, must exist), ‘a’ (read-write, create if does not exist), ‘w’ (create, clobber if exists), ‘w-’ (create, fail if exists).
  • auto (int) – The intial state of auto-read mode. If the QGroup handle returned by openh5 is f, then f.varname reads an array variable, but not a subgroup when auto=1, the default. With auto=0, the variable reference reads neither (permitting later partial reads in the case of array variables). With auto=2, a variable reference recursively reads subgroups, bringing a whole tree into memory.
  • **kwargs – Other keywords. The maxsize keyword sets the size of files in a family generated in recording==1 mode; a new file will begin when the first item in a new record would begin beyond maxsize. The default maxsize is 128 MiB (134 MB). The v64 keyword, if provided and true, causes new files to be created using the 64-bit netCDF format; the default is to create 32-bit files. (But a file family always uses a single format.) The nextaddr_mode keyword can be used to indicate whether the next new record in ‘a’ or ‘r+’ mode should go into a new file. The default behavior is that it should, which is the pdbf module default; this is nextaddr_mode true. Use nextaddr_mode=0 to continue filling the final existing file until maxsize.
Returns:

f – A file handle implementing the QnD interface.
Return type:

QGroup

Notes

The filename may be an iterable, one string per file in order. The sequence may extend beyond the files which actually exist for ‘r+’, ‘a’, ‘w’, or ‘w-’ modes.

Alternatively filename specifies a family if it contains shell globbing wildcard characters. Existing matching files are sorted first by length, then alphabetically (ensuring that ‘file100’ comes after ‘file99’, for example). If there is only a single wildcard group, it also serves to define a sequence of future family names beyond those currently existing for ‘r+’, ‘a’, ‘w’, or ‘w-’ modes. A ‘?’ pattern is treated the same as a ‘[0-9]’ pattern if all its matches are digits or if the pattern matches no existing files. Similarly, a ‘*’ acts like the minimum number of all-digit matches, or three digits if there are no matches.