Lesson 13 Using HDF Data

The Hierarchical Data Format (HDF), developed and supported by the National Center for Supercomputing Applications is a very flexible and platform independent binary format for all kinds of data. Similar to managing files in folders with your computer, HDF organizes datasets in groups and a group may contain other groups. Datasets and groups may have attributes. A group may also have a comment.

Comparing with netCDF, which is mainly for grid data, HDF allows storage of many different types of scientific data, including images, multidimensional data arrays, record oriented data, and point data. Thus the interfaces are much more complex. zeGraph uses the simple array part of the interfaces for data input and output that are not convenient or simply impossible by using netCDF. The following example shows how to use zeHDF object to create a HDF file and put some data in it.

require("register")

h5, data, spec = zeUtl.new("hdf", "int", "uint")

h5:open("my.h5", "w")

-- Create a group because it does not exist and the file is for writing.

h5:group("group1")

-- Wite comment to the created group.

h5:putcmm("/group1", "Test creating group")

-- Wite comment to the root group.

h5:putcmm("/", "Test HDF writing")

-- Preparing the data and specification

data:range(1, 1, 10);
spec:setele(0, 0, 10)
ptr, type = data:getptr()

-- Create a dataset with the name "data" in the currently opened/greated group.

h5:dataset("data", spec, type)

-- write data to the dataset.

h5:putvar(data)

-- write attribute to the data

h5:putatt("dataset", "range", 1, 10)

-- write attribute to the group

h5:putatt("group", "g_att", "Group can have attibute too.")

If you use the HDF dump tool to extract contents in the file, you will get:

HDF5 "d:\lua\my.h5" {
GROUP "/" {
   COMMENT "Test HDF writing"
   GROUP "group1" {
      COMMENT "Test creating group"
      ATTRIBUTE "g_att" {
         DATATYPE  H5T_STD_I8LE  
         DATASPACE  SIMPLE { ( 28 ) / ( 28 ) } 
         DATA {
            71, 114, 111, 117, 112, 32, 99, 97, 110, 32, 104, 97, 118, 101, 32,
            97, 116, 116, 105, 98, 117, 116, 101, 32, 116, 111, 111, 46
         } 
      } 
      DATASET "data" {
         DATATYPE  H5T_STD_I32LE  
         DATASPACE  SIMPLE { ( 10 ) / ( 10 ) } 
         DATA {
            1, 2, 3, 4, 5, 6, 7, 8, 9, 10
         } 
         ATTRIBUTE "range" {
            DATATYPE  H5T_IEEE_F64LE  
            DATASPACE  SIMPLE { ( 2 ) / ( 2 ) } 
            DATA {
               1, 10
            } 
         } 
      } 
   } 
} 
}

Now let's read the file contents back:

require("register")

h5, data = zeUtl.new("hdf", "int")

h5:open("my.h5")

h5:group("group1")
print(h5:getcmm("/"))
print(h5:getcmm("/group1"))
print(h5:getatt("group", "g_att"))

h5:dataset("data")
print(h5:getatt("dataset", "range"))

h5:getvar(data)
data:transpose()

io.flush()
data:print()

--[[Outputs:
Test HDF writing
Test creating group
Group can have attibute too.
1	10
1,2,3,4,5,6,7,8,9,10
]]