LazyArrayDict

class temporaldata.LazyArrayDict(**kwargs)

Bases: ArrayDict

Lazy variant of ArrayDict. The data is not loaded until it is accessed. This class is meant to be used when the data is too large to fit in memory, and is intended to be intantiated via. LazyArrayDict.from_hdf5.

Note

To access an attribute without triggering the in-memory loading use self.__dict__[key] otherwise using self.key or getattr(self, key) will trigger the lazy loading and will automatically convert the h5py dataset to a numpy array as well as apply any outstanding masks.

select_by_mask(mask)

Index all arrays with a boolean mask and return a copy.

Lazy attributes will remain lazy, and masking will be applied to them upon access.

Parameters:

mask (ndarray) – Boolean array used for masking. The mask needs to be 1-dimensional, and of equal length as the object itself.

classmethod from_dataframe(df, unsigned_to_long=True)

Creates an ArrayDict object from a pandas DataFrame.

The columns in the DataFrame are converted to arrays when possible, otherwise they will be skipped.

Parameters:

• df (pandas.DataFrame) – DataFrame.

• unsigned_to_long (bool, optional) – If True, automatically converts unsigned integers to int64. Defaults to True.

to_hdf5(file)

Saves the data object to an HDF5 file.

Parameters:

file (h5py.File) – HDF5 file.

import h5py
from temporaldata import ArrayDict

data = ArrayDict(
    unit_id=np.array(["unit01", "unit02"]),
    brain_region=np.array(["M1", "M1"]),
    waveform_mean=np.zeros((2, 48)),
)

with h5py.File("data.h5", "w") as f:
    data.to_hdf5(f)

classmethod from_hdf5(file)

Loads the data object from an HDF5 file.

Parameters:

file (h5py.File) – HDF5 file.

import h5py
from temporaldata import ArrayDict

with h5py.File("data.h5", "r") as f:
    data = ArrayDict.from_hdf5(f)

keys()

Returns a list of all array attribute names.

Return type:

List[str]

materialize()

Materializes the data object, i.e., loads into memory all of the data that is still referenced in the HDF5 file.

Return type:

ArrayDict