higher rank datasets

[rimmartin]

Looking to make the setting of rank and dimensions generic so users can go to any rank and define all dimensions.

Adding for rank 4 continuing current style of rows, columns, sections and now files http://io9.gizmodo.com/5823271/the-many-dimensions-of-the-tesseract. Pillars might be another name for sections.

In the end may add an array called dimensions but was also looking to do it without properties which could conflict with user property names; have the usual javascript properties in braces

h5lt.makeDataset(id, name, buffer , {dimensions: [2,2,2,2]});

for example

[rimmartin]

Got rank 4 basically working and tested. But am forgetting why the order of rank names changes for the rank 3 and then subsequently rank 4; at https://github.com/HDF-NI/hdf5.node/blob/master/src/h5_lt.hpp#L1119

Is there an hdf5 convention? Are there other defined patterns? That the user will want to control?

[rimmartin]

Also H5Screate_simple doesn't like 4 dimensions with H5S_UNLIMITED passed to it even for small dimensions ; https://github.com/HDF-NI/hdf5.node/blob/master/src/h5_lt.hpp#L36

I set to NULL and it works although the docs talk about chunking requirement needs H5S_UNLIMITED

Committing where I'm at with the order of rows, columns, sections, files as 2, 3, 1, 0 in the dimensions array both in making and reading.

[sstathatos]

Hello there! Not sure if i am in the same subject, i was trying to use the 'Rank 2 Subsets' example from the documentation, but changed it a bit to use rank=3 instead of 2. When i change it to 2 again it works, so i think this causes the error. I am getting this response: HDF5-DIAG: Error detected in HDF5 (1.10.0-patch1) thread 0:

000: H5Pdcpl.c line 2030 in H5Pset_chunk(): all chunk dimensions must be positive

major: Invalid arguments to routine
minor: Out of range

SyntaxError: Failed to set chunked layout Any ideas why? Thanks in advance!

[rimmartin]

as good a place as any. 1 & 2 ranks have been tested. I'll add a test for 3 rank; I got need for it with data from http://pointclouds.org/ computations

[rimmartin]

I think it has to do with the order of rows, columns and sections. Being made 1, 2, 0 so the start, stride an count need that order.

Added a test at https://github.com/HDF-NI/hdf5.node/blob/master/test/test_h5lt.js#L614

Also chunking might be effected by the H5Screate_simple above with higher rank 4

[rimmartin]

[sections, rows, columns] I believe has to do with efficiency

[sstathatos]

You are right, i was actually using wrong name for sections, didn't know i should use that. I changed it and it works, although the returned typedArray confuses and changes the values of the dimensions. getDatasetDimensions returns the correct values, but inside the typedArray (typedArray.sections, rows etc), some values are wrong. I tried changing order of rows columns etc but nothing changed.The array itself though is correct!

[rimmartin]

Yea but what should the order be as rank increases? What is the consensus of users or will it need to be a choice by users...

I'll study typed array vs Buffer to make them match TypedArray is one long array and the rows, columns, sections are how to interpret to a rectangle or pillar.