search

HDFGroup / vol-rest

HDF5 REST VOL Connector
Other
5 stars 8 forks source link

H5Fget_name Require a File Object (API indicates differently) #14

Closed ron-kuhn closed 1 year ago

ron-kuhn commented 1 year ago

@jhendersonHDF or anyone else, From the documentation for H5Fget_name: hid_t obj_id - IN: Identifier of the object for which the associated file name is sought; may be a file, group, dataset, attribute, or named datatype

H5VLnative_file.c supports this but vol-rest does NOT.

Is there a workaround to get the file_id from any obj_id until this is supported?

ron-kuhn commented 1 year ago

@jreadey, do you have a workaround for the above problem?

jhendersonHDF commented 1 year ago

Hi @ron-kuhn, is it possible to check if https://github.com/HDFGroup/vol-rest/pull/17 fixes this for you? The alternative would have been to call H5Iget_file_id, but the connector doesn't support that call currently.

ron-kuhn commented 1 year ago

2 projects (h5vl_test and h5_test_testhdf5) are both giving unresolved externals for snprintf. Any ideas?

ron-kuhn commented 1 year ago

I support file based and s3 based hdf5 files. How can I check whether I am using native or a VOL?

mattjala commented 1 year ago

I support file based and s3 based hdf5 files. How can I check whether I am using native or a VOL?

Native will be used unless the environment variable HDF5_PLUGIN_PATH is set to a VOL installation, and the environment variable HDF5_VOL_CONNECTOR is set to the corresponding VOL name (in this case, REST).

ron-kuhn commented 1 year ago

@mattjala, I didn't explain good enough. I am not dynamically loading the VOL. I link in the rest vol library and used file create properties to support both file and HSDS based files. How can I tell if I am using the VOL or native (files) having access to a downstream Group object?

mattjala commented 1 year ago

If you've linked the library and included rest_vol_public.h, then it should be using the VOL. One way to check would be to try to read from a file on your server. You could also try calling a function which isn't yet supported in the REST VOL, such as H5Fget_fileno, and checking that it returns an error message saying it is unsupported.

ron-kuhn commented 1 year ago

@mattjala, I get my rest settings from the .hscfg (not the environment). The H5_rest_init fails now because I don't have the HSDS_ENDPOINT environment variable set and you aren't checking like it does in H5Pset_fapl_rest_vol. strncmp is called with NULL pointer.

ron-kuhn commented 1 year ago

@mattjala, I had to comment out the definitions for HDsnprintf and HDvsnprintf in \vol-rest\test\vol-tests\h5win32defs.h to get the VOL to build.

ron-kuhn commented 1 year ago

@mattjala, after making above changes, your fix for H5Fget_name worked. FYI... there are other H5F calls that do not require file_id but I am not using them currently.

ron-kuhn commented 1 year ago

@mattjala, your changes look good for what I requested.

I see that H5Dextend is not supported for rest vol. I want to store many images but don't necessarily know how many will be maximum. I will workaround for now, but do you know if it is planned in the near future?

mattjala commented 1 year ago

@mattjala, your changes look good for what I requested.

I see that H5Dextend is not supported for rest vol. I want to store many images but don't necessarily know how many will be maximum. I will workaround for now, but do you know if it is planned in the near future?

Yes, it is planned for implementation soon.