search

HDFGroup / vol-rest

HDF5 REST VOL Connector
Other
5 stars 8 forks source link

Optimize RV_find_object_by_path #32

Closed mattjala closed 1 year ago

mattjala commented 1 year ago

Reduced the number of requests to HSDS made during calls to RV_find_object_by_path.

Most calls now provide the type of object they expect, and use only a single external request.

If the type of the final object is unknown at the time the function is called, then the type is figured out in a single request (previously, this required two calls).

Does not yet handle soft links to other soft links.

This depends on the changes to HSDS in HDFGroup/HSDS#219 and HDFGroup/HSDS#224

After this change, the icesat2 benchmark completes about 25% faster (5 seconds -> 4 seconds), see HDFGroup/nasa_cloud#4

mattjala commented 1 year ago

Using the recurse_until_hard query option added in https://github.com/HDFGroup/hsds/pull/224, RV_find_object_by_path can now find its target object in a single request, regardless of whether or not its type is known, or whether the known path is to a symbolic link rather than a hard one.

To get a metric for how much this reduces overhead on top of the previous change, the icesat2 benchmark time went from ~4 seconds to ~3.7 seconds.