search

CrayLabs / SmartRedis

SmartSim Infrastructure Library Clients.
BSD 2-Clause "Simplified" License
54 stars 25 forks source link

Merge put_bytes into develop #519

Open rickybalin opened 3 weeks ago

rickybalin commented 3 weeks ago

Description

Some time ago you added a feature to SmartRedis to add a byte string array to the Orchestrator. This feature was added in a branch called put_bytes. Would it be possible to merge this feature into the develop branch?

Justification

This feature is very useful for sharing AI/ML model checkpoints between workflow components across the Orchestrator. Models can be serialized, with something like torch.jit.trace, and then stored in the Orchestrator to then be loaded and run from various components. This is helpful when models are not supported by RedisAI, such as GNNs.

mellis13 commented 3 weeks ago

Thanks for posting this, @rickybalin -- we can definitely do that. Do you need an initial implementation in all clients or just a subset?

rickybalin commented 3 weeks ago

Just Python for the current application, but likely all clients will be useful (maybe with a priority on the C++ client over C and Fortran). Thanks!

mellis13 commented 3 weeks ago

C++ and Python APIs have been merged into develop. We will follow-up with C and Fortran interfaces later. Please let us know if you have any feedback/issues with it!

rickybalin commented 3 weeks ago

Thanks, no issues so far with the Python API.