is it possible to upload bytes object instead files?

allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

https://clear.ml/docs

Apache License 2.0

5.67k stars 653 forks source link

is it possible to upload bytes object instead files? #1304

Open S0NNER10N opened 3 months ago

S0NNER10N commented 3 months ago

Proposal Summary

Add option to upload not from existing (local, s3, azure, etc) files but directly bytes-objects

Motivation

Subset of datasets presented as inmemory bytes data and handler to upload it without saving tmp file would be great

eugen-ajechiloae-clearml commented 3 months ago

Hi @S0NNER10N ! Indeed, we currently don't expose an easy way to upload objects directly. We will add this to our development roadmap.