Open DanielVoogsgerd opened 3 weeks ago
Good point!
brane data push
; it's not the scientist's job to make data available on the worker node simply because it requires updating the policy on the worker. Instead, we can make a branectl data push
(which is the worker-side tool, after all). Longer-term, I have in my mind to either have central host a package-like repository for data to which scientists can push (but necessarily isn't shielded by policy); or to make an optional flag for workers to allow "guest data".That aside, I agree it's a hassle. Something like branectl data push
is most definitely a very nice option (or perhaps branectl data build
, to mirror brane
's operation but for worker managers).
Moving data into a worker node is often quite a hassle. We have to copy data into the workers data directory, after which we have to change the absolute path to match the new location.
I suggest the following changes:
brane data push
orbrane data upload
needs to be implemented, which takes an asset. Which sends it to the worker node.@lut99, what do you think?