search

cubed-dev / cubed

Bounded-memory serverless distributed N-dimensional array processing
https://cubed-dev.github.io/cubed/
Apache License 2.0
116 stars 14 forks source link

Example using "dragon" start method #555

Open applio opened 1 month ago

applio commented 1 month ago

In discussions with Tom and Tom, it was unclear where this example involving the use of the "dragon" start method should land. This PR is intended to help further those discussions and ensure we do not lose track of this working example.

This PR is dependent upon PR #554 and is not intended to be merged independent of that PR.

tomwhite commented 1 month ago

I'm still not sure where the best place to add an example would be. The Dragon documentation would be one option (with a link from the Cubed documentation and examples). Or it could go here, but with the script named matmul-random-dragon.py or something so that it is standalone. We'd also want a link to how to set up Dragon.

@TomNicholas you've been trying out Dragon so perhaps you have thoughts?

tomwhite commented 1 month ago

Thinking about this more, it should be possible to add a dragon/cubed.yaml file under examples and then run

export CUBED_CONFIG=$(pwd)/dragon/cubed.yaml
python add-asarray.py

This would make it just like the other executors and allow any of the examples to be run on Dragon.