spacepy / dbprocessing

Automated processing controller for heliophysics data
5 stars 4 forks source link

Make ProcessQueue containerization-aware #132

Open jtniehof opened 9 months ago

jtniehof commented 9 months ago

ProcessQueue normally spawns processing codes on the same machine. It would be interesting to look into support for containerization; is there anything we can / should do to make it easier to handle containerized processing codes?

Relation to an issue

Related to #131 although they are probably not dependencies on each other (interesting to look at for possible ins/outs though).

Closure condition

It's hard to know the specifics before digging in to this, but this should be closed when it's clear how to launch a fully-containerized processing code and bring the results into the dbp database. This might be largely documentation or might involve a fair bit of code.