terascope / teraslice

Scalable data processing pipelines in JavaScript
https://terascope.github.io/teraslice/
Apache License 2.0
50 stars 13 forks source link

Add earl jobs export that creates tjm file #3695

Open godber opened 1 month ago

godber commented 1 month ago

This earl jobs ts-tmp1 <JOBID> to a file.json that is compatible with tjm.

busma13 commented 1 month ago

This seems similar to earl jobs save. We could default to saving in the .teraslice directory like it does and provide the same options if they are applicable.

busma13 commented 4 weeks ago

If we want this command to be able to export multiple jobs, then it would be nice to have the file names come from some default, so the user doesn't have to provide a name for every file. This could simply be the name property from the job config. It could possibly be helpful to include the cluster alias and the date in the file name.

If we only want to support exporting one job file at a time we can simply pass the name we want as an argument to the command.

busma13 commented 4 weeks ago

Current setup:

➜ teraslice git:(earl-tjm-export) ✗ ls ./config/test test382b.json test9988.json

busma13 commented 4 weeks ago

TBD: