This code enables running a collection of tasks in paralell. Initially the code runs locally using
multiprocessing. Remote targets could be added in the future.
- Integration with our theme.
- Uses fork() as part of Multiprocess().
- Uses rich for screen management.
- Tasks can have dependencies.
(v-rich) sdake@wise-a40x2:~/repos/origin/mouseion/mouseion$ python tasks.py
Artificial Wisdom™ NLP Tools • RETRO Tokenizer
• Load jsonl • ━━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 0:00:00 • 0:00:03 • 10% • •
• Tokenize • ━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 0:00:00 • 0:00:06 • 5% • •