Implement an SQLAlchemy based database to hold job and submission info

MikeDacre / fyrd

Submit functions and shell scripts to torque and slurm clusters or local machines using python.

https://fyrd.science

MIT License

22 stars 8 forks source link

Implement an SQLAlchemy based database to hold job and submission info #23

Open MikeDacre opened 8 years ago

MikeDacre commented 8 years ago

Needs:

Job table that will hold all submitted jobs
- Link to job_file table that will contain any submission files

Importantly:

Make Job and Queue classes inherit from an SQLAlchemy Base to make information persistent.

MikeDacre commented 7 years ago

Currently implemented for local jobs in 0.6.2, could potentially expand to all jobs. Would alter syntax to be a queue montor daemon, identical in operation to the current local queue but for all jobs.

This will be tricky in cases where there are multiple login nodes, as concurrent operation could allow the DB to be corrupted. The only way to get around that with SQLite would be to make the DB node-specific