Project Log (First stage)

[swiri021]

• Main Dashboard Brainstorming

  • Pipeline group status (# of Running, # of Failed, # of Completed - Chart)
  • Latest pipeline status
  • Number of pipelines
  • Spec (?), notification for overflow pipeline (Optional)

• Functional list (Pipeline = Nextflow)

  • Login process
  • Pipeline installation process
  • Pipeline execution
  • Pipeline monitor, Pipeline log managing(optional)
    • Basic Status (RUNNING, COMPLETED, FAILED, UNKNOWN, PENDING) (Primary goal),
    • Background status checking (Celery ?, Redis + RabbiMQ)
    • Log streaming (Not realtime, Updating with regular interval(30s, 1m)) (Secondary goal)
  • Pipeline parameters management by profile, Profile config file loading (json, yaml)
  • report.html, time.html viewer
  • Nextflow config parsing (cpus, mem, hour) and checking with current local capacity

• Essential background

  • Nextflow (Snakemake)
  • SQL
  • Database
  • Sqlite3

[swiri021]

Engineering stack: VSCode, Conda, Docker, Nextflow, Flask, Sqlite, python3.9, Celery, Localstack

[duaghk]

Engineering stack: Currently: VSCode, Singularity, Django, SQLite, MariaDB, Snakemake, Python 3.9~3.12, Not currently: Conda, Docker, AWS batch, AWS step-functions