VizierDB / vizier-scala

The Vizier kernel-free notebook programming environment
Other
34 stars 11 forks source link

Optimize RunningWorkflow scheduler #222

Open okennedy opened 2 years ago

okennedy commented 2 years ago

Describe the bug As reported by @Nachiket18 , RunningWorkflow repeatedly calls Workflow.cellsAndModulesInOrder

Expected behavior This is not a bug, strictly, but calling this function instantiates a large number of objects, which leads to a considerable slowdown in the scheduler.

Ideas to Fix

  1. Have RunningWorkflow call a raw SQL query to avoid object construction overheads
  2. Have RunningWorkflow become the canonical source of truth about workflow execution (and make it responsible for mirroring that state back into the database). This would mitigate many of the new objects that need to be computed.