search

dchaley / deepcell-imaging

Tools & guidance to scale DeepCell imaging on Google Cloud Batch
8 stars 2 forks source link

Change benchmark.py to new schema #268

Closed dchaley closed 2 months ago

dchaley commented 2 months ago

benchmark.py uploads to BQ using the old schema (assuming one instance runs everything).

The batch results schema was updated to have parameters per phase (preprocess, predict, postprocess).

We need to change the benchmark.py process to upload the new schema. Hopefully we have all the data we need.

If that's not possible, we need to bite the bullet and unify the implementations.

dchaley commented 2 months ago

It's possible, but pretty janky to do this in the benchmark.py script. We're starting to diverge substantially in terms of functionality between the multi-step versions, and the benchmark.py version.

The main(/only?) advantage of benchmark.py is that it doesn't push files to/from cloud storage in between steps. So, this change is likely to regress performance by introducing the intra-step copies.