search

choderalab / fah-xchem

Tools and infrastructure for automated compound discovery using Folding@home
MIT License
6 stars 3 forks source link

Don't regenerate plots or snapshots that don't need to be updated #124

Closed jchodera closed 2 years ago

jchodera commented 3 years ago

It currently takes ~40 min to regenerate snapshots and ~40 min to generate plots for Sprint 5.

If we have received no new data for a RUN since the last analysis pass, we shouldn't need to regenerate a snapshot and can skip over it.

Perhaps we could record how much data we had processed in the last pass in the JSON file and note to skip over these steps for that RUN if this has not changed.

dotsdl commented 3 years ago

See #136 for a proposed solution to this.

dotsdl commented 3 years ago

@jchodera even with the changes from #136, do we still see very slow behavior in production when we hit fah_xchem.analysis.generate_representative_snapshots?

I want to verify that skipping generation of representative snapshots that are already present is insufficient performance-wise for us, which is what #136 should be doing for us.

dotsdl commented 2 years ago

Closing as resolved; please re-open if this issue persists are arises again.