search

stephlj / slopey

Quantification of single-molecule FRET trajectories containing fast, non-instantaneous transitions
1 stars 0 forks source link

Handling discarded traces: #12

Closed stephlj closed 8 years ago

stephlj commented 8 years ago

If I run Make for the first time in a new data directory, and some *Results.params.yml files have discard: true, python ignores them. that's fine.

However if I decide to discard a trace later, after Make has already generated the .pkl file etc for that trace, python ignores the corresponding .mat file ... but the .pkl etc still exist, so the discarded trace still ends up in all_results.mat. Also, because the python code ignores it, the params in all_results.mat don't get updated, so the params field for that trace still has discard: false (or no discard field).

What's the best way to fix this?

stephlj commented 8 years ago

Luke had a suggestion: instead of FILES = $(wildcard *Results.mat), write a shell script that will go through the params.yml files and return a list of only the files that don't have discard=true. Then FILES = output of that shell script, and make only knows about the files that it actually should be doing something with.

Also, is it possible to make an additional clean command, like "clean_discards", that will remove any make targets generate for files that are now discards?

stephlj commented 8 years ago

We basically implemented Luke's suggestion; need to run the analysis from scratch to make sure it works. See commit 99d21e0.

Luke wants to know why it breaks the make file to put list_discards.py in scripts instead of in the main folder?

stephlj commented 8 years ago

Debugged, see commit b727316. But see Luke's question: "Luke wants to know why it breaks the make file to put list_discards.py in scripts instead of in the main folder?"