Right now, it's straightforward to recognize experiments that have already been run: we just check for the output file, and if it exists we skip it.
However, recognizing experiments that were skipped before (e.g. because of too few positive labels, etc) is slower, as we have to preprocess the data again to arrive at that conclusion. It would be faster to create some sort of file for each gene showing which experiments were skipped, then on rerun we can load the file (if it exists) and know which experiments to skip again, much faster.
I don't think this will improve runtime that much, but it should help a bit.
Right now, it's straightforward to recognize experiments that have already been run: we just check for the output file, and if it exists we skip it.
However, recognizing experiments that were skipped before (e.g. because of too few positive labels, etc) is slower, as we have to preprocess the data again to arrive at that conclusion. It would be faster to create some sort of file for each gene showing which experiments were skipped, then on rerun we can load the file (if it exists) and know which experiments to skip again, much faster.
I don't think this will improve runtime that much, but it should help a bit.