Princeton-LSI-ResearchComputing / tracebase

Mouse Metabolite Tracing Data Repository for the Rabinowitz Lab
MIT License
4 stars 1 forks source link

Add the ability to exclude/skip sheets in `load_study` #1352

Closed hepcat72 closed 2 days ago

hepcat72 commented 3 days ago

ISSUE SUBMITTER SECTION

Feature Inspiration

The load tests in the rabinowitz data repo (for 3 studies) are significantly slower than the rest. cold_exposure is the current winner at 2h20m to complete. (This is excluding the mzXML load.) This results in the action status page showing 4.75 "billable" hours, but regardless, the tests should be fast. It would all be MUCH faster if you could skip the load of the Peak Annotation Files sheet (i.e. the PeakAnnotationFilesLoader).

Feature Description

This minimum feature suggestion is to add an --exclude option that takes a series of enumerated values. We shouldn't use sheet names, because those can be customized and the value should be the same for all studies to work in the github action.

Note, this will mean that some problems might be overlooked, but that's already the case with mzXML files. However, it could be implemented such that the skipped sheet is still checked for simple things like missing required values, but that would be a separate issue. For this issue, just completely skip that loader.

Alternatives

An alternative would be to be able to specify the actual sheets to load instead of which sheets to skip. That could be a separate issue. The options could co-exist and be mutually exclusive.

Dependencies

None

Comment

None


ISSUE OWNER SECTION

Assumptions

None

Limitations

None

Affected Components

None provided

Requirements

None provided

DESIGN

Interface Change description

None provided

Code Change Description

None provided

Tests

None provided

hepcat72 commented 3 days ago

I marked this as critical so that the rabinowitz repo can be fully functional once-again and we're not incurring charges from github.