Screen Positives Count study for Small Molecule screens times out

[seanderickson]

• for study 200001 Reagent Counts for Small Molecule Screens
• It appears that this process has reached the runtime memory limits in the system, due to the recent increase in the number of stored results for Small Molecule screens.
• A suggested fix is to separate out the Natural Product screens to reduce the memory footprint, as a quick fix.
  • TODO: must verify that the Natural Product screens are distinct from the Small Molecule screens.
• longer term fix will be to reduce the runtime memory footprint of this process.

@JAS60

[dwrobel1]

There is no distinction between natural product and small molecule screens. Natural products are a subset of the small molecule compound libraries, and all screens that include natural products are small molecule screens. Thus a single small molecule screen can include natural product libraries and non-natural product libraries (commercial and known bioactives).

[seanderickson]

Yes, I was afraid of this - so there are screens that combine natural product and non-natural product reagents?

[dwrobel1]

I doubt we have any purely natural product screens, so any screen that tested natural products will also have screened non-natural product reagents.

[seanderickson]

Actually, this is still not a problem, because the studies are done on a per-well basis, so the study for small molecule screens (200004) could still be divided into two -

• 200004 Confirmed Positives for Small Molecule Screens - non-natural product\
• 200004 Confirmed Positives for Small Molecule Screens - natural product

\ @JAS60 - what would be a proper name for "Non-Natural Product" - that is, to use as a description/title in this study? In SS, the controlled vocabulary for library_types:

Current Library Types:

Academic Collection siRNA miRNA Inhibitor Known Bioactives Other Natural Products Discrete Commercial miRNA Mimic DOS Annotation Fragment NCI

[JAS60]

Hi Sean,

I would list them as:

200004 Confirmed Positives for Small Molecule Screens - Natural Product Collections

200004 Confirmed Positives for Small Molecule Screens - Known Bioactive, Academic and Commercial Collections

Thanks, Jen

---

Jennifer Smith, Ph.D.

Assistant Director ICCB-Longwood Screening Facility Harvard Medical School 250 Longwood Ave., SGM 604 Boston, MA 02115

jennifer_smith@hms.harvard.edumailto:jennifer_smith@hms.harvard.edu

617.432.5735

http://iccb.med.harvard.edu

On Jul 31, 2015, at 2:17 PM, Sean Erickson wrote:

Actually, this is still not a problem, because the studies are done on a per-well basis, so the study for small molecule screens (200004) could still be divided into two -

• 200004 Confirmed Positives for Small Molecule Screens - non-natural product**
• 200004 Confirmed Positives for Small Molecule Screens - natural product

\ @JAS60https://github.com/JAS60 - what would be a proper name for "Non-Natural Product" - that is, to use as a description/title in this study? In SS, the controlled vocabulary for library_types:

Current Library Types:

Academic Collection siRNA miRNA Inhibitor Known Bioactives Other Natural Products Discrete Commercial miRNA Mimic DOS Annotation Fragment NCI

— Reply to this email directly or view it on GitHubhttps://github.com/hmsiccbl/screensaver/issues/166#issuecomment-126775242.

[seanderickson]

I've decided to fix this by bypassing the ORM/Hibernate layer in Screensaver.

• This study already relies on non-ORM code, see "populateStudyReagentLink" method
• loading a large portion of the reagent table into memory will always be a source of potential memory issues.
• execution time speed up from ~3 hrs to ~3 min.
• sql method should be performant for forseable future, capable of handling growth in the database size.