[OpenMS] Add versioned deployment for OpenMS Workflow

mwang87 commented 5 years ago

Current Status: Merged code in and deployed to proteomics2 and testing

[x] Adding tools
[x] updating workflow xml to be compatible
[x] Version
[ ] Testing
[x] Deployment to GNPS
[ ] Testing on GNPS

The test workflows will be:

Small Large Large 2

mwang87 commented 5 years ago

@abisarvepalli Any ideas?

Failures:

https://proteomics3.ucsd.edu/ProteoSAFe/status.jsp?task=30880cb91f624271bdb491a4f49d006a https://proteomics3.ucsd.edu/ProteoSAFe/status.jsp?task=a500055a171b48c39d85e04c01b19dbc https://proteomics3.ucsd.edu/ProteoSAFe/status.jsp?task=f23deedcb24f4fa89dff3ef0ccada37b

abipalli commented 5 years ago

Will take a look at it today! On Sep 9, 2019, 9:18 PM -0400, Ming Wang notifications@github.com, wrote:

@abisarvepalli Any ideas? Failures: https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=30880cb91f624271bdb491a4f49d006a https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=a500055a171b48c39d85e04c01b19dbc https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=f23deedcb24f4fa89dff3ef0ccada37b — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

lfnothias commented 5 years ago

Hi @abisarvepalli , there is indeed an issue with the python wrapper. https://gnps.ucsd.edu/ProteoSAFe/status.jsp?task=a000a574dc9e4eec9b5fc64ab36e836f

lfnothias commented 5 years ago

The workflow works great 👍. Currently, we are running into scalability, probably due to the RAM allocated by default. These are various test jobs:

Instrument	Size	Aligner tool	GNPSExport mode	Job URL
QTOF	100 samples	FeatureAlignerUnlabeledQT	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=d374e74c0f5041798bd9fe52b32861f7
QTOF	1K samples	FeatureAlignerUnlabeledQT	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=91d1ab2c55ee4d37afe467350c3983b7
QTOF	2K samples	FeatureAlignerUnlabeledQT	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=62dd9af9ceb04114a6fdbcd9114be299
QExactive	6 samples	FeatureAlignerUnlabeledQT	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=fac8bd0f56b94fd984bac7c9251ba797
QExactive	800 samples	FeatureAlignerUnlabeledQT	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=7eb5ff62a80e4fdcadee313b1a32fdfb
QExactive	6 samples	FeatureAlignerUnlabeledQT	Merged spectra	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=cd98ec2900d64e5bb478e98cfa08770a
QExactive	6 samples	FeatureAlignerUnlabeledKD	Most intense	https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=5c1de3bccd8443eaae532cebb8907ba0

abipalli commented 5 years ago

Identifying determinism of MapAlignerPoseClustering +/or MetaboliteAdductDechargerSteps by cloning QTOF 1K job 10 times:

lfnothias commented 5 years ago

Like before, all jobs failed at the MapAlignerPoseClustering. I tested the parameters from Optimus that were used to scale up to 2K. https://proteomics2.ucsd.edu/ProteoSAFe/status.jsp?task=b74a82124fc34f7a98fd08c88ddb0a8d

Unfortunately, I got the same issue as before for the 1K samples jobs. @abisarvepalli @mwang87 can we test with more RAM to see if that helps.

mwang87 commented 5 years ago

This has already been given 20GB of RAM, and the error is not an error indicating a lack of ram.

lfnothias commented 5 years ago

@abisarvepalli can you open an issue on the OpenMS repo ?

mwang87 commented 4 years ago

@abisarvepalli Is the OpenMS workflow ready for merge into master and deployment into GNPS?

CCMS-UCSD / GNPS_Workflows

[OpenMS] Add versioned deployment for OpenMS Workflow #106