ProteoWizard / pwiz

The ProteoWizard Library is a set of software libraries and tools for rapid development of mass spectrometry and proteomic data analysis software.
http://proteowizard.sourceforge.net/
Apache License 2.0
216 stars 99 forks source link

Filtering ITMS/FTMS from Lumos data files losing all scan data #144

Closed chrashwood closed 6 years ago

chrashwood commented 6 years ago

Dear Proteowizard developers,

I hope you are well. I'm currently having some trouble with msconvert regarding filtering of a Thermo Lumos raw data file containing multiple mass analyzer steps. I am using ProteoWizard 3.0.8725 x64. Direct and non-filtered conversion from raw to mzML, then subsequent import into Skyline works fine and the filter string names are preserved in the unfiltered mzML file.

Curiously however, if implementing filtering for only FTMS (--filter "analyzer FT" OR --filter "analyzer orbi" OR "--filter "analyzerType FTMS"), all scans are lost with only the chromatogram/metadata preserved. I have provided a copy of the command line process run below:

`W:\1_RAW_Data\4_Lumos>> msconvert Lumos_data.raw --filter "analyzer FT" format: mzML m/z: Compression-None, 64-bit intensity: Compression-None, 32-bit rt: Compression-None, 64-bit ByteOrder_LittleEndian indexed="true" outputPath: . extension: .mzML contactFilename:

filters: analyzer FT

filenames: Lumos_data.raw

processing file: Lumos_data.raw writing output file: .\Lumos_data.mzML `

This same process works fine with my Velos Orbitrap data so I don't believe my commands to be the issue.

Cheers, Chris

chambm commented 6 years ago

What are your <instrumentConfiguration> sections in the resulting mzML?

chrashwood commented 6 years ago

Thanks for the quick reply.

Comparing the filtered vs non-filtered mzml conversion: run id spectrumList is missing run id chromatogramList is the same.

index name="spectrum" is different (no details present for this section and no offset values) index name="chromatogram" is different (TIC intensity is 0.001% of the non-filtered mzML file)

indexedmzML, mzML, cvList, fileDescription, referenceableParamGroupList, softwareList, instrumentConfigurationList and dataProcessingList are the same.

On Mon, Jul 30, 2018 at 1:14 PM, Matt Chambers notifications@github.com wrote:

What are your sections in the resulting mzML?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ProteoWizard/pwiz/issues/144#issuecomment-408959329, or mute the thread https://github.com/notifications/unsubscribe-auth/Aca4SK1RGjbFD68LhdCphmdK5pKXBK1sks5uL0z5gaJpZM4VkqW0 .

chambm commented 6 years ago

I mean what are the contents in the <instrumentConfigurationList>

chrashwood commented 6 years ago

Sorry for the misinterpretation:

  <instrumentConfiguration id="IC2">
    <referenceableParamGroupRef ref="CommonInstrumentParams"/>
    <componentList count="4">
      <source order="1">
        <cvParam cvRef="MS" accession="MS:1000398"

name="nanoelectrospray" value=""/>

      <analyzer order="3">
        <cvParam cvRef="MS" accession="MS:1000083" name="radial

ejection linear ion trap" value=""/>

    </componentList>
    <softwareRef ref="Xcalibur"/>
  </instrumentConfiguration>
</instrumentConfigurationList>

On Tue, Jul 31, 2018 at 12:33 PM, Matt Chambers notifications@github.com wrote:

I mean what are the contents in the

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ProteoWizard/pwiz/issues/144#issuecomment-409304102, or mute the thread https://github.com/notifications/unsubscribe-auth/Aca4SJ8eQeaUbhFaTIWppfNXS3-m5QS6ks5uMJTbgaJpZM4VkqW0 .

chambm commented 6 years ago

Hmm, are you sure that your data contains FT scans? The scan filters say FTMS? Can you send a RAW file?

chrashwood commented 6 years ago

100% sure. It's a triggered neutral loss experiment. I've compressed the raw file so it's a 400MB zip archive. Do you have a preferred method to share the data? It's not a sensitive method/sample so can be public.

chambm commented 6 years ago

This will work for non-sensitive data: https://skyline.ms/filecontent/home/support/file%20sharing/begin.view?

chrashwood commented 6 years ago

Thanks. It's uploaded and available now.

chambm commented 6 years ago

I can reproduce the problem and have figured out the cause, I just haven't gotten to fix it yet. (the filter only checks the first analyzer, when it really should check the last, or all)

chrashwood commented 6 years ago

That's great to hear, thanks Matt! Is this problem specific for msconvert or are fixes required for the entire Proteowizard suite (I'm particularly interested in msaccess)

chambm commented 6 years ago

Should be fixed in #204. This problem would happen anywhere --filter is used.

jspaezp commented 3 years ago

Sorry for resurrecting this issue but I am having the same problem using the msconvert in ProteoWizard 3.0.18215.f6864290f

I am using this file from PRIDE Pride repo: PXD004732 Test File: 01625b_GB2-TUM_first_pool_10_01_01-2xIT_2xHCD-1h-R1.raw

MSCONVERT="/c/Program Files/ProteoWizard/ProteoWizard 3.0.18215.f6864290f/msconvert.exe"
TESTFILE="01625b_GB2-TUM_first_pool_10_01_01-2xIT_2xHCD-1h-R1.raw"
count_filters() {
    grep -P "filter" ./out/*.mzML | grep -oP "value=\".*?ms(\d|\s)" | sort | uniq -c 
}

"${MSCONVERT}" ${TESTFILE} --filter "peakPicking true 1-" -o out/
count_filters
#   28830 value="FTMS + c NSI d Full ms2
#    3227 value="FTMS + p NSI Full ms
#   28830 value="ITMS + c NSI r d Full ms2

"${MSCONVERT}" ${TESTFILE} --filter "analyzer orbi" --filter "peakPicking true 1-" -o out/
count_filters
# EMPTY

"${MSCONVERT}" ${TESTFILE} --filter  "peakPicking true 1-" --filter "analyzer orbi" -o out/
count_filters
# EMPTY

"${MSCONVERT}" ${TESTFILE} --filter "analyzer orbi" -o out/
count_filters
# EMPTY

"${MSCONVERT}" ${TESTFILE} --filter "analyzer FT" -o out/
count_filters
# EMPTY

"${MSCONVERT}" ${TESTFILE} --filter "analyzer FTMS" -o out/
count_filters
# EMPTY

Is there anything I am doing wrong?

Kindest wishes, Sebastian

chambm commented 3 years ago

Try a newer version of pwiz. Even though this issue was fixed in 2018, it seems your version manages to predate it.

jspaezp commented 3 years ago

Noticed the error was due to the path not using msconvert from the actual updated installation (so I guess the issue is that updating pwiz does not remove older versions, which in itself could be a design choice...)

Thanks for the reply.