search

PaulBrack / Yamato

SWATH-QC metrics
Apache License 2.0
1 stars 1 forks source link

RC1 - Can't load file #127

Closed mwalzer closed 4 years ago

mwalzer commented 4 years ago

Congrats on the new RC! Did some container building and tests. With RC1 it seems I cannot load a file to process. Here is the output

Singularity> pwd
/tmp
Singularity> ls -l
total 592168
-rw-rw-r-- 1 walzer walzer 584235503 Mar  2 10:09  collinsb_I180316_005_SW-A.mzML
-rw-rw-r-- 1 walzer walzer  21074890 Mar  2 10:08  split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML
Singularity> SwaMe.Console -i split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML  
2020-03-02 10:15:57.6715|INFO|Yamato.Console.Program|Loading file: split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML
2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|Unable to open the file: split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML.
2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|An unexpected error occured:
2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|Could not find file '/tmp/QC_Results/split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp/2020-03-02_10-15-57/split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML'.
2020-03-02 10:15:57.7049|ERROR|Yamato.Console.Program|   at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode)
   at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options)
   at System.IO.FileStream..ctor(String path, FileMode mode)
   at Yamato.Console.Program.CheckFileIsReadableOrComplain(String inputFilePath) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 142
   at Yamato.Console.Program.<>c.<Main>b__1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 74
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

You'll notice, It tries to access a folder not given as parameter. After that, the folder(s) exists.

Singularity> ls -l
total 592168
drwxr-xr-x 3 walzer walzer      4096 Mar  2 10:15  QC_Results
-rw-rw-r-- 1 walzer walzer 584235503 Mar  2 10:09  collinsb_I180316_005_SW-A.mzML
-rw-rw-r-- 1 walzer walzer  21074890 Mar  2 10:08  split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML
mwalzer commented 4 years ago

hotfix copy the file to QC_results doesn't work either because the subfolder seems to be a timestamp folder generated at exec time.

marinaPauw commented 4 years ago

Hi Mathias!

Paul will possibly give a better answer when his union returns from strike in two weeks' time, but I believe the problem to be that unlike the proteowizard tools for example, the "-i" argument requires the full inputfile path and does not take the working directory into account.

The QC_Results directory error is only third on the list and may have occurred because the analysis was never started because the file was not found.

Could you try it with the full path?

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Mar 2, 2020 at 12:23 PM Mathias Walzer notifications@github.com wrote:

Congrats on the new RC! Did some container building and tests. With RC1 https://github.com/PaulBrack/Yamato/tree/v1.1.0-RC1 it seems I cannot load a file to process. Here is the output

Singularity> pwd /tmp Singularity> ls -l total 592168 -rw-rw-r-- 1 walzer walzer 584235503 Mar 2 10:09 collinsb_I180316_005_SW-A.mzML -rw-rw-r-- 1 walzer walzer 21074890 Mar 2 10:08 split_napedro_L120420_010_SW-400AQUAhuman_2ul_dilution_1_1.nf.pp.mzML Singularity> SwaMe.Console -i split_napedro_L120420_010_SW-400AQUAhuman_2ul_dilution_1_1.nf.pp.mzML 2020-03-02 10:15:57.6715|INFO|Yamato.Console.Program|Loading file: split_napedro_L120420_010_SW-400AQUAhuman_2ul_dilution_1_1.nf.pp.mzML 2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|Unable to open the file: split_napedro_L120420_010_SW-400AQUAhuman_2ul_dilution_1_1.nf.pp.mzML. 2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|An unexpected error occured: 2020-03-02 10:15:57.6964|ERROR|Yamato.Console.Program|Could not find file '/tmp/QC_Results/split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp/2020-03-02_10-15-57/split_napedro_L120420_010_SW-400AQUAhuman_2ul_dilution_1_1.nf.pp.mzML'. 2020-03-02 10:15:57.7049|ERROR|Yamato.Console.Program| at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter) at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode) at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) at System.IO.FileStream..ctor(String path, FileMode mode) at Yamato.Console.Program.CheckFileIsReadableOrComplain(String inputFilePath) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 142 at Yamato.Console.Program.<>c.

b1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 74 at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action1 action) at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

You'll notice, It tries to access a folder not given as parameter. After that, the folder(s) exists.

Singularity> ls -l total 592168 drwxr-xr-x 3 walzer walzer 4096 Mar 2 10:15 QC_Results -rw-rw-r-- 1 walzer walzer 584235503 Mar 2 10:09 collinsb_I180316_005_SW-A.mzML -rw-rw-r-- 1 walzer walzer 21074890 Mar 2 10:08 split_napedro_L120420_010_SW-400AQUA__human_2ul_dilution_1_1.nf.pp.mzML

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/PaulBrack/Yamato/issues/127?email_source=notifications&email_token=AKP4IQ3N5JX7RETMSYR2AF3RFOCIPA5CNFSM4K7REB62YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IRUR5TQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKP4IQZUVXWZ4PCIS6VITODRFOCIPANCNFSM4K7REB6Q .

mwalzer commented 4 years ago

Thanks, indeed, now it's

... maaaaany lines of |INFO|MzmlParser.MzmlReader| ...
2020-03-02 13:16:23.5529|INFO|MzmlParser.MzmlReader|Finding base peak spectra...
2020-03-02 13:16:31.6254|INFO|MzmlParser.MzmlReader|Selecting best IRT peptide candidates...
2020-03-02 13:16:31.6279|INFO|Yamato.Console.Program|Generating metrics...
2020-03-02 13:16:37.7924|ERROR|Yamato.Console.Program|An unexpected error occured:
2020-03-02 13:16:37.7924|ERROR|Yamato.Console.Program|Index was out of range. Must be non-negative and less than the size of the collection.
Parameter name: index
2020-03-02 13:16:37.8180|ERROR|Yamato.Console.Program|   at System.Collections.Generic.List`1.get_Item(Int32 index)
   at SwaMe.RTGrouper.DivideByRT(Run run, Int32 division, Double rtDuration) in /home/runner/work/Yamato/Yamato/SwaMe/RTGrouper.cs:line 179
   at SwaMe.MetricGenerator.GenerateMetrics(Run run, Int32 division, String inputFilePath, Boolean irt, Boolean combine, Boolean lastFile, String date) in /home/runner/work/Yamato/Yamato/SwaMe/MetricGenerator.cs:line 60
   at Yamato.Console.Program.<>c.<Main>b__1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 106
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

or with a different file 

2020-03-02 13:20:57.6256|ERROR|Yamato.Console.Program|An unexpected error occured:
2020-03-02 13:20:57.6256|ERROR|Yamato.Console.Program|Sequence contains no matching element
2020-03-02 13:20:57.6366|ERROR|Yamato.Console.Program|   at System.Linq.Enumerable.Single[TSource](IEnumerable`1 source, Func`2 predicate)
   at MzmlParser.MzmlReader.ReadSpectrum(XmlReader reader, Run run, Boolean irt) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 169
   at MzmlParser.MzmlReader.ReadMzml(String path, Run run, Boolean irt) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 106
   at MzmlParser.MzmlReader.LoadMzml(String path, AnalysisSettings analysisSettings) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 52
   at Yamato.Console.Program.<>c.<Main>b__1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 94
   at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult`1 result, Action`1 action)
   at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

are there other parameters assumed to be set that I haven't? (just -i)

marinaPauw commented 4 years ago

Your parameters were correct, only inputfile (-i) is required. The two errors above appear unrelated. I'll do some sleuthing and get back to you.

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Mar 2, 2020 at 3:24 PM Mathias Walzer notifications@github.com wrote:

Thanks, indeed, now it's

... maaaaany lines of |INFO|MzmlParser.MzmlReader| ... 2020-03-02 13:16:23.5529|INFO|MzmlParser.MzmlReader|Finding base peak spectra... 2020-03-02 13:16:31.6254|INFO|MzmlParser.MzmlReader|Selecting best IRT peptide candidates... 2020-03-02 13:16:31.6279|INFO|Yamato.Console.Program|Generating metrics... 2020-03-02 13:16:37.7924|ERROR|Yamato.Console.Program|An unexpected error occured: 2020-03-02 13:16:37.7924|ERROR|Yamato.Console.Program|Index was out of range. Must be non-negative and less than the size of the collection. Parameter name: index 2020-03-02 13:16:37.8180|ERROR|Yamato.Console.Program| at System.Collections.Generic.List1.get_Item(Int32 index) at SwaMe.RTGrouper.DivideByRT(Run run, Int32 division, Double rtDuration) in /home/runner/work/Yamato/Yamato/SwaMe/RTGrouper.cs:line 179 at SwaMe.MetricGenerator.GenerateMetrics(Run run, Int32 division, String inputFilePath, Boolean irt, Boolean combine, Boolean lastFile, String date) in /home/runner/work/Yamato/Yamato/SwaMe/MetricGenerator.cs:line 60 at Yamato.Console.Program.<>c.<Main>b__1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 106 at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action`1 action) at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

or with a different file

2020-03-02 13:20:57.6256|ERROR|Yamato.Console.Program|An unexpected error occured: 2020-03-02 13:20:57.6256|ERROR|Yamato.Console.Program|Sequence contains no matching element 2020-03-02 13:20:57.6366|ERROR|Yamato.Console.Program| at System.Linq.Enumerable.Single[TSource](IEnumerable1 source, Func2 predicate) at MzmlParser.MzmlReader.ReadSpectrum(XmlReader reader, Run run, Boolean irt) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 169 at MzmlParser.MzmlReader.ReadMzml(String path, Run run, Boolean irt) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 106 at MzmlParser.MzmlReader.LoadMzml(String path, AnalysisSettings analysisSettings) in /home/runner/work/Yamato/Yamato/MzmlParser/MzmlReader.cs:line 52 at Yamato.Console.Program.<>c.

b__1_0(Options options) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 94 at CommandLine.ParserResultExtensions.WithParsed[T](ParserResult1 result, Action1 action) at Yamato.Console.Program.Main(String[] args) in /home/runner/work/Yamato/Yamato/SwaMe.Console/Program.cs:line 22

are there other parameters assumed to be set that I haven't? (just -i)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PaulBrack/Yamato/issues/127?email_source=notifications&email_token=AKP4IQYDFMWOZHCNXJX7ZP3RFOXPXA5CNFSM4K7REB62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPI6VA#issuecomment-593399636, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKP4IQ2RMM3DUHGBJUJJE2TRFOXPXANCNFSM4K7REB6Q .

mwalzer commented 4 years ago

Thanks! I've seen from the console output that there is some iRT peptide peak detection going on. As a feature-request of sorts, it might be a very cool thing to have a metric that measures the confidence that some well-known(?) iRT peptides and/or kits were used. Is that possible, or even done yet? (Haven't produced an output file yet, to check.)

marinaPauw commented 4 years ago

Regarding the second file mentioned above: I believe this may be an incorrect assumption of ours that a wiff file will always have an "id" component in the mzML. If I'm right it will be easily fixed, but I will open a separate issue for that one.

Thanks

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Mar 2, 2020 at 4:14 PM Mathias Walzer notifications@github.com wrote:

Thanks! I've seen from the console output that there is some iRT peptide peak detection going on. As a feature-request of sorts, it might be a very cool thing to have a metric that measures the confidence that some well-known(?) iRT peptides and/or kits were used. Is that possible, or even done yet? (Haven't produced an output file yet, to check.)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PaulBrack/Yamato/issues/127?email_source=notifications&email_token=AKP4IQ6OXBOLADDNVMM2QG3RFO5K5A5CNFSM4K7REB62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPONJI#issuecomment-593421989, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKP4IQ5FJSZKFBYSUKC4XS3RFO5K5ANCNFSM4K7REB6Q .

marinaPauw commented 4 years ago

Hi Matthias!

Could you maybe try running the second file again with the updated master branch?

On Mon, Mar 2, 2020 at 4:44 PM Marina Pauw pauwmarina@gmail.com wrote:

Regarding the second file mentioned above: I believe this may be an incorrect assumption of ours that a wiff file will always have an "id" component in the mzML. If I'm right it will be easily fixed, but I will open a separate issue for that one.

Thanks

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail <#m_-5973503875507336382_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Mar 2, 2020 at 4:14 PM Mathias Walzer notifications@github.com wrote:

Thanks! I've seen from the console output that there is some iRT peptide peak detection going on. As a feature-request of sorts, it might be a very cool thing to have a metric that measures the confidence that some well-known(?) iRT peptides and/or kits were used. Is that possible, or even done yet? (Haven't produced an output file yet, to check.)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PaulBrack/Yamato/issues/127?email_source=notifications&email_token=AKP4IQ6OXBOLADDNVMM2QG3RFO5K5A5CNFSM4K7REB62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPONJI#issuecomment-593421989, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKP4IQ5FJSZKFBYSUKC4XS3RFO5K5ANCNFSM4K7REB6Q .

mwalzer commented 4 years ago

tested, resolved, all well with maintenance release v1.0.4