Open jsaintvanne opened 1 year ago
Hello,
Yes the error is due to SIRIUS4 and the webservices linked to SIRIUS4 are not supported anymore. We are currently working on adding the latest SIRIUS version (in a non-interactive way), however, the SIRIUS5 integration might take some time.
Could you let me know if you are using MAW to run your dataset and how big the size of your whole dataset (all .mzML files)? I can help with using the workflow interactively with SIRIUS5.
Mahnoor
Hello,
Thanks for your answer it is most clear for me now why it doesn't work.
We have to test on 4 DDA pos files :
And 4 DDA neg files :
It is few things but it is just to test your tool to see what it is able to do
Should you tell me how you think to do ? First part in R, then extract to run Sirius GUI then again R part for the post Sirius steps ? Thanks !
Alright. Here are few points that you can consider and let me know:
If you only plan to use SIRIUS5, you can use the MAW-R module (without the spectral database dereplication function which takes substantial time, based on the size of your file). This option will only give .ms input files for SIRIUS.
Do you also want to run your data against spectral DBs (HMDB, MassBank, GNPS) as well? If yes, you would need the three databases as R objects which I can share with you. In this option you run the whole MAW-R module. At the end you will have spectral database dereplication results AND .ms files that can be used as an input to SIRIUS.
Once you perform the MAW-R module, and have .ms files; I can also write quick script to run SIRIUS for all files instead of using the GUI, but that depends on your preference. Using the script would only be possible if you already have SIRIUS CLI installed in your system and you have already logged in using your credentials in terminal (also which OS you are using)?
Also, the workflow for now takes only one file (we are working on a different route for parallelisation), so you can only run one file at a time for now.
My next question would be: Would you like a docker container for this task?
Let me know your thoughts here or write to me on mahnoor.zulfiqar@uni-jena.de
Hope this information helps.
Kind regards, Mahnoor
Dear Mahnoor, I also meet this problem. "I can also write quick script to run SIRIUS for all files instead of using the GUI......" It is very useful! Thank you very much!
Best regards
Li Zhihua
Hi I meet another maybe similar reason problem as below, The following object is masked from ‘package:readr’:
parse_date
Error in unserialize(node$con) :
MultisessionFuture (
Hi I have not found the file spectral_results_for_xxxx.csv in the folder spectral_dereplication root@ed67702f821e:/opt/workdir/data/HY1/spectral_dereplication# ls GNPS HMDB MassBank
Hello,
Please refer to the updated sections in the "provenance" branch README.md.
README.md link from provenance branch
I have mentioned SIRIUS5 as a separate section from MAW-R and following the steps to use SIRIUS5 should work now. It is important to make changes in the parameters to the function run_sirius
according to your data. The function also only takes results from MAW-R which should be run previously to have .ms input files in the directory /file_Name/insilico/SIRIUS
. And a list of the .ms files and their corresponding .json output files are written in a file called /file_Name/insilico/MS1DATA_SiriusP.tsv
.
I hope this is clear enough to run SIRIUS5. Please let me know when you encounter any further issue.
Hi I have not found the file spectral_results_for_xxxx.csv in the folder spectral_dereplication root@ed67702f821e:/opt/workdir/data/HY1/spectral_dereplication# ls GNPS HMDB MassBank
I would assume that the function wasn't finished because this file is generated after the function is finished. Did you encounter any error message or do you think the function was interrupted?
Hi I meet another maybe similar reason problem as below, The following object is masked from ‘package:readr’:
parse_date
Error in unserialize(node$con) : MultisessionFuture () failed to receive results from cluster RichSOCKnode #1 (PID 105 on ‘localhost’). The reason reported was ‘error reading from connection’. Post-mortem diagnostic: The total size of the 31 globals exported is 1.63 MiB. The three largest globals are ‘spec_dereplication_file’ (479.83 KiB of class ‘function’), ‘order’ (358.62 KiB of class ‘function’) and ‘sirius_param’ (114.21 KiB of class ‘function’) Calls: future ... resolved -> resolved.ClusterFuture -> receiveMessageFromWorker Execution halted
If you re run the workflow, do you still get this error? Generally it should work, as it is an error from the parallelisation of the workflow. If you re run and the error persists, let me know.
Hi, Yes I also this errors both in MAW-r:1.0.0 and NAW-r:1.0.1 root@9e99c13fa386:/opt/workdir# Rscript --no-save --no-restore --verbose Workflow_R_Script.r >outputFile.txt running '/usr/local/lib/R/bin/R --no-echo --no-restore --no-save --no-restore --file=Workflow_R_Script.r'
Loading required package: foreach Loading required package: iterators Loading required package: S4Vectors Loading required package: stats4 Loading required package: BiocGenerics
Attaching package: ‘BiocGenerics’
The following objects are masked from ‘package:stats’:
IQR, mad, sd, var, xtabs
The following objects are masked from ‘package:base’:
anyDuplicated, append, as.data.frame, basename, cbind, colnames,
dirname, do.call, duplicated, eval, evalq, Filter, Find, get, grep,
grepl, intersect, is.unsorted, lapply, Map, mapply, match, mget,
order, paste, pmax, pmax.int, pmin, pmin.int, Position, rank,
rbind, Reduce, rownames, sapply, setdiff, sort, table, tapply,
union, unique, unsplit, which.max, which.min
Attaching package: ‘S4Vectors’
The following object is masked from ‘package:future’:
values
The following objects are masked from ‘package:base’:
expand.grid, I, unname
Loading required package: BiocParallel Loading required package: ProtGenerics
Attaching package: ‘ProtGenerics’
The following object is masked from ‘package:stats’:
smooth
Attaching package: ‘Spectra’
The following object is masked from ‘package:ProtGenerics’:
addProcessing
Attaching package: ‘MsCoreUtils’
The following objects are masked from ‘package:Spectra’:
bin, smooth
The following objects are masked from ‘package:ProtGenerics’:
bin, smooth
The following object is masked from ‘package:stats’:
smooth
Attaching package: ‘dplyr’
The following object is masked from ‘package:MsCoreUtils’:
between
The following objects are masked from ‘package:S4Vectors’:
first, intersect, rename, setdiff, setequal, union
The following objects are masked from ‘package:BiocGenerics’:
combine, intersect, setdiff, union
The following objects are masked from ‘package:stats’:
filter, lag
The following objects are masked from ‘package:base’:
intersect, setdiff, setequal, union
Attaching package: ‘rvest’
The following object is masked from ‘package:readr’:
guess_encoding
Loading required package: Rcpp Using libcurl 7.68.0 with OpenSSL/1.1.1f
Attaching package: ‘curl’
The following object is masked from ‘package:readr’:
parse_date
Error in unserialize(node$con) :
MultisessionFuture (
Dear Mahnoor, Maybe this link http://gforge.se/2015/02/how-to-go-parallel-in-r-basics-tips/ will be useful for fixing this problem. Thanks.
Sorry for the delayed response here, I am currently trying to fix all these issues with CWL, but it is still not ready yet. So to work around these issues here are some solutions:
docker pull zmahnoor/maw-r:1.0.8 docker run --name checkMAW_R -v $(pwd):/opt/workdir -i -t zmahnoor/maw-r:1.0.8 /bin/bash
This way, you will have all your files in your pwd(or working directory) mounted to /opt/workdir
inside the container. Now from within the container you can write:
Rscript Workflow_R_Script_all.r your_input.mzML gnps.rda hmdb.rda mbankNIST.rda any-string-as-id 15 coconut COCONUT_Jan2022.csv &
This will give you a PID for this task within the container. You can disown id
to be able to run it in the background. To leave the container you can CTRL+p
and then CTRL+q
to exit in daemon mode.
Now your container will run in the background and write all the outputs in your pwd.You will get the spectral database results, and in insilico folder you will find metfrag parameter files (if you want to use metfrag CLI) and SIRIUS parameter files (that you can use to run SIRIUS or SIRIUS CLI).
once you have these results, ideally you can use MAW-Py to perform candidate selection. I have already prepared scripts for candidate selection with Spectral DBs and Metfrag and but due to some changes in SIRIUS results, that is not currently possible. However, I will work on it after the MAW-CWL is ready.
Let me know if this solves the problem. If the future library problem persists, can you send me an example file, and your systems RAM? I will try to recreate the error and solve it with Luiz.
Kind regards, Mahnoor
Dear Prof.Mahnoor, Thank you very much for your explanation! I have tried to fix this problem.
I want to express thanks for your idea in MAW again and I really very like it.
Best regards
Li Zhihua
发件人: Mahnoor Zulfiqar @.> 发送时间: 2023年5月1日 20:41 收件人: zmahnoor14/MAW @.> 抄送: Li Zhihua @.>; Comment @.> 主题: Re: [zmahnoor14/MAW] Think I can't connect to Sirius because of the new 5.0 version (Issue #33)
Sorry for the delayed response here, I am currently trying to fix all these issues with CWL, but it is still not ready yet. So to work around these issues here are some solutions:
Running spectral database dereplication: I understand the error is from future library and since my collaborator @lgadelhahttps://github.com/lgadelha added this function, I will ask him for this possible error. While I run the files that I have here I don't get this error so, I would ask you to send me one mzML example so that I also take a look at it and try to reproduce the error @.**@. Meanwhile you can try to run the latest R script. You will need the following:
You will get the spectral database results, and in insilico folder you will find metfrag parameter files (if you want to use metfrag CLI) and SIRIUS parameter files (that you can use to run SIRIUS or SIRIUS CLI).
once you have these results, ideally you can use MAW-Py to perform candidate selection. I have already prepared scripts for candidate selection with Spectral DBs and Metfrag and but due to some changes in SIRIUS results, that is not currently possible. However, I will work on it after the MAW-CWL is ready.
Let me know if this solves the problem. If the future library problem persists, can you send me an example file, and your systems RAM? I will try to recreate the error and solve it with Luiz.
Kind regards, Mahnoor
― Reply to this email directly, view it on GitHubhttps://github.com/zmahnoor14/MAW/issues/33#issuecomment-1529674327, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJI2BJOWVEWDOQADQEAFPPTXD6VQBANCNFSM6AAAAAAV2JNORA. You are receiving this because you commented.Message ID: @.***>
Dear Prof.Mahnoor,
[cid:6a423f58-8d28-4c2e-9b76-206fb2a17ad1]
Thanks again!
Best regards
Li Zhihua
发件人: Mahnoor Zulfiqar @.> 发送时间: 2023年5月1日 20:41 收件人: zmahnoor14/MAW @.> 抄送: Li Zhihua @.>; Comment @.> 主题: Re: [zmahnoor14/MAW] Think I can't connect to Sirius because of the new 5.0 version (Issue #33)
Sorry for the delayed response here, I am currently trying to fix all these issues with CWL, but it is still not ready yet. So to work around these issues here are some solutions:
Running spectral database dereplication: I understand the error is from future library and since my collaborator @lgadelhahttps://github.com/lgadelha added this function, I will ask him for this possible error. While I run the files that I have here I don't get this error so, I would ask you to send me one mzML example so that I also take a look at it and try to reproduce the error @.**@. Meanwhile you can try to run the latest R script. You will need the following:
You will get the spectral database results, and in insilico folder you will find metfrag parameter files (if you want to use metfrag CLI) and SIRIUS parameter files (that you can use to run SIRIUS or SIRIUS CLI).
once you have these results, ideally you can use MAW-Py to perform candidate selection. I have already prepared scripts for candidate selection with Spectral DBs and Metfrag and but due to some changes in SIRIUS results, that is not currently possible. However, I will work on it after the MAW-CWL is ready.
Let me know if this solves the problem. If the future library problem persists, can you send me an example file, and your systems RAM? I will try to recreate the error and solve it with Luiz.
Kind regards, Mahnoor
― Reply to this email directly, view it on GitHubhttps://github.com/zmahnoor14/MAW/issues/33#issuecomment-1529674327, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJI2BJOWVEWDOQADQEAFPPTXD6VQBANCNFSM6AAAAAAV2JNORA. You are receiving this because you commented.Message ID: @.***>
Dear Prof.Mahnoor, 1.Just Now, the MAW is still works (May be 4-5days log), it works at the parse_data step as below. The computer is 64 RAM and Ubuntu 18.04.
Becuse it cannot access the zenodo in China, so I copy the GNPs.rda, hmdb.rda... from the maw-r:1.0.1, but I can not find the COCONUT database for MetFrag, So where I can find it in the maw-r:1.0.1[https://res-h3.public.cdn.office.net/assets/mail/file-icon/png/generic_16x16.png]QC21.mzMLhttps://1drv.ms/u/s!An_qG-a0TjfhgVO604WuokuiQU6g
Best regards
Li Zhihua
[cid:aaafae04-3cce-410a-98e4-65bb641e561f]
发件人: Mahnoor Zulfiqar @.> 发送时间: 2023年5月1日 20:41 收件人: zmahnoor14/MAW @.> 抄送: Li Zhihua @.>; Comment @.> 主题: Re: [zmahnoor14/MAW] Think I can't connect to Sirius because of the new 5.0 version (Issue #33)
Sorry for the delayed response here, I am currently trying to fix all these issues with CWL, but it is still not ready yet. So to work around these issues here are some solutions:
Running spectral database dereplication: I understand the error is from future library and since my collaborator @lgadelhahttps://github.com/lgadelha added this function, I will ask him for this possible error. While I run the files that I have here I don't get this error so, I would ask you to send me one mzML example so that I also take a look at it and try to reproduce the error @.**@. Meanwhile you can try to run the latest R script. You will need the following:
You will get the spectral database results, and in insilico folder you will find metfrag parameter files (if you want to use metfrag CLI) and SIRIUS parameter files (that you can use to run SIRIUS or SIRIUS CLI).
once you have these results, ideally you can use MAW-Py to perform candidate selection. I have already prepared scripts for candidate selection with Spectral DBs and Metfrag and but due to some changes in SIRIUS results, that is not currently possible. However, I will work on it after the MAW-CWL is ready.
Let me know if this solves the problem. If the future library problem persists, can you send me an example file, and your systems RAM? I will try to recreate the error and solve it with Luiz.
Kind regards, Mahnoor
― Reply to this email directly, view it on GitHubhttps://github.com/zmahnoor14/MAW/issues/33#issuecomment-1529674327, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJI2BJOWVEWDOQADQEAFPPTXD6VQBANCNFSM6AAAAAAV2JNORA. You are receiving this because you commented.Message ID: @.***>
Hi Mahnoor, After I pull the WAM:1.0.8, there is no Rscript Workflow_R_Script_all.r
Best regards
Li Zhihua
获取 Outlook for iOShttps://aka.ms/o0ukef
发件人: li zhihua @.> 发送时间: Monday, May 1, 2023 9:30:17 PM 收件人: zmahnoor14/MAW @.> 主题: 回复: [zmahnoor14/MAW] Think I can't connect to Sirius because of the new 5.0 version (Issue #33)
Dear Prof.Mahnoor,
[cid:6a423f58-8d28-4c2e-9b76-206fb2a17ad1]
Thanks again!
Best regards
Li Zhihua
发件人: Mahnoor Zulfiqar @.> 发送时间: 2023年5月1日 20:41 收件人: zmahnoor14/MAW @.> 抄送: Li Zhihua @.>; Comment @.> 主题: Re: [zmahnoor14/MAW] Think I can't connect to Sirius because of the new 5.0 version (Issue #33)
Sorry for the delayed response here, I am currently trying to fix all these issues with CWL, but it is still not ready yet. So to work around these issues here are some solutions:
Running spectral database dereplication: I understand the error is from future library and since my collaborator @lgadelhahttps://github.com/lgadelha added this function, I will ask him for this possible error. While I run the files that I have here I don't get this error so, I would ask you to send me one mzML example so that I also take a look at it and try to reproduce the error @.**@. Meanwhile you can try to run the latest R script. You will need the following:
You will get the spectral database results, and in insilico folder you will find metfrag parameter files (if you want to use metfrag CLI) and SIRIUS parameter files (that you can use to run SIRIUS or SIRIUS CLI).
once you have these results, ideally you can use MAW-Py to perform candidate selection. I have already prepared scripts for candidate selection with Spectral DBs and Metfrag and but due to some changes in SIRIUS results, that is not currently possible. However, I will work on it after the MAW-CWL is ready.
Let me know if this solves the problem. If the future library problem persists, can you send me an example file, and your systems RAM? I will try to recreate the error and solve it with Luiz.
Kind regards, Mahnoor
― Reply to this email directly, view it on GitHubhttps://github.com/zmahnoor14/MAW/issues/33#issuecomment-1529674327, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AJI2BJOWVEWDOQADQEAFPPTXD6VQBANCNFSM6AAAAAAV2JNORA. You are receiving this because you commented.Message ID: @.***>
Dear Li Zhihua,
sorry for my late response, I am still a PhD student and currently trying to finish my projects and have a lot of work on hands : )
Here is the link to the COCONUT installation: https://upload.uni-jena.de/data/6459b9bd932378.80487188/COCONUT_Jan2022.csv
I would also suggest to use the updated HMDB5.0 version and here is the link for the download: https://upload.uni-jena.de/data/6459ba4f540612.50374105/hmdb.rda
Regarding Workflow_R_Script_all.r, you can download the latest one on the GitHub repository with the link: https://github.com/zmahnoor14/MAW/blob/provenance/cwl/Workflow_R_Script_all.r
Hi,
I would like to try your tool and workflow but when I launch your data test I obtain a lot of informations but the following also :
And after it restart trying to connect in loop and never finish.... Think that Sirius closed all connection with 4.9 version...
Can someone help me please ?