Connection Issues

[Zhangantai]

Dear developers,

I am using Sirius in CLI within a batch activition script. I login with my account and password when initializing the script, but I found the web server drops after around 10 processes, with tons of "Error: read time out" and " Cause: Timeout deadline: 9000 MILLISECONDS, actual: 9003 MILLISECONDS"

Here I would provide my log file, could you give me some suggestions to solve this problem? sirius.log.4.txt

Best, Antai

[joXemMx]

Hi, could you give some further information on how your script works? It would be good to know how it initializes Sirius and how the calculations are done. What version of Sirius are you running? I would like to try and reproduce the error if possible. Greetings, Jonas

[Zhangantai]

Dear developer,  Thanks for you reply. Sure, my sirius version is 5.63, and here is my batch script. 

@echo off sirius login -u @.*** -p setlocal enabledelayedexpansion setlocal

set "dir_path=D:\ORGANIC_ANALYSIS\APCI_POS\processed" set "output_dir=D:\ORGANIC_ANALYSIS\APCI_POS\test_gen"

if not exist "%output_dir%" mkdir "%output_dir%"

for %%f in ("%dir_path%*.*") do (     if %%~xf neq "" (         set "filename=%%~nf"         set "folder=%output_dir%!filename!"         if not exist "!folder!" mkdir "!folder!"         pushd "!folder!"         sirius --input="%%~ff" --output="!folder!" --workspace="!folder!" config --IsotopeSettings.filter=true --FormulaSearchDB= --Timeout.secondsPerTree=0 --FormulaSettings.enforced=H[2-]C[2-]N[5]OS[5] --Timeout.secondsPerInstance=0 --AdductSettings.detectable=[[M+K]+,[M-H2O+H]+,[M-H4O2+H]+,[M+Na]+,[M+H]+,[M+H3N+H]+] --UseHeuristic.mzToUseHeuristicOnly=650 --AlgorithmProfile=orbitrap --IsotopeMs2Settings=SCORE --MS2MassDeviation.allowedMassDeviation=5.0ppm --NumberOfCandidatesPerIon=1 --UseHeuristic.mzToUseHeuristic=300 --FormulaSettings.detectable=S --NumberOfCandidates=10 --ZodiacNumberOfConsideredCandidatesAt300Mz=10 --ZodiacRunInTwoSteps=true --ZodiacEdgeFilterThresholds.minLocalConnections=10 --ZodiacEdgeFilterThresholds.thresholdFilter=0.95 --ZodiacEpochs.burnInPeriod=2000 --ZodiacEpochs.numberOfMarkovChains=10 --ZodiacNumberOfConsideredCandidatesAt800Mz=50 --ZodiacEpochs.iterations=20000 --AdductSettings.enforced=, --AdductSettings.fallback=[[M+K]+,[M+H]+,[M+Na]+] --FormulaResultThreshold=true --InjectElGordoCompounds=true --StructureSearchDB=ALL --RecomputeResults=false formula zodiac fingerprint structure W --output="!folder!"         popd     ) )

endlocal

Thanks for helping, if you need furthur infomation, please leave me a message soon. 

Best, 

Zhang Antai

张安泰

南方科技大学/学生/研究生/2021级研究生

广东省深圳市南山区学苑大道1088号

      ------------------ Original ------------------ From: @.>; Date:  Wed, Apr 5, 2023 03:44 AM To: @.>; Cc: @.>; @.>; Subject:  Re: [boecker-lab/sirius] Connection Issues (Issue #113)

 

Hi, could you give some further information on how your script works? It would be good to know how it initializes Sirius and how the calculations are done. What version of Sirius are you running? I would like to try and reproduce the error if possible. Greetings, Jonas

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[joXemMx]

Hello Antai, so far I could not find any apparent issue with your script and was not able to reproduce these errors. Looking at your error log, the current best guess would simply be that the service was occupied. It may also be the case that your threads have to wait too long until they get a pooled connection, should your device not have a sufficient amount of cores (in this case, try running it without any other tasks going on). Does the error still occur when running the script, or may it have just been during that time? Greetings, Jonas

[Zhangantai]

Hello Jonas, Thanks for your reply. Here is my device information, and I think my computer is capable enough to run Sirius with multiple tasks. And I found the error was still occuring while running the script, and it seems that the error occurred due to poor web services. Thanks for your reply.

Device information Processor AMD Ryzen 9 5950X 16-Core Processor 3.40 GHz RAM 64.0 GB (63.9 GB 可用) Operation system 64 位操作系统, 基于 x64 的处理器

The latest log file I will send you an email as an attachment later. Thanks, Antai

[Zhangantai]

sirius.log.4.4.txt My bad, here is the log file

[joXemMx]

Hello Antai, thanks for your feedback. You are right, your device is definitely not the problem here. Would you be able / allowed to share the files that your script runs on? That way I should have a better chance at reproducing the error. Greetings, Jonas

[joXemMx]

Hi again,

I have to apologize, I got some things twisted. Regarding the number of running threads on the users machine, the error you encounter might actually be caused by having too many.

SIRIUS has a fixed amount of connections to the servers and excess threads will have to wait for connections to become available. As your logs show Timeout deadline: 9000 MILLISECONDS, this is compliant with the connectRequestTimeout = 9000 set in the sirius_frontend.build.properties file of release 5.6.3.

Therefore, your error is likely to be fixed simply by specifying the amount of cores used by SIRIUS via the --cores=<numOfCores> parameter or by migrating to the current 5.6.4-SNAPSHOT version which uses connectRequestTimeout = 60000.

Please try those two options first before going through the trouble of sending me your data.

Greetings, Jonas

[Zhangantai]

Thanks for your reply, I didn't found the "sirius_frontend.build.properties" file, so i added a line "de.unijena.bioinf.sirius.connectRequestTimeout = 60000" in sirius.properties file. And could you tell me how to specify the amount of cores paramerter?

[joXemMx]

Hi, sorry for any confusion. I was talking about the sirius/sirius_cli/src/main/resources/sirius_frontend.build.properties file in this GitHub repository. You will not find it in your local installation, thus you can not change it manually but would have to migrate to 5.6.4-SNAPSHOT.

Regarding the cores parameter, just add --cores=<numOfCores> to your sirius call in the batch script. For example, you could try:

sirius --cores=8 --input="%%~ff" --output="!folder!" --workspace="!folder!" config --IsotopeSettings.filter=true --FormulaSearchDB= --Timeout.secondsPerTree=0 --FormulaSettings.enforced=H[2-]C[2-]N[5]OS[5] --Timeout.secondsPerInstance=0 --AdductSettings.detectable=[[M+K]+,[M-H2O+H]+,[M-H4O2+H]+,[M+Na]+,[M+H]+,[M+H3N+H]+] --UseHeuristic.mzToUseHeuristicOnly=650 --AlgorithmProfile=orbitrap --IsotopeMs2Settings=SCORE --MS2MassDeviation.allowedMassDeviation=5.0ppm --NumberOfCandidatesPerIon=1 --UseHeuristic.mzToUseHeuristic=300 --FormulaSettings.detectable=S --NumberOfCandidates=10 --ZodiacNumberOfConsideredCandidatesAt300Mz=10 --ZodiacRunInTwoSteps=true --ZodiacEdgeFilterThresholds.minLocalConnections=10 --ZodiacEdgeFilterThresholds.thresholdFilter=0.95 --ZodiacEpochs.burnInPeriod=2000 --ZodiacEpochs.numberOfMarkovChains=10 --ZodiacNumberOfConsideredCandidatesAt800Mz=50 --ZodiacEpochs.iterations=20000 --AdductSettings.enforced=, --AdductSettings.fallback=[[M+K]+,[M+H]+,[M+Na]+] --FormulaResultThreshold=true --InjectElGordoCompounds=true --StructureSearchDB=ALL --RecomputeResults=false formula zodiac fingerprint structure W --output="!folder!"

[Zhangantai]

Hello Jonas, Thanks for your reply. I have upgraded my version of sirius to v5.64 snapshot, the connection seems OK, but it is still taking long time to finish the job, which is out of my exceptation. And here I would provide one of my ongoing processing data, could you help me test this data to reproduce the connection problem? 38.zip this is a zipped .mgf data, and the data is in negative ion mode, which i used the following code for config: sirius --cores=16 --input="%%~ff" --output="!folder!" --workspace="!folder!" config --IsotopeSettings.filter=true --FormulaSearchDB= --Timeout.secondsPerTree=0 --FormulaSettings.enforced=H[2-]C[2-]N[5]OS[5] --Timeout.secondsPerInstance=0 --AdductSettings.detectable=[[M-H2O-H]-,[M-H]-] --UseHeuristic.mzToUseHeuristicOnly=650 --AlgorithmProfile=orbitrap --IsotopeMs2Settings=SCORE --MS2MassDeviation.allowedMassDeviation=5.0ppm --NumberOfCandidatesPerIon=1 --UseHeuristic.mzToUseHeuristic=300 --FormulaSettings.detectable=, --NumberOfCandidates=10 --ZodiacNumberOfConsideredCandidatesAt300Mz=10 --ZodiacRunInTwoSteps=true --ZodiacEdgeFilterThresholds.minLocalConnections=10 --ZodiacEdgeFilterThresholds.thresholdFilter=0.95 --ZodiacEpochs.burnInPeriod=2000 --ZodiacEpochs.numberOfMarkovChains=10 --ZodiacNumberOfConsideredCandidatesAt800Mz=50 --ZodiacEpochs.iterations=20000 --AdductSettings.enforced=, --AdductSettings.fallback=[[M-H]-] --FormulaResultThreshold=true --InjectElGordoCompounds=true --StructureSearchDB=BIO,METACYC,CHEBI,COCONUT,ECOCYCMINE,GNPS,HMDB,HSDB,KEGG,KEGGMINE,KNAPSACK,MACONDA,MESH,NORMAN,UNDP,PLANTCYC,PUBCHEM,PUBMED,YMDB,YMDBMINE,ZINCBIO --RecomputeResults=false formula zodiac fingerprint structure W --output="!folder!" Best, Antai

[Zhangantai]

Meanwhile, I am using Sirius v5.6.4 GUI version to recalculate some of my results. The whole process works smoothly except structure finding, and here is the error message from webservice connection.

[joXemMx]

Sorry for not anwering for a bit. Is the connection still not working? I just tested for myself on the 5.6.4-SNAPSHOT GUI and encounter no such problems.

[Zhangantai]

Thanks for your reply. The network error sometimes would occur, but that's not the most severe problem. The problem is in CLI mode, the structure finding process is taking days to finish one project which contains no more than 3000 compounds (and not finished yet). The duration of finding is out of my exceptation, and I think this issue is caused by poor quality of web connection. Intrestingly, the GUI mode performed well today, without encountering the conncetion issue.

[Zhangantai]

Thanks for your assistance, I finally identify the problem: the existance of few threads with connection difficulties, the structure finding process then would take days to finish. Since my study not asking to identify all of the detected compounds with structures, results with some lack of structure assignment are acceptable. The problem i met is shown in the figure, and these running tasks should be canceled after hours attempts, and these threads could be cancelled manually in GUI version. However, these threads could not be easily canceled in CLI.

And therefore, I would like to suggest an additional improvement for your software, which is leave an option to cancel the threads with hours long. I know little about java programming, so i would illustrate it in pseudocode: moniter the progress; #like the progress bar in GUI mode when (total progress greater than 95%) : { counting time; when (time greater than [placeholder, like 1200000] milliseconds) : { kill all activating threads }; }