MannLabs / alphapept

A modular, python-based framework for mass spectrometry. Powered by nbdev.
https://mannlabs.github.io/alphapept/
Apache License 2.0
167 stars 29 forks source link

Process hangs when using a Bruker's .d file #562

Open kenll99minecart opened 2 months ago

kenll99minecart commented 2 months ago

Dear all, I was trying to search for a .d file using the gui with alphapept but the process hangs without any response in the feature finding step. I had waited for 2 hours and there is no extra output line to the gui log. I am using AlphaPept Version 0.5.0 for the stable version.

Describe the bug Process hangs when try to process .d file

To Reproduce Steps to reproduce the behavior: Installed anaconda3 on linux Ubuntu 22.04.4 LTS

conda create -n alphapept python=3.8.19 pip install pillow==9.0.0 (As suggested in issue #561 ) pip install alphapept[develop,gui] conda install conda-forge::pythonnet sudo cp /programs/conda/anaconda3/envs/alphapept/lib/python3.8/site-packages/alphapept/ext/bruker/FF/linux64/libtbb.so.2 /programs/conda/anaconda3/envs/alphapept/lib/libtbb.so.2 sudo cp /programs/conda/anaconda3/envs/alphapept/lib/python3.8/site-packages/alphapept/ext/bruker/FF/linux64/libtbb.so.2 /usr/lib/libtbb.so.2 conda install conda-forge::mono pip install python-lzf alphapept gui -p 8511

Results:

 ___    __      __          ____             __
/   |  / /___  / /_  ____  / __ \___  ____  / /_

/ /| | / / \/ \/ \/ // / \/ \/ / / _ |/ / // / / / / // / __/ _/ // / / // |/_/ .// //_,// _/ ./\/ // /_/ .................................................... . https://github.com/MannLabs/alphapept . . Mann Labs . . 0.5.0 -> Update available (0.5.3) . ....................................................

Starting AlphaPept Background Process 2024-07-03 02:32:43.839 WARNING streamlit.runtime.caching.cache_data_api: No runtime found, using MemoryCacheStorageManager 2024-07-03 02:32:43.849254 Started queue_watcher Starting AlphaPept Server

You can now view your Streamlit app in your browser.

Local URL: http://localhost:8511 Network URL: ###Masked###:8511

2024-07-03 02:32:46.203 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2024-07-03 02:32:46.213 Please replace st.experimental_data_editor with st.data_editor.

st.experimental_data_editor will be removed after 2023-09-01.

Breaking change: The data editor's representation in st.session_state was changed. The edited_cells dictionary is now called edited_rows and uses a different format ({0: {"column name": "edited value"}} instead of {"0:1": "edited value"}). You may need to adjust the code if your app uses st.experimental_data_editor in combination with st.session_state."

2024-07-03 02:32:46.391 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2024-07-03 02:35:45.925 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2024-07-03 02:35:45.932 Please replace st.experimental_data_editor with st.data_editor.

st.experimental_data_editor will be removed after 2023-09-01.

Breaking change: The data editor's representation in st.session_state was changed. The edited_cells dictionary is now called edited_rows and uses a different format ({0: {"column name": "edited value"}} instead of {"0:1": "edited value"}). You may need to adjust the code if your app uses st.experimental_data_editor in combination with st.session_state."

2024-07-03 02:35:48.217 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed.

2024-07-03 02:35:59> Logging to /home/james/alphapept/logs/log_20240703023559.txt. 2024-07-03 02:35:59> Code location /programs/conda/anaconda3/envs/alphapept/lib/python3.8/site-packages/alphapept 2024-07-03 02:35:59> Python location /programs/conda/anaconda3/envs/alphapept/bin/python 2024-07-03 02:35:59> Logging to /home/james/.alphapept/finished/2024_07_03_Test3.log. 2024-07-03 02:35:59> Code location /programs/conda/anaconda3/envs/alphapept/lib/python3.8/site-packages/alphapept 2024-07-03 02:35:59> Python location /programs/conda/anaconda3/envs/alphapept/bin/python 2024-07-03 02:35:59> Platform information: 2024-07-03 02:35:59> system - Linux 2024-07-03 02:35:59> release - 6.5.0-41-generic 2024-07-03 02:35:59> version - #41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC Mon Jun 3 11:32:55 UTC 2 2024-07-03 02:35:59> machine - x86_64 2024-07-03 02:35:59> processor - x86_64 2024-07-03 02:35:59> cpu count - 64 2024-07-03 02:35:59> ram memory - 452.9/503.5 Gb (available/total) 2024-07-03 02:35:59> processor - x86_64 2024-07-03 02:35:59> Python information: 2024-07-03 02:35:59> SQLAlchemy - 2.0.31 2024-07-03 02:35:59> alphapept - 0.5.0 2024-07-03 02:35:59> alpharaw - 0.4.5 2024-07-03 02:35:59> alphatims - 1.0.8 2024-07-03 02:35:59> biopython - 1.83 2024-07-03 02:35:59> bumpversion - 0.6.0 2024-07-03 02:35:59> click - 8.1.7 2024-07-03 02:35:59> fastcore - 1.5.48 2024-07-03 02:35:59> h5py - 3.11.0 2024-07-03 02:35:59> jupyter - 1.0.0 2024-07-03 02:35:59> jupyter-contrib-nbextensions - 0.7.0 2024-07-03 02:35:59> matplotlib - 3.7.5 2024-07-03 02:35:59> nbdev - 2.3.25 2024-07-03 02:35:59> networkx - 3.1 2024-07-03 02:35:59> numba - 0.58.1 2024-07-03 02:35:59> numpy - 1.24.4 2024-07-03 02:35:59> pandas - 2.0.3 2024-07-03 02:35:59> plotly-express - 0.4.1 2024-07-03 02:35:59> psutil - 6.0.0 2024-07-03 02:35:59> pyinstaller - 6.8.0 2024-07-03 02:35:59> pyteomics - 4.7.2 2024-07-03 02:35:59> python - 3.8.19 2024-07-03 02:35:59> python-lzf - 0.2.6 2024-07-03 02:35:59> pythonnet - 3.0.3 2024-07-03 02:35:59> pywin32 - 2024-07-03 02:35:59> pyyaml - 6.0.1 2024-07-03 02:35:59> scikit-learn - 1.3.2 2024-07-03 02:35:59> scipy - 1.10.1 2024-07-03 02:35:59> streamlit - 1.36.0 2024-07-03 02:35:59> tables - 3.8.0 2024-07-03 02:35:59> tqdm - 4.66.4 2024-07-03 02:35:59> twine - 5.1.1 2024-07-03 02:35:59> watchdog - 4.0.1 2024-07-03 02:35:59> wget - 3.2 2024-07-03 02:35:59> 2024-07-03 02:35:59> Check for settings not completely implemented yet. 2024-07-03 02:35:59> Size check: 2024-07-03 02:35:59> Size of job (raw files) 1.71 Gb 2024-07-03 02:35:59> Required disk space for / - 1.71 Gb, Available 105.20 Gb OK. 2024-07-03 02:35:59> 2024-07-03 02:35:59> Workflow Settings: 2024-07-03 02:35:59> align - False 2024-07-03 02:35:59> continue_runs - False 2024-07-03 02:35:59> create_database - True 2024-07-03 02:35:59> find_features - True 2024-07-03 02:35:59> import_raw_data - True 2024-07-03 02:35:59> lfq_quantification - True 2024-07-03 02:35:59> match - False 2024-07-03 02:35:59> recalibrate_data - True 2024-07-03 02:35:59> search_data - True 2024-07-03 02:35:59> 2024-07-03 02:35:59> Checking if files exist. 2024-07-03 02:35:59> FASTA Files have a total size of 1.803 Mb 2024-07-03 02:35:59> Results path was not set. Setting to /data2/results.hdf 2024-07-03 02:35:59> No database path set and save_db option checked. Using default path /data2/database.hdf 2024-07-03 02:35:59> Deleted /data2/SASn-culture-1_Slot1-37_1_9051.ms_data.hdf 2024-07-03 02:35:59> Workflow has 11 steps 2024-07-03 02:35:59> Setting callback to logger. 2024-07-03 02:35:59> ==== create_database ==== 2024-07-03 02:35:59> current_task create_database 2024-07-03 02:35:59> progress_current 0.000 2024-07-03 02:35:59> progress_overall 0.000 2024-07-03 02:35:59> Database path set and exists. Using /data2/database.hdf as database. 2024-07-03 02:35:59> __progress_current 1.000 2024-07-03 02:35:59> progress_overall 0.091 2024-07-03 02:35:59> ==== import_raw_data ==== 2024-07-03 02:35:59> current_task import_raw_data 2024-07-03 02:35:59> __progress_current 0.000 2024-07-03 02:35:59> progress_overall 0.091 2024-07-03 02:35:59> Processing 1 files for step raw_conversion 2024-07-03 02:35:59> File /data2/SASn-culture-1_Slot1-37_1_9051 has extension .d - converting from Bruker. 2024-07-03 02:35:59> Importing data from /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:35:59> Using .d import for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:35:59> Reading frame metadata for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:35:59.326 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2024-07-03 02:35:59.333 Please replace st.experimental_data_editor with st.data_editor.

st.experimental_data_editor will be removed after 2023-09-01.

Breaking change: The data editor's representation in st.session_state was changed. The edited_cells dictionary is now called edited_rows and uses a different format ({0: {"column name": "edited value"}} instead of {"0:1": "edited value"}). You may need to adjust the code if your app uses st.experimental_data_editor in combination with st.session_state."

2024-07-03 02:35:59> Reading 16,935 frames with 506,282,260 detector events for /data2/SASn-culture-1_Slot1-37_1_9051.d 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16935/16935 [00:03<00:00, 4937.32it/s] 2024-07-03 02:36:03> Indexing /data2/SASn-culture-1_Slot1-37_1_9051.d... 2024-07-03 02:36:03> Opening handle for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Fetching mobility values from /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Closing handle for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Opening handle for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Fetching mz values from /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Closing handle for /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:03> Indexing quadrupole dimension 2024-07-03 02:36:05> Successfully imported data from /data2/SASn-culture-1_Slot1-37_1_9051.d 0%| | 0/109396 [00:00<?, ?it/s]2024-07-03 02:36:06.009 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 2024-07-03 02:36:06.017 Please replace st.experimental_data_editor with st.data_editor.

st.experimental_data_editor will be removed after 2023-09-01.

Breaking change: The data editor's representation in st.session_state was changed. The edited_cells dictionary is now called edited_rows and uses a different format ({0: {"column name": "edited value"}} instead of {"0:1": "edited value"}). You may need to adjust the code if your app uses st.experimental_data_editor in combination with st.session_state."

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 109396/109396 [00:01<00:00, 57320.85it/s] 0%| | 0/109396 [00:00<?, ?it/s]2024-07-03 02:36:07.785 label got an empty value. This is discouraged for accessibility reasons and may be disallowed in the future by raising an exception. Please provide a non-empty label and hide it with label_visibility if needed. 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 109396/109396 [00:01<00:00, 107454.00it/s] 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 109396/109396 [00:02<00:00, 43598.69it/s] 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 109396/109396 [00:00<00:00, 212907.37it/s] 2024-07-03 02:36:11> File conversion complete. Extracted 109,396 precursors. 2024-07-03 02:36:12> File conversion of file /data2/SASn-culture-1_Slot1-37_1_9051.d complete. 2024-07-03 02:36:12> progress_current 1.000 2024-07-03 02:36:12> progress_overall 0.182 2024-07-03 02:36:12> ==== feature_finding ==== 2024-07-03 02:36:12> current_task feature_finding 2024-07-03 02:36:12> __progress_current 0.000 2024-07-03 02:36:12> progress_overall 0.182 2024-07-03 02:36:13> Processing 1 files for step find_features 2024-07-03 02:36:13> No *.hdf file with features found for /data2/SASn-culture-1_Slot1-37_1_9051.ms_data.hdf. Adding to feature finding list. 2024-07-03 02:36:13> Feature finding on /data2/SASn-culture-1_Slot1-37_1_9051.d 2024-07-03 02:36:13> Using Linux FF 2024-07-03 02:36:13.426380 [tid=0x9cfdd940] [WARN ] bdal.cmdline.readconfig: '--/programs/conda/anaconda3/envs/alphapept/lib/python3.8/site-packages/alphapept/ext/bruker/FF/linux64/uff-cmdline2' in config file does not match any command line parameter key and is ignored.

Expected behavior Feature finding

Screenshots Processing Steps image Default Setting is used for all processing steps Screenshot for AlphaPept Status, with CPU usage also stays at 100% during the whole 2 hours image

Additional context I have also tried other .d files and it do not works. However, this installation works with .raw files, so I am quite confused on what's happening.