import warning from ms_deisotope and ms_peak_picker

[ZhixuNi]

Describe the bug I observed these warnings for a while but my program was working fine. However, this may leads to some confusion that there is something wrong with our software. Is there any way to avoid / disable these warnings?

To Reproduce

import pymzml
No module named 'ms_peak_picker._c.peak_picker'
No module named 'ms_deisotope._c.averagine' averagine
No module named 'ms_deisotope._c.scoring'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'

Expected behavior Should be no warning, or warning on first use, or can be disabled.

Desktop (please complete the following information):

• OS: Ubuntu 18.04, Windows 10
• Python 3.7.7

cython==0.29.17
ms_deisotope==0.0.14
ms-peak-picker==0.1.27
numpy==1.18.4
pymzml==2.4.6

[MKoesters]

Hi,

this warning is created by ms_deisotope, so I can't really fix it in pymzML I guess. Do you need the deconvolution functionality for you software? If not, you could deinstall ms_deisotope.

Also, If I either install pymzml using pip install pymzml or if I clone the git and use pip install . ms_deisotopes is not installed, it's only installed if you install all the requirements from the requirements.txt or if you run either "pip install pymzml[full]" or "pip install pymzml[deconvolution]"

This seems to be related to https://github.com/mobiusklein/ms_deisotope/issues/12, however I doubt thats your problem. If you need the deconvolution functionality, maybe @mobiusklein can give us a hint how to fix that?

Best, Manuel

[ZhixuNi]

Thanks for your reply! I think I had similar issues like issue #157, so I some how got ms_deisotope installed to avoid previous issue. Since then ms_desiotope stayed in my dev env and case this issue some time ago. Now I removed ms_deisotope using pip and now there is no warning and my program works fine. I will try to see if I can solve this ms_desiotope issue when I need it referring to mobiusklein/ms_deisotope#12, thanks!

You can close this issue :thumbsup:

[mobiusklein]

Those warnings indicate that the C extensions weren't successfully compiled/imported. Could you please run pip install --no-cache-dir -v --log piplog.txt ms_deisotope and share the log? That way I could see if there were any errors during the build process.

There is one more issue caused by PEP518 which is that while build isolation lets you ensure that a C-extension is compiled before another one, it doesn't let you pin the version of those dependencies to "whatever the user has installed". This might mean that the version of NumPy installed in your environment is not the same as the version used to build the library, which can cause a successful installation to fail at import time.

[ZhixuNi]

Hi! I reinstalled again using pip install --no-cache-dir -v --log piplog.txt ms_deisotope Please have a look of the log I got.

piplog.txt

I still get the same error. Ubuntu 18.04 with Anaconda 3

Python 3.7.7 (default, Mar 26 2020, 15:48:22) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymzml
No module named 'ms_deisotope._c.averagine' averagine
No module named 'ms_deisotope._c.scoring'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'

[ZhixuNi]

I updated the numpy in base env of conda so now both numpy 1.18.4 and Cython 0.29.17, but still get the same error message. I set conda to be my default python in zsh and bash. However, the system numpy package from ubuntu is version 1.13.3. Here is the log: piplog_2.txt

Thanks for the help!

[mobiusklein]

Thank you for going through the investigation steps. From what I'm seeing in the first log, when it tried to build ms_deisotope, ms_peak_picker wasn't built first, so compilation was skipped. In the second log, it looks like ms_deisotope was seen as already installed, so it skipped building ms_deisotope but it saw that ms_peak_picker wasn't installed so it built and installed that instead.

In the first re-install message, you showed that the ms_peak_picker import warning disappeared, which makes this second log re-installing ms_peak_picker weird. I'll need to trace through the logs more carefully.

[mobiusklein]

You might need to uninstall both ms_deisotope and ms_peak_picker and then redo the installation to be able to track down the problem.

[ZhixuNi]

You might need to uninstall both ms_deisotope and ms_peak_picker and then redo the installation to be able to track down the problem.

I just tired to setup a clean and new environment in conda using following commands. *. my conda base has python=3.7.3 numpy==1.18.4 cython==0.29.17

conda create -n envtest python=3.7.3 
conda activate envtest
pip install numpy==1.18.4 cython==0.29.17 
pip install ms-peak-picker
pip install --no-cache-dir -v --log piplog_3.txt ms_deisotope
pip install pymzml matplotlib

This error still occurs:

Python 3.7.3 (default, Mar 27 2019, 22:11:17) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymzml
No module named 'ms_peak_picker._c.peak_picker'
No module named 'ms_deisotope._c.averagine' averagine
No module named 'ms_deisotope._c.scoring'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'
No module named 'ms_deisotope._c.deconvoluter_base'
>>> 

Here is the log piplog_3.txt

Maybe this log is what you need.

[mobiusklein]

Okay, now the build is complaining that it can't find brainpy, but it thinks it found ms_peak_picker

2020-05-08T14:52:40,454     Installation requires `brainpy`, install with `python -m pip install brain-isotopic-distribution`

It's obvious it's not a problem with compilation on your end. I'll need to debug the build isolation configuration. Thank you for your patience and sending these logs.

[mobiusklein]

I think I've found the problem. Older versions of setuptools do not automatically include the pyproject.toml file in a source distribution. This meant that when installing a tar.gz from PyPI, you weren't getting the PEP518 build system definition, just the vanilla setuptools build process with no compile time dependency resolution.

I've explicitly added the pyproject.toml files to ms_deisotope and ms_peak_picker in new PyPI releases, so that shouldn't happen. I've also removed the diagnostic print statements as you're right, they did produce misleading beliefs.

[MKoesters]

Thanks for taking this over @mobiusklein!

I'll close this issue for now, since it seems to be resolved

[ZhixuNi]

This issue is fixed in my test env with: ms-deisotope-0.0.16 ms-peak-picker-0.1.28 Thanks @mobiusklein!

[weantony]

I think I've found the problem. Older versions of setuptools do not automatically include the pyproject.toml file in a source distribution. This meant that when installing a tar.gz from PyPI, you weren't getting the PEP518 build system definition, just the vanilla setuptools build process with no compile time dependency resolution.

I've explicitly added the pyproject.toml files to ms_deisotope and ms_peak_picker in new PyPI releases, so that shouldn't happen. I've also removed the diagnostic print statements as you're right, they did produce misleading beliefs.

Hi mobiusklein,

I can still found the same warning on my end. Problem is resolved by upgrading ms-deisotope. Just want to share the steps. Pypzml was downloaded and installed just last week. It looks like pymzml could work but it will show warning during import pymzml. Please see the warning below.

No module named 'ms_deisotope._c.averagine' averagine No module named 'ms_deisotope._c.scoring' No module named 'ms_deisotope._c.deconvoluter_base' No module named 'ms_deisotope._c.deconvoluter_base' No module named 'ms_deisotope._c.deconvoluter_base'

I tried installed ms-deisotope, it looks like its already installed with version 0.0.14. The message is below.

C:\Users\weant>python -m pip install ms-deisotope Requirement already satisfied: ms-deisotope in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (0.0.14) Requirement already satisfied: dill in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.3.3) Requirement already satisfied: lxml in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (4.6.1) Requirement already satisfied: scipy in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (1.5.3) Requirement already satisfied: psims>=0.1.28 in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.1.31) Requirement already satisfied: brain-isotopic-distribution in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (1.5.4) Requirement already satisfied: numpy in c:\users\weant\appdata\roaming\python\python38\site-packages (from ms-deisotope) (1.19.2) Requirement already satisfied: ms-peak-picker in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.1.29) Requirement already satisfied: pyteomics>=4.0 in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (4.3.3) Requirement already satisfied: six in c:\users\weant\appdata\roaming\python\python38\site-packages (from ms-deisotope) (1.15.0) Requirement already satisfied: sqlalchemy in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from psims>=0.1.28->ms-deisotope) (1.3.20)

Then I tried to upgrade ms-deisotope with -m pip install --upgrade ms-deisotope. It is upgraded to 0.0.17. The message is below.

C:\Users\weant>python -m pip install --upgrade ms-deisotope Collecting ms-deisotope Downloading ms_deisotope-0.0.17-cp38-cp38-win_amd64.whl (4.2 MB) |████████████████████████████████| 4.2 MB 23 kB/s Requirement already satisfied, skipping upgrade: six in c:\users\weant\appdata\roaming\python\python38\site-packages (from ms-deisotope) (1.15.0) Requirement already satisfied, skipping upgrade: brain-isotopic-distribution in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (1.5.4) Requirement already satisfied, skipping upgrade: psims>=0.1.28 in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.1.31) Requirement already satisfied, skipping upgrade: lxml in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (4.6.1) Requirement already satisfied, skipping upgrade: numpy in c:\users\weant\appdata\roaming\python\python38\site-packages (from ms-deisotope) (1.19.2) Requirement already satisfied, skipping upgrade: pyteomics>=4.0 in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (4.3.3) Requirement already satisfied, skipping upgrade: ms-peak-picker in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.1.29) Requirement already satisfied, skipping upgrade: dill in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (0.3.3) Requirement already satisfied, skipping upgrade: scipy in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from ms-deisotope) (1.5.3) Requirement already satisfied, skipping upgrade: sqlalchemy in c:\users\weant\appdata\local\programs\python\python38\lib\site-packages (from psims>=0.1.28->ms-deisotope) (1.3.20) Installing collected packages: ms-deisotope Attempting uninstall: ms-deisotope Found existing installation: ms-deisotope 0.0.14 Uninstalling ms-deisotope-0.0.14: Successfully uninstalled ms-deisotope-0.0.14 Successfully installed ms-deisotope-0.0.17

Then the warnings are gone. Just to share the information and steps with you guys. Hope could help others with same problem