search

picrust / picrust2

Code, unit tests, and tutorials for running PICRUSt2
GNU General Public License v3.0
306 stars 104 forks source link

Macbook M1 chip pytest error (sqlite3 import??) #342

Open SinaedaA opened 5 months ago

SinaedaA commented 5 months ago

Hi !

I was trying to download PICRUSt2 on my Macbook M1, using micromamba, and I encountered some problems which I tried to solve independently, but in the end the pytest didn't work.

At first, I tried installing with micromamba create -n picrust2 -c bioconda -c conda-forge picrust2=2.5.2, but this caused problems because of r-castor and biom-format. Therefore, I tried again but to install from source, and commenting out these 2 packages from the .yaml file. Here's what I did : 

wget https://github.com/picrust/picrust2/archive/picrust2-2.5.2.tar.gz
tar -xvzf picrust2-2.5.2.tar.gz
cd picrust2-2.5.2/

## commented out 2 lines in yaml which caused problems: - r-castor and - biom-format
micromamba env create -f picrust2-env.yaml
micromamba activate picrust2
pip install --editable .
pip install biom-format
conda install r-castor -c conda-forge

The biom-format didn't appear in installed packages inside the environment, so I re-installed it with conda as well, and it worked (conda install biom-format -c bioconda).

pytest

Returns:

Traceback (most recent call last):
  File "/Users/sinaeda/micromamba/envs/picrust2/bin/pytest", line 10, in <module>
    sys.exit(console_main())
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 197, in console_main
    code = main()
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 155, in main
    config = _prepareconfig(args, plugins)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 337, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_hooks.py", line 501, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_manager.py", line 119, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_callers.py", line 138, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_callers.py", line 121, in _multicall
    teardown.throw(exception)  # type: ignore[union-attr]
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/helpconfig.py", line 104, in pytest_cmdline_parse
    config = yield
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_callers.py", line 102, in _multicall
    res = hook_impl.function(*args)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1094, in pytest_cmdline_parse
    self.parse(args)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1446, in parse
    self._preparse(args, addopts=addopts)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/config/__init__.py", line 1324, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pluggy/_manager.py", line 414, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/_pytest/assertion/rewrite.py", line 175, in exec_module
    exec(co, module.__dict__)
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/pytest_cov/plugin.py", line 6, in <module>
    import coverage
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/coverage/__init__.py", line 24, in <module>
    from coverage.control import (
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/coverage/control.py", line 28, in <module>
    from coverage.collector import Collector, HAS_CTRACER
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/coverage/collector.py", line 19, in <module>
    from coverage.data import CoverageData
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/coverage/data.py", line 24, in <module>
    from coverage.sqldata import CoverageData
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/site-packages/coverage/sqldata.py", line 16, in <module>
    import sqlite3
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/sqlite3/__init__.py", line 23, in <module>
    from sqlite3.dbapi2 import *
  File "/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/sqlite3/dbapi2.py", line 27, in <module>
    from _sqlite3 import *
ImportError: dlopen(/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/lib-dynload/_sqlite3.cpython-38-darwin.so, 0x0002): Symbol not found: (_sqlite3_enable_load_extension)
  Referenced from: '/Users/sinaeda/micromamba/envs/picrust2/lib/python3.8/lib-dynload/_sqlite3.cpython-38-darwin.so'
  Expected in: '/usr/lib/libsqlite3.dylib'

It seems to have a problem importing sqlite3? I'm not entirely sure. Therefore, I also re-installed sqlite with conda install sqlite but this didn't change the output of pytest.

Here are some of my machine specifications:

Output of micromamba list

List of packages in environment: "/Users/sinaeda/micromamba/envs/picrust2"

  Name                         Version       Build                   Channel    
──────────────────────────────────────────────────────────────────────────────────
  _r-mutex                     1.0.1         anacondar_1             conda-forge
  biom-format                  2.1.15        py38hae2e43d_1          conda-forge
  brotli-python                1.1.0         py38he333c0f_1          conda-forge
  bwidget                      1.9.14        hce30654_1              conda-forge
  bzip2                        1.0.8         h93a5062_5              conda-forge
  c-ares                       1.26.0        h93a5062_0              conda-forge
  ca-certificates              2024.2.2      hf0a4a13_0              conda-forge
  cached-property              1.5.2         hd8ed1ab_1              conda-forge
  cached_property              1.5.2         pyha770c72_1            conda-forge
  cairo                        1.16.0        h3ce6f7e_5                         
  cctools                      921           h5ba7a2e_4              conda-forge
  certifi                      2024.2.2      pyhd8ed1ab_0            conda-forge
  charset-normalizer           3.3.2         pyhd8ed1ab_0            conda-forge
  clang                        9.0.0         default_hf57f61e_4      conda-forge
  clang_osx-64                 9.0.0         h22b1bf0_3              conda-forge
  clangxx                      9.0.0         default_hf57f61e_4      conda-forge
  clangxx_osx-64               9.0.0         h05bbb7f_6              conda-forge
  click                        8.1.3         py38h50d1736_1          conda-forge
  colorama                     0.4.6         pyhd8ed1ab_0            conda-forge
  compiler-rt                  9.0.0         h2b4a0d1_1              conda-forge
  coverage                     7.4.1         py38h336bac9_0          conda-forge
  curl                         7.68.0        h8754def_0              conda-forge
  cython                       3.0.8         py38hb9fa5a8_0          conda-forge
  dendropy                     4.6.1         pyhdfd78af_0            bioconda   
  epa-ng                       0.3.8         hf7ba07b_4              bioconda   
  exceptiongroup               1.2.0         pyhd8ed1ab_2            conda-forge
  expat                        2.5.0         hb7217d7_1              conda-forge
  font-ttf-dejavu-sans-mono    2.37          hab24e00_0              conda-forge
  font-ttf-inconsolata         3.000         h77eed37_0              conda-forge
  font-ttf-source-code-pro     2.038         h77eed37_0              conda-forge
  font-ttf-ubuntu              0.83          h77eed37_1              conda-forge
  fontconfig                   2.14.2        h82840c6_0              conda-forge
  fonts-conda-ecosystem        1             0                       conda-forge
  fonts-conda-forge            1             0                       conda-forge
  freetype                     2.12.1        hadb7bae_2              conda-forge
  fribidi                      1.0.10        h27ca646_0              conda-forge
  gappa                        0.8.0         hf7ba07b_3              bioconda   
  gettext                      0.21.1        h0186832_0              conda-forge
  gfortran_impl_osx-64         7.5.0         hae4d780_7              conda-forge
  gfortran_osx-64              7.5.0         h22b1bf0_0              conda-forge
  glib                         2.69.1        hfff2838_2                         
  glpk                         5.0           h6d7a090_0              conda-forge
  gmp                          6.3.0         h965bd2d_0              conda-forge
  graphite2                    1.3.13        h9f76cd9_1001           conda-forge
  gsl                          2.5           ha2d443c_1              conda-forge
  h5py                         3.3.0         nompi_py38h9a16e60_100  conda-forge
  harfbuzz                     2.4.0         hd8d2a14_3              conda-forge
  hdf5                         1.10.6        nompi_haae91d6_101      conda-forge
  hmmer                        3.1b2         3                       bioconda   
  icu                          64.2          h6de7cb9_1              conda-forge
  idna                         3.6           pyhd8ed1ab_0            conda-forge
  iniconfig                    2.0.0         pyhd8ed1ab_0            conda-forge
  isl                          0.22.1        hb1e8313_2              conda-forge
  jinja2                       3.1.3         pyhd8ed1ab_0            conda-forge
  joblib                       1.3.2         pyhd8ed1ab_0            conda-forge
  jpeg                         9e            hb7f2c08_3              conda-forge
  krb5                         1.16.4        h1752a42_0              conda-forge
  ld64                         274.2         h7c2db76_0              conda-forge
  lerc                         4.0.0         h9a09cb3_0              conda-forge
  libaec                       1.1.2         h13dd4ca_1              conda-forge
  libblas                      3.9.0         8_openblas              conda-forge
  libcblas                     3.9.0         8_openblas              conda-forge
  libcurl                      7.68.0        h709d2b2_0              conda-forge
  libcxx                       16.0.6        h4653b0c_0              conda-forge
  libdeflate                   1.19          hb547adb_0              conda-forge
  libedit                      3.1.20191231  h0678c8f_2              conda-forge
  libev                        4.33          h93a5062_2              conda-forge
  libexpat                     2.5.0         hb7217d7_1              conda-forge
  libffi                       3.4.2         h3422bc3_5              conda-forge
  libgfortran                  4.0.0         7_5_0_h1a10cd1_23       conda-forge
  libgfortran-devel_osx-arm64  12.3.0        hc62be1c_3              conda-forge
  libgfortran4                 7.5.0         h1a10cd1_23             conda-forge
  libiconv                     1.17          h0d3ecfb_2              conda-forge
  liblapack                    3.9.0         8_openblas              conda-forge
  libllvm9                     9.0.1         default_h2671707_7      conda-forge
  libopenblas                  0.3.12        openmp_h63d9170_1       conda-forge
  libpng                       1.6.42        h091b4b1_0              conda-forge
  libsqlite                    3.45.1        h091b4b1_0              conda-forge
  libssh2                      1.10.0        h7535e13_3              conda-forge
  libtiff                      4.2.0         h1167814_3              conda-forge
  libwebp-base                 1.3.2         hb547adb_0              conda-forge
  libxml2                      2.9.10        h53d96d6_0              conda-forge
  libzlib                      1.2.13        h53f4e23_5              conda-forge
  llvm-lto-tapi                4.0.1         h6701bc3_0              conda-forge
  llvm-openmp                  17.0.6        hcd81f8e_0              conda-forge
  make                         4.3           he57ea6c_1              conda-forge
  markupsafe                   2.1.5         py38h336bac9_0          conda-forge
  mpc                          1.3.1         h91ba8db_0              conda-forge
  mpfr                         4.2.1         h9546428_0              conda-forge
  ncurses                      6.4           h463b476_2              conda-forge
  numpy                        1.24.4        py38ha84db1f_0          conda-forge
  openjdk                      21.0.2        hbeb2e11_0              conda-forge
  openssl                      1.1.1w        h8a1eda9_0              conda-forge
  packaging                    23.2          pyhd8ed1ab_0            conda-forge
  pandas                       2.0.3         py38hefb543e_1          conda-forge
  pango                        1.42.4        hc249df1_1                         
  pcre                         8.45          he49afe7_0              conda-forge
  pcre2                        10.42         h26f9a81_0              conda-forge
  pip                          24.0          pyhd8ed1ab_0            conda-forge
  pixman                       0.43.2        hebf3989_0              conda-forge
  platformdirs                 4.2.0         pyhd8ed1ab_0            conda-forge
  pluggy                       1.4.0         pyhd8ed1ab_0            conda-forge
  pooch                        1.8.0         pyhd8ed1ab_0            conda-forge
  pysocks                      1.7.1         pyha2e5f31_6            conda-forge
  pytest                       8.0.0         pyhd8ed1ab_0            conda-forge
  pytest-cov                   4.1.0         pyhd8ed1ab_0            conda-forge
  python                       3.8.15        hc915b28_0_cpython      conda-forge
  python-dateutil              2.8.2         pyhd8ed1ab_0            conda-forge
  python-tzdata                2024.1        pyhd8ed1ab_0            conda-forge
  python_abi                   3.8           4_cp38                  conda-forge
  pytz                         2024.1        pyhd8ed1ab_0            conda-forge
  r-base                       3.5.1         hc03ab29_1012           conda-forge
  r-castor                     1.5.7         r35hc5da6b9_0           conda-forge
  r-naturalsort                0.1.3         r351_1000               conda-forge
  r-nloptr                     1.2.2.1       r35hc5da6b9_0           conda-forge
  r-rcpp                       1.0.4.6       r35hc5da6b9_0           conda-forge
  readline                     8.2           h92ec313_1              conda-forge
  requests                     2.31.0        pyhd8ed1ab_0            conda-forge
  scipy                        1.5.3         py38h352ea5d_0          conda-forge
  sepp                         4.3.10        py38h3252c3a_2          bioconda   
  setuptools                   69.0.3        pyhd8ed1ab_0            conda-forge
  six                          1.16.0        pyh6c4a22f_0            conda-forge
  sqlite                       3.41.2        h80987f9_0              anaconda   
  tapi                         1100.0.11     he4954df_0              conda-forge
  tk                           8.6.13        h5083fa2_1              conda-forge
  tktable                      2.10          hd996620_5              conda-forge
  toml                         0.10.2        pyhd8ed1ab_0            conda-forge
  tomli                        2.0.1         pyhd8ed1ab_0            conda-forge
  urllib3                      2.2.0         pyhd8ed1ab_0            conda-forge
  wheel                        0.42.0        pyhd8ed1ab_0            conda-forge
  xz                           5.2.6         h57fd34a_0              conda-forge
  zlib                         1.2.13        h53f4e23_5              conda-forge
  zstd                         1.5.5         h4f39d0f_0              conda-forge

I would really appreciate some help with this ^^

Thank you for any advice you might have ! Kind regards, Sina

R-Wright-1 commented 4 months ago

Hi Sina,

Apologies for the slow response. Have you tried actually running PICRUSt2? Does it work or do you still get errors? pytest will only work if you have either installed from source or also downloaded the source code and navigated to the main folder (see bottom of this page). Let me know if it doesn't work/what the errors are when you try to run it with your own data.

Note: I initially wrote the above earlier, and then looked into it a bit further with this issue: #314. Maybe you can try following the steps that I used there to install it on my M1 Macbook Air?

Robyn