bschilder
commented
3 years ago Also tried a version of this where I installed pegasuspy via conda (instead of pip), but got the same error message as above. pegasus.yml.txt
Closed bschilder closed 3 years ago
bschilder
commented
3 years ago Also tried a version of this where I installed pegasuspy via conda (instead of pip), but got the same error message as above. pegasus.yml.txt
yihming
commented
3 years ago Hi @bschilder . Thank you for bringing this issue up!
For the compatibility issue with umap v0.5+, we'll adapt Pegasus' API in the next release.
For the conda environment, it's actually more complicated. One of Pegasus' dependencies, hnswlib, has the issue on compilation across platforms of different CPU models. This is why Pegasus conda version is 1.2.0, not the latest stable v1.3.0, because its build failed on conda-forge platform. A systematic way of resolving it in our plan is to update hnswlib by resolving its issue on conda. But it can take time.
yihming
commented
3 years ago Actually, besides the incompatibility with umap v0.5+, you also need to restrict your numba version no later than 0.52.0.
bschilder
commented
3 years ago Ah right, you totally mentioned the numba issue to me earlier and forgot to add it. Here's the corrected yaml. Using this yaml, the conda env seems to be working fine!
The only exception so far is when i try to use Harmony:
2021-04-23 20:45:57,439 - pegasus.tools.batch_correction - INFO - Start integration using Harmony.
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-20-a2534b9d859e> in <module>
----> 1 harmony_key = pg.run_harmony(data_harmony)
~/anaconda3/envs/pegasus/lib/python3.8/site-packages/pegasusio/decorators.py in wrapper_timer(*args, **kwargs)
10 def wrapper_timer(*args, **kwargs):
11 start = time.perf_counter()
---> 12 result = func(*args, **kwargs)
13 end = time.perf_counter()
14 message = f"Function '{func.__name__}' finished in {{:.{precision}f}}s.".format(end - start)
~/anaconda3/envs/pegasus/lib/python3.8/site-packages/pegasus/tools/batch_correction.py in run_harmony(data, rep, n_jobs, n_clusters, random_state)
247 logger.info("Start integration using Harmony.")
248 out_rep = rep + '_harmony'
--> 249 data.obsm['X_' + out_rep] = harmonize(X_from_rep(data, rep), data.obs, 'Channel', n_clusters = n_clusters, n_jobs_kmeans = n_jobs, random_state = random_state)
250 return out_rep
251
TypeError: harmonize() got an unexpected keyword argument 'n_jobs_kmeans'This also seems to be a version conflict issue. Do you have a recommended max version of Harmony?
yihming
commented
3 years ago Thank you for working on this issue!
The incompatibility with harmony-pytorch is because we've changed API a little bit in its latest version v0.1.6, so it won't be compatible with pegasus v1.2.0. For now, it would be easier to use an older version of harmony-pytorch, say v0.1.5.
bschilder
commented
3 years ago Great! Thanks, added this dep to the conda yaml as well. Feel free to include this in the GitHub repo if you think people will find it helpful.
hoondy
commented
3 years ago Actually, besides the incompatibility with umap v0.5+, you also need to restrict your
numbaversion no later than 0.52.0.
I was having trouble loading pegasuspy with errors related to numba and umap. downgrading numba to 0.52 resolved issue. I think it would be helpful to revise https://github.com/klarman-cell-observatory/pegasus/blob/master/requirements.txt to be explicit.
- numba<=0.52.0 Hello @hoondy ,
Which version of Pegasuspy you were running? The current release v1.4.3 is already compatible with umap-learn v0.5+. And numba's version later than 0.52.0 also works well with it in my test.
hoondy
commented
3 years ago Hello @hoondy ,
Which version of Pegasuspy you were running? The current release v1.4.3 is already compatible with umap-learn v0.5+. And numba's version later than 0.52.0 also works well with it in my test.
I was using the latest v1.4.3. It was working when I upgraded from v1.3.0 but I had to wipe and reinstall pegasus v1.4.3 from scratch. On the new installation, I had numba-0.53.1 and the following is the error I saw upon calling import pegasus as pg. Upon downgrading to 0.52.0, it worked again.
I am currently using
numba 0.52.0 py38h51da96c_0 conda-forge
pegasusio 0.3.1.post2 pypi_0 pypi
pegasuspy 1.4.3 pypi_0 pypi
umap-learn 0.5.1 pypi_0 pypi---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/errors.py in new_error_context(fmt_, *args, **kwargs)
743 try:
--> 744 yield
745 except NumbaError as e:
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower_block(self, block)
229 loc=self.loc, errcls_=defaulterrcls):
--> 230 self.lower_inst(inst)
231 self.post_block(block)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower_inst(self, inst)
327 val = self.lower_assign(ty, inst)
--> 328 self.storevar(val, inst.target.name)
329
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in storevar(self, value, name)
1277 name=name)
-> 1278 raise AssertionError(msg)
1279
AssertionError: Storing i64 to ptr of i32 ('dim'). FE type int32
During handling of the above exception, another exception occurred:
LoweringError Traceback (most recent call last)
/tmp/ipykernel_320047/2641037559.py in <module>
----> 1 import pegasus as pg
2 import matplotlib.pyplot as plt
3 import seaborn as sns
4 import pandas as pd
5 # import anndata
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/pegasus/__init__.py in <module>
24 from pegasusio import infer_file_type, read_input, write_output, aggregate_matrices
25 from demuxEM import estimate_background_probs, demultiplex, attach_demux_results
---> 26 from .tools import (
27 qc_metrics,
28 get_filter_stats,
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/pegasus/tools/__init__.py in <module>
36 from .clustering import jump_method, louvain, leiden, spectral_louvain, spectral_leiden, cluster, split_one_cluster
37 from .net_regressor import net_train_and_predict
---> 38 from .visualization import (
39 tsne,
40 umap,
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/pegasus/tools/visualization.py in <module>
2 import numpy as np
3 import scipy
----> 4 import umap as umap_module
5 import forceatlas2 as fa2
6 import uuid
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/umap/__init__.py in <module>
----> 1 from .umap_ import UMAP
2
3 # Workaround: https://github.com/numba/numba/issues/3341
4 import numba
5
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/umap/umap_.py in <module>
52 from umap.spectral import spectral_layout
53 from umap.utils import deheap_sort, submatrix
---> 54 from umap.layouts import (
55 optimize_layout_euclidean,
56 optimize_layout_generic,
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/umap/layouts.py in <module>
37 },
38 )
---> 39 def rdist(x, y):
40 """Reduced Euclidean distance.
41
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/decorators.py in wrapper(func)
219 with typeinfer.register_dispatcher(disp):
220 for sig in sigs:
--> 221 disp.compile(sig)
222 disp.disable_compile()
223 return disp
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/dispatcher.py in compile(self, sig)
907 with ev.trigger_event("numba:compile", data=ev_details):
908 try:
--> 909 cres = self._compiler.compile(args, return_type)
910 except errors.ForceLiteralArg as e:
911 def folded(args, kws):
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/dispatcher.py in compile(self, args, return_type)
77
78 def compile(self, args, return_type):
---> 79 status, retval = self._compile_cached(args, return_type)
80 if status:
81 return retval
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/dispatcher.py in _compile_cached(self, args, return_type)
91
92 try:
---> 93 retval = self._compile_core(args, return_type)
94 except errors.TypingError as e:
95 self._failed_cache[key] = e
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/dispatcher.py in _compile_core(self, args, return_type)
104
105 impl = self._get_implementation(args, {})
--> 106 cres = compiler.compile_extra(self.targetdescr.typing_context,
107 self.targetdescr.target_context,
108 impl,
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler.py in compile_extra(typingctx, targetctx, func, args, return_type, flags, locals, library, pipeline_class)
604 pipeline = pipeline_class(typingctx, targetctx, library,
605 args, return_type, flags, locals)
--> 606 return pipeline.compile_extra(func)
607
608
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler.py in compile_extra(self, func)
351 self.state.lifted = ()
352 self.state.lifted_from = None
--> 353 return self._compile_bytecode()
354
355 def compile_ir(self, func_ir, lifted=(), lifted_from=None):
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler.py in _compile_bytecode(self)
413 """
414 assert self.state.func_ir is None
--> 415 return self._compile_core()
416
417 def _compile_ir(self):
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler.py in _compile_core(self)
393 self.state.status.fail_reason = e
394 if is_final_pipeline:
--> 395 raise e
396 else:
397 raise CompilerError("All available pipelines exhausted")
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler.py in _compile_core(self)
384 res = None
385 try:
--> 386 pm.run(self.state)
387 if self.state.cr is not None:
388 break
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler_machinery.py in run(self, state)
337 (self.pipeline_name, pass_desc)
338 patched_exception = self._patch_error(msg, e)
--> 339 raise patched_exception
340
341 def dependency_analysis(self):
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler_machinery.py in run(self, state)
328 pass_inst = _pass_registry.get(pss).pass_inst
329 if isinstance(pass_inst, CompilerPass):
--> 330 self._runPass(idx, pass_inst, state)
331 else:
332 raise BaseException("Legacy pass in use")
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler_lock.py in _acquire_compile_lock(*args, **kwargs)
33 def _acquire_compile_lock(*args, **kwargs):
34 with self:
---> 35 return func(*args, **kwargs)
36 return _acquire_compile_lock
37
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler_machinery.py in _runPass(self, index, pss, internal_state)
287 mutated |= check(pss.run_initialization, internal_state)
288 with SimpleTimer() as pass_time:
--> 289 mutated |= check(pss.run_pass, internal_state)
290 with SimpleTimer() as finalize_time:
291 mutated |= check(pss.run_finalizer, internal_state)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/compiler_machinery.py in check(func, compiler_state)
260
261 def check(func, compiler_state):
--> 262 mangled = func(compiler_state)
263 if mangled not in (True, False):
264 msg = ("CompilerPass implementations should return True/False. "
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/typed_passes.py in run_pass(self, state)
461
462 # TODO: Pull this out into the pipeline
--> 463 NativeLowering().run_pass(state)
464 lowered = state['cr']
465 signature = typing.signature(state.return_type, *state.args)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/typed_passes.py in run_pass(self, state)
382 lower = lowering.Lower(targetctx, library, fndesc, interp,
383 metadata=metadata)
--> 384 lower.lower()
385 if not flags.no_cpython_wrapper:
386 lower.create_cpython_wrapper(flags.release_gil)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower(self)
134 if self.generator_info is None:
135 self.genlower = None
--> 136 self.lower_normal_function(self.fndesc)
137 else:
138 self.genlower = self.GeneratorLower(self)
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower_normal_function(self, fndesc)
188 # Init argument values
189 self.extract_function_arguments()
--> 190 entry_block_tail = self.lower_function_body()
191
192 # Close tail of entry block
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower_function_body(self)
214 bb = self.blkmap[offset]
215 self.builder.position_at_end(bb)
--> 216 self.lower_block(block)
217 self.post_lower()
218 return entry_block_tail
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/lowering.py in lower_block(self, block)
228 with new_error_context('lowering "{inst}" at {loc}', inst=inst,
229 loc=self.loc, errcls_=defaulterrcls):
--> 230 self.lower_inst(inst)
231 self.post_block(block)
232
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/contextlib.py in __exit__(self, type, value, traceback)
129 value = type()
130 try:
--> 131 self.gen.throw(type, value, traceback)
132 except StopIteration as exc:
133 # Suppress StopIteration *unless* it's the same exception that
/xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/numba/core/errors.py in new_error_context(fmt_, *args, **kwargs)
749 newerr = errcls(e).add_context(_format_msg(fmt_, args, kwargs))
750 tb = sys.exc_info()[2] if numba.core.config.FULL_TRACEBACKS else None
--> 751 raise newerr.with_traceback(tb)
752
753
LoweringError: Failed in nopython mode pipeline (step: nopython mode backend)
Storing i64 to ptr of i32 ('dim'). FE type int32
File "../../../xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/umap/layouts.py", line 52:
def rdist(x, y):
<source elided>
result = 0.0
dim = x.shape[0]
^
During: lowering "dim = static_getitem(value=$8load_attr.2, index=0, index_var=$const10.3, fn=<built-in function getitem>)" at /xx/xxx/projects/xxx/xxx/conda/envs/pegasus/lib/python3.8/site-packages/umap/layouts.py (52)
I see that your numba was installed from conda. But after trying to repeat your conda environment, pegasuspy was still able to be imported. My test was on both macOS and Ubuntu 20.04 with miniconda3.
Could you please send me the output of conda list command of your conda environment? Also, which OS are you using?
hoondy
commented
3 years ago Thanks for looking into this. I may have cause a corner case when I was installing other single cell tools. After seeing the error, I manually installed numba using conda install -c anaconda numba=0.52.0. I tested this on CentOS 7, using miniconda3. Here's output of conda list:
(pegasus) [xxx@xxx analysis_pegasus]$ conda list
# packages in environment at /xx/xxx/projects/xxx/xx/conda/envs/pegasus:
#
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
_r-mutex 1.0.1 anacondar_1 conda-forge
anndata 0.7.6 py38h578d9bd_0 conda-forge
appdirs 1.4.4 pypi_0 pypi
arpack 3.7.0 hdefa2d7_2 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 py_2 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
backports.zoneinfo 0.2.1 py38h497a2fe_4 conda-forge
beautifulsoup4 4.9.3 pypi_0 pypi
binutils_impl_linux-64 2.36.1 h193b22a_2 conda-forge
binutils_linux-64 2.36 hf3e587d_0 conda-forge
bioservices 1.7.11 pypi_0 pypi
biothings-client 0.2.6 pypi_0 pypi
blosc 1.21.0 h9c3ff4c_0 conda-forge
bwidget 1.9.14 ha770c72_0 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.17.2 h7f98852_0 conda-forge
ca-certificates 2020.10.14 0 anaconda
cached-property 1.5.2 hd8ed1ab_1 conda-forge
cached_property 1.5.2 pyha770c72_1 conda-forge
cairo 1.16.0 h9f066cc_1006 conda-forge
certifi 2020.6.20 py38_0 anaconda
cffi 1.14.4 py38ha312104_0 conda-forge
chardet 4.0.0 pypi_0 pypi
click 8.0.1 pypi_0 pypi
colorama 0.4.4 pypi_0 pypi
colorlog 5.0.1 pypi_0 pypi
curl 7.78.0 hea6ffbf_0 conda-forge
cycler 0.10.0 py_2 conda-forge
cython 0.29.23 pypi_0 pypi
debugpy 1.4.1 py38h709712a_0 conda-forge
decorator 4.4.2 py_0 anaconda
dunamai 1.5.5 pypi_0 pypi
easydev 0.11.1 pypi_0 pypi
entrypoints 0.3 pyhd8ed1ab_1003 conda-forge
fontconfig 2.13.1 hba837de_1005 conda-forge
freetype 2.10.4 h0708190_1 conda-forge
fribidi 1.0.10 h36c2ea0_0 conda-forge
gcc_impl_linux-64 9.4.0 h03d3576_8 conda-forge
gcc_linux-64 9.4.0 h391b98a_0 conda-forge
get-version 3.2 pypi_0 pypi
gettext 0.19.8.1 hf34092f_1004 conda-forge
gevent 21.1.2 pypi_0 pypi
gfortran_impl_linux-64 9.4.0 h0003116_8 conda-forge
gfortran_linux-64 9.4.0 hf0ab688_0 conda-forge
glpk 4.65 h9202a9a_1004 conda-forge
gmp 6.2.1 h58526e2_0 conda-forge
graphite2 1.3.13 h58526e2_1001 conda-forge
greenlet 1.1.0 pypi_0 pypi
grequests 0.6.0 pypi_0 pypi
gseapy 0.10.4 pypi_0 pypi
gsl 2.6 he838d99_2 conda-forge
gxx_impl_linux-64 9.4.0 h03d3576_8 conda-forge
gxx_linux-64 9.4.0 h0316aca_0 conda-forge
h5py 3.4.0 nompi_py38hfbb2109_100 conda-forge
harfbuzz 2.7.2 ha5b49bf_1 conda-forge
hdf5 1.12.1 nompi_h2750804_100 conda-forge
icu 67.1 he1b5a44_0 conda-forge
idna 2.10 pypi_0 pypi
igraph 0.9.2 ha184e22_0 conda-forge
ipykernel 6.3.1 py38he5a9106_0 conda-forge
ipython 7.27.0 py38he5a9106_0 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
itsdangerous 2.0.1 pypi_0 pypi
jbig 2.1 h7f98852_2003 conda-forge
jedi 0.18.0 py38h578d9bd_2 conda-forge
jinja2 3.0.1 pyhd8ed1ab_0 conda-forge
joblib 1.0.1 pyhd8ed1ab_0 conda-forge
jpeg 9d h36c2ea0_0 conda-forge
jupyter_client 7.0.2 pyhd8ed1ab_0 conda-forge
jupyter_core 4.7.1 py38h578d9bd_0 conda-forge
kernel-headers_linux-64 2.6.32 he073ed8_14 conda-forge
kiwisolver 1.3.1 pypi_0 pypi
krb5 1.19.2 hcc1bbae_0 conda-forge
lcms2 2.12 hddcbb42_0 conda-forge
ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge
legacy-api-wrap 1.2 pypi_0 pypi
leidenalg 0.8.4 pypi_0 pypi
lerc 2.2.1 h9c3ff4c_0 conda-forge
libblas 3.9.0 11_linux64_openblas conda-forge
libcblas 3.9.0 11_linux64_openblas conda-forge
libcurl 7.78.0 h2574ce0_0 conda-forge
libdeflate 1.7 h7f98852_5 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.2.1 he1b5a44_1007 conda-forge
libgcc 7.2.0 h69d50b8_2 conda-forge
libgcc-devel_linux-64 9.4.0 hd854feb_8 conda-forge
libgcc-ng 11.1.0 hc902ee8_8 conda-forge
libgfortran-ng 11.1.0 h69a702a_8 conda-forge
libgfortran5 11.1.0 h6c583b3_8 conda-forge
libglib 2.66.3 hbe7bbb4_0 conda-forge
libgomp 11.1.0 hc902ee8_8 conda-forge
libiconv 1.16 h516909a_0 conda-forge
liblapack 3.9.0 11_linux64_openblas conda-forge
libllvm10 10.0.1 he513fc3_3 conda-forge
libnghttp2 1.43.0 h812cca2_0 conda-forge
libopenblas 0.3.17 pthreads_h8fe5266_1 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libsanitizer 9.4.0 h79bfe98_8 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libssh2 1.10.0 ha56f1ee_0 conda-forge
libstdcxx-devel_linux-64 9.4.0 hd854feb_8 conda-forge
libstdcxx-ng 11.1.0 h56837e0_8 conda-forge
libtiff 4.3.0 hf544144_1 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libwebp-base 1.2.1 h7f98852_0 conda-forge
libxcb 1.13 h7f98852_1003 conda-forge
libxml2 2.9.10 h68273f3_2 conda-forge
llvmlite 0.35.0 pypi_0 pypi
lxml 4.6.3 pypi_0 pypi
lz4-c 1.9.3 h9c3ff4c_1 conda-forge
lzo 2.10 h516909a_1000 conda-forge
make 4.3 hd18ef5c_1 conda-forge
markupsafe 2.0.1 py38h497a2fe_0 conda-forge
matplotlib 3.4.2 pypi_0 pypi
matplotlib-base 3.4.3 py38hf4fb855_0 conda-forge
matplotlib-inline 0.1.2 pyhd8ed1ab_2 conda-forge
metis 5.1.0 h58526e2_1006 conda-forge
mock 4.0.3 py38h578d9bd_1 conda-forge
mpfr 4.1.0 h9202a9a_1 conda-forge
mygene 3.2.2 pypi_0 pypi
natsort 7.1.1 pyhd8ed1ab_0 conda-forge
ncurses 6.2 h58526e2_4 conda-forge
nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge
networkx 2.5.1 pypi_0 pypi
nmf-torch 0.1.1 pypi_0 pypi
numba 0.52.0 py38h51da96c_0 conda-forge
numexpr 2.7.3 py38h51da96c_0 conda-forge
numpy 1.20.3 pypi_0 pypi
olefile 0.46 pyh9f0ad1d_1 conda-forge
openjpeg 2.4.0 hb52868f_1 conda-forge
openssl 1.1.1l h7f98852_0 conda-forge
packaging 20.9 pypi_0 pypi
pandas 1.3.2 py38h43a58ef_0 conda-forge
pango 1.42.4 h69149e4_5 conda-forge
parso 0.8.2 pyhd8ed1ab_0 conda-forge
patsy 0.5.1 py_0 conda-forge
pcre 8.45 h9c3ff4c_0 conda-forge
pcre2 10.36 h032f7d1_1 conda-forge
pegasusio 0.3.1.post2 pypi_0 pypi
pegasuspy 1.4.3 pypi_0 pypi
pexpect 4.8.0 pyh9f0ad1d_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 8.2.0 pypi_0 pypi
pip 21.2.4 pyhd8ed1ab_0 conda-forge
pixman 0.40.0 h36c2ea0_0 conda-forge
prompt-toolkit 3.0.20 pyha770c72_0 conda-forge
psutil 5.8.0 pypi_0 pypi
pthread-stubs 0.4 h36c2ea0_1001 conda-forge
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pycparser 2.20 pyh9f0ad1d_2 conda-forge
pygments 2.10.0 pyhd8ed1ab_0 conda-forge
pynndescent 0.5.2 pypi_0 pypi
pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge
pytables 3.6.1 py38hdb04529_4 conda-forge
python 3.8.0 h357f687_5 conda-forge
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
python-igraph 0.9.1 py38h2af5540_0 conda-forge
python_abi 3.8 2_cp38 conda-forge
pytz 2021.1 pyhd8ed1ab_0 conda-forge
pyzmq 22.2.1 py38h2035c66_0 conda-forge
r-base 3.2.2 0
readline 8.1 h46c0cb4_0 conda-forge
requests 2.25.1 pypi_0 pypi
requests-cache 0.6.4 pypi_0 pypi
scanorama 1.7.1 pypi_0 pypi
scanpy 1.8.1 pypi_0 pypi
scikit-learn 0.24.2 py38h1561384_1 conda-forge
scipy 1.7.1 py38h56a6a73_0 conda-forge
seaborn 0.11.1 pypi_0 pypi
seaborn-base 0.11.2 pyhd8ed1ab_0 conda-forge
sed 4.8 he412f7d_0 conda-forge
setuptools 57.4.0 py38h578d9bd_0 conda-forge
simplegeneric 0.8.1 py_1 conda-forge
sinfo 0.3.4 pypi_0 pypi
six 1.16.0 pyh6c4a22f_0 conda-forge
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.2.1 pypi_0 pypi
sqlite 3.36.0 h9cd32fc_0 conda-forge
statsmodels 0.12.2 py38h5c078b8_0 conda-forge
stdlib-list 0.8.0 pypi_0 pypi
suds-jurko 0.6 pypi_0 pypi
suitesparse 5.10.1 hd8046ac_0 conda-forge
sysroot_linux-64 2.12 he073ed8_14 conda-forge
tbb 2020.2 h4bd325d_4 conda-forge
texttable 1.6.4 pyhd8ed1ab_0 conda-forge
threadpoolctl 2.1.0 pypi_0 pypi
tk 8.6.11 h21135ba_0 conda-forge
tktable 2.10 hb7b940f_3 conda-forge
tornado 6.1 py38h497a2fe_1 conda-forge
tqdm 4.61.1 pypi_0 pypi
traitlets 5.1.0 pyhd8ed1ab_0 conda-forge
typing-extensions 3.10.0.0 pypi_0 pypi
tzdata 2021a he74cb21_1 conda-forge
tzlocal 3.0 py38h578d9bd_2 conda-forge
umap-learn 0.5.1 pypi_0 pypi
url-normalize 1.4.3 pypi_0 pypi
urllib3 1.26.5 pypi_0 pypi
wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge
wheel 0.37.0 pyhd8ed1ab_1 conda-forge
wordcloud 1.8.1 pypi_0 pypi
xlrd 1.2.0 pypi_0 pypi
xlsxwriter 1.4.3 pypi_0 pypi
xmltodict 0.12.0 pypi_0 pypi
xorg-kbproto 1.0.7 h7f98852_1002 conda-forge
xorg-libice 1.0.10 h7f98852_0 conda-forge
xorg-libsm 1.2.3 hd9c2040_1000 conda-forge
xorg-libx11 1.7.2 h7f98852_0 conda-forge
xorg-libxau 1.0.9 h7f98852_0 conda-forge
xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge
xorg-libxext 1.3.4 h7f98852_1 conda-forge
xorg-libxrender 0.9.10 h7f98852_1003 conda-forge
xorg-libxt 1.2.1 h7f98852_2 conda-forge
xorg-renderproto 0.11.1 h7f98852_1002 conda-forge
xorg-xextproto 7.3.0 h7f98852_1002 conda-forge
xorg-xproto 7.0.31 h7f98852_1007 conda-forge
xz 5.2.5 h516909a_1 conda-forge
zeromq 4.3.4 h9c3ff4c_1 conda-forge
zlib 1.2.11 h516909a_1010 conda-forge
zope-event 4.5.0 pypi_0 pypi
zope-interface 5.4.0 pypi_0 pypi
zstd 1.5.0 ha95c52a_0 conda-forgeMy apology. Could you send the result of
conda list --export > package-list.txtcommand instead, so that I can directly create a conda environment from the generated package-list.txt file?
Besides, could you also send me the command you update numba to v0.53.1 within this environment? In specific, did you update it using pip or conda?
I have a guess that your issue was probably due to the mixture of conda and pip installations. But I'll need further test, especially within your environment.
hoondy
commented
3 years ago No problem. I installed numba v0.53.1 using conda install numba. Then, downgraded it using conda install numba=0.52.0 Simply put, I installed all packages using conda whenever it was available. Rest of github repo and dev software, I used pip install git+https://github.com/xyz/xyz.git@v1.2.3
Just to be clear, my installation issue is resolved. However, I am happy to help so you can debug what was causing this error.
yihming
commented
3 years ago Well, I still could not reproduce your issue. However, based on my reading on your conda environment, there are some required dependencies of Pegasus missing. Moreover, your import error message is quite similar to the case when using Pegasuspy v1.3.0 with numba v0.53.0+. So I suspect this might be because your upgrade on Pegasuspy was not complete.
Bellow are my suggestions:
pegasus environment, run
pip install -U pegasuspyto install the missing dependencies. Then upgrade your numba and see if Pegasus could still be loaded.
pegasus), and in that environment, repeat Step 3 onward in installation to install the current release of Pegasus.hoondy
commented
3 years ago Many thanks for your help!
yihming
commented
3 years ago @hoondy ,
I'll go ahead to close this issue, as well as your pull request, if there is no more question. Please feel free to reopen it, or maybe more appropriately create a new issue, if you encounter new problems.
Sincerely, Yiming
Is there a yaml file to create a fully functional conda env for pegasus? The instructions in the README install the latest umap-learn which isn't compatible with pegasus currently, so I made a conda yaml file to try and be more explicit:
Conda yaml
pegasus.yml.txt
conda create env -f pegasus.yml.txtConda env: full
However, when I tried to import pegasus I get the following:
Error: snippet
Error: full
Many thanks!, Brian