search

broadinstitute / 2022_PERISCOPE

This repository contains all supporting analyses and files for Ramezani, Bauman, Singh, and Weisbart, et al. "A genome-wide atlas of human cell morphology".
BSD 3-Clause "New" or "Revised" License
2 stars 0 forks source link

Clean up environment #10

Closed ErinWeisbart closed 1 year ago

ErinWeisbart commented 1 year ago

Can we clean up the conda environment? Creating the environment currently takes forever...

@bethac07 Is this something you can help with when you get the chance?

ErinWeisbart commented 1 year ago

And by forever it seems like maybe quite literally forever - I left it for hours and it never solved.

So in 5467aa3867343f5c058d1824f48fcb1f2a57d635 I did my best to clean it up into something that solves quickly. Following various troubleshooting guides, I:

Will run all the notebooks with this environment to ensure that I didn't accidentally remove something necessary and/or that bumping versions doesn't affect results.

Major caveat that this is out of my area of expertise, so will be good to have folks weigh in on this and let me know if it should be handled differently.

bethac07 commented 1 year ago

Looking right now, but I don't think you committed that file

bethac07 commented 1 year ago

Partly this may be because I'm on an M1, but conda-forge said it wouldn't give me python 3.7.1 (or 3.7 at all)

Updating 3 lines got a working environment in my hands -

Here's the diff

@@ -2,14 +2,14 @@ name: periscope_2022
 dependencies:
-- conda-forge::python=3.7.1
+- conda-forge::python=3.8
-- conda-forge::ipykernel=5.1.0
-- conda-forge::nb_conda_kernels=2.2.1
+- conda-forge::ipykernel=5
+- conda-forge::nb_conda_kernels

Here's the full enviroment

conda list ``` (test) bcimini@wm4f8-761 2022_PERISCOPE % conda list # packages in environment at /opt/homebrew/Caskroom/miniconda/base/envs/test: # # Name Version Build Channel _ipython_minor_entry_point 8.7.0 h3b92ee0_0 conda-forge adjusttext 0.7.3.1 py_1 conda-forge aiobotocore 2.4.0 pyhd8ed1ab_0 conda-forge aiohttp 3.8.3 py38hb991d35_1 conda-forge aiohttp-retry 2.8.3 pyhd8ed1ab_0 conda-forge aioitertools 0.11.0 pyhd8ed1ab_0 conda-forge aiosignal 1.3.1 pyhd8ed1ab_0 conda-forge amqp 5.1.1 pyhd8ed1ab_0 conda-forge antlr-python-runtime 4.9.3 pyhd8ed1ab_1 conda-forge anyio 3.6.2 pyhd8ed1ab_0 conda-forge aom 3.5.0 h7ea286d_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge appnope 0.1.3 pyhd8ed1ab_0 conda-forge argon2-cffi 21.3.0 pyhd8ed1ab_0 conda-forge argon2-cffi-bindings 21.2.0 py38hb991d35_3 conda-forge asttokens 2.2.1 pyhd8ed1ab_0 conda-forge async-timeout 4.0.2 pyhd8ed1ab_0 conda-forge asyncssh 2.12.0 pyhd8ed1ab_0 conda-forge atk-1.0 2.38.0 hcb7b3dd_1 conda-forge atpublic 3.0.1 pyhd8ed1ab_0 conda-forge attrs 22.1.0 pyh71513ae_1 conda-forge babel 2.11.0 pyhd8ed1ab_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.11.1 pyha770c72_0 conda-forge billiard 3.6.4.0 py38hb991d35_3 conda-forge black 22.10.0 py38h10201cd_2 conda-forge bleach 5.0.1 pyhd8ed1ab_0 conda-forge blosc 1.21.2 h1d6ff8b_0 conda-forge boto3 1.24.59 pyhd8ed1ab_0 conda-forge botocore 1.27.59 pyhd8ed1ab_0 conda-forge brotli 1.0.9 h1a8c8d9_8 conda-forge brotli-bin 1.0.9 h1a8c8d9_8 conda-forge brotlipy 0.7.0 py38hb991d35_1005 conda-forge brunsli 0.1 h9f76cd9_0 conda-forge bzip2 1.0.8 h3422bc3_4 conda-forge c-ares 1.18.1 h3422bc3_0 conda-forge c-blosc2 2.6.0 h303ed30_0 conda-forge ca-certificates 2022.12.7 h4653dfc_0 conda-forge cairo 1.16.0 h73a0509_1014 conda-forge celery 5.2.7 pyhd8ed1ab_0 conda-forge certifi 2022.12.7 pyhd8ed1ab_0 conda-forge cffi 1.15.1 py38ha45ccd6_2 conda-forge cfitsio 4.2.0 h2f961c4_0 conda-forge charls 2.3.4 hbdafb3b_0 conda-forge charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge click 8.1.3 unix_pyhd8ed1ab_2 conda-forge click-didyoumean 0.3.0 pyhd8ed1ab_0 conda-forge click-plugins 1.1.1 py_0 conda-forge click-repl 0.2.0 pyhd8ed1ab_0 conda-forge cloudpickle 2.2.0 pyhd8ed1ab_0 conda-forge colorama 0.4.6 pyhd8ed1ab_0 conda-forge commonmark 0.9.1 py_0 conda-forge configobj 5.0.6 py_0 conda-forge contourpy 1.0.6 py38h9dc3d6a_0 conda-forge cryptography 38.0.4 py38h23f6d3d_0 conda-forge cycler 0.11.0 pyhd8ed1ab_0 conda-forge cytominer-eval 0.1 pypi_0 pypi cytoolz 0.12.0 py38hb991d35_1 conda-forge dask-core 2022.12.0 pyhd8ed1ab_0 conda-forge dataclasses 0.8 pyhc8e2a94_3 conda-forge dav1d 1.0.0 he4db4b2_1 conda-forge decorator 5.1.1 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge descartes 1.1.0 py_4 conda-forge dictdiffer 0.9.0 pyhd8ed1ab_0 conda-forge diskcache 5.4.0 pyhd8ed1ab_0 conda-forge distro 1.6.0 pyhd8ed1ab_0 conda-forge dpath 2.1.2 py38h10201cd_0 conda-forge dulwich 0.20.50 py38hb991d35_0 conda-forge dvc 2.37.0 pyhd8ed1ab_0 conda-forge dvc-data 0.28.4 pyhd8ed1ab_0 conda-forge dvc-http 2.27.2 pyhd8ed1ab_0 conda-forge dvc-objects 0.14.0 pyhd8ed1ab_0 conda-forge dvc-render 0.0.15 pyhd8ed1ab_0 conda-forge dvc-s3 2.21.0 pyhd8ed1ab_0 conda-forge dvc-task 0.1.6 pyhd8ed1ab_0 conda-forge dvclive 1.1.2 pyhd8ed1ab_0 conda-forge entrypoints 0.4 pyhd8ed1ab_0 conda-forge executing 1.2.0 pyhd8ed1ab_0 conda-forge expat 2.5.0 hb7217d7_0 conda-forge filelock 3.8.2 pyhd8ed1ab_0 conda-forge flatten-dict 0.4.2 pyhd8ed1ab_1 conda-forge flit-core 3.8.0 pyhd8ed1ab_0 conda-forge flufl.lock 7.1 pyhd8ed1ab_0 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 hab24e00_0 conda-forge fontconfig 2.14.1 h82840c6_0 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge fonttools 4.38.0 py38hb991d35_1 conda-forge freetype 2.12.1 hd633e50_1 conda-forge fribidi 1.0.10 h27ca646_0 conda-forge frozenlist 1.3.3 py38hb991d35_0 conda-forge fsspec 2022.11.0 pyhd8ed1ab_0 conda-forge funcy 1.17 pyhd8ed1ab_0 conda-forge future 0.18.2 pyhd8ed1ab_6 conda-forge gdk-pixbuf 2.42.8 h4f389bb_1 conda-forge gettext 0.21.1 h0186832_0 conda-forge giflib 5.2.1 h27ca646_2 conda-forge gitdb 4.0.10 pyhd8ed1ab_0 conda-forge gitpython 3.1.29 pyhd8ed1ab_0 conda-forge grandalf 0.6 py_0 conda-forge graphite2 1.3.13 h9f76cd9_1001 conda-forge graphviz 6.0.2 h4f8fbd6_0 conda-forge gtk2 2.24.33 h57013de_2 conda-forge gts 0.7.6 h4b6d4d6_2 conda-forge harfbuzz 5.3.0 hddbc195_0 conda-forge hdbscan 0.8.29 py38hb39dbe9_1 conda-forge hydra-core 1.3.0 pyhd8ed1ab_0 conda-forge icu 70.1 h6b3803e_0 conda-forge idna 3.4 pyhd8ed1ab_0 conda-forge imagecodecs 2022.9.26 py38h8c96dfd_4 conda-forge imageio 2.22.4 pyhfa7a67d_1 conda-forge importlib-metadata 5.1.0 pyha770c72_0 conda-forge importlib_resources 5.10.1 pyhd8ed1ab_0 conda-forge ipykernel 5.5.5 py38h2cb4d76_0 conda-forge ipython 8.7.0 pyhd1c38e8_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.6.5 pyhd8ed1ab_0 conda-forge iterative-telemetry 0.0.6 pyhd8ed1ab_0 conda-forge jedi 0.18.2 pyhd8ed1ab_0 conda-forge jinja2 3.1.2 pyhd8ed1ab_1 conda-forge jmespath 1.0.1 pyhd8ed1ab_0 conda-forge joblib 1.1.0 pyhd8ed1ab_0 conda-forge jpeg 9e he4db4b2_2 conda-forge json5 0.9.5 pyh9f0ad1d_0 conda-forge jsonschema 4.17.3 pyhd8ed1ab_0 conda-forge jupyter 1.0.0 py38h10201cd_8 conda-forge jupyter_client 7.4.8 pyhd8ed1ab_0 conda-forge jupyter_console 6.4.4 pyhd8ed1ab_0 conda-forge jupyter_contrib_core 0.4.0 pyhd8ed1ab_0 conda-forge jupyter_contrib_nbextensions 0.5.1 pyhd8ed1ab_2 conda-forge jupyter_core 5.1.0 py38h10201cd_0 conda-forge jupyter_events 0.5.0 pyhd8ed1ab_0 conda-forge jupyter_highlight_selected_word 0.2.0 py38h10201cd_1005 conda-forge jupyter_latex_envs 1.4.6 pyhd8ed1ab_1002 conda-forge jupyter_nbextensions_configurator 0.6.1 pyhd8ed1ab_0 conda-forge jupyter_server 2.0.1 pyhd8ed1ab_0 conda-forge jupyter_server_terminals 0.4.2 pyhd8ed1ab_0 conda-forge jupyterlab 3.5.1 pyhd8ed1ab_0 conda-forge jupyterlab_pygments 0.2.2 pyhd8ed1ab_0 conda-forge jupyterlab_server 2.16.5 pyhd8ed1ab_0 conda-forge jupyterlab_widgets 3.0.4 pyhd8ed1ab_0 conda-forge jxrlib 1.1 h27ca646_2 conda-forge kiwisolver 1.4.4 py38h9dc3d6a_1 conda-forge kombu 5.2.4 py38h10201cd_2 conda-forge krb5 1.19.3 he492e65_0 conda-forge lcms2 2.14 h8193b64_0 conda-forge lerc 4.0.0 h9a09cb3_0 conda-forge libaec 1.0.6 hbdafb3b_0 conda-forge libavif 0.11.1 h3d80962_0 conda-forge libblas 3.9.0 16_osxarm64_openblas conda-forge libbrotlicommon 1.0.9 h1a8c8d9_8 conda-forge libbrotlidec 1.0.9 h1a8c8d9_8 conda-forge libbrotlienc 1.0.9 h1a8c8d9_8 conda-forge libcblas 3.9.0 16_osxarm64_openblas conda-forge libcurl 7.86.0 h1c293e1_1 conda-forge libcxx 14.0.6 h2692d47_0 conda-forge libdeflate 1.14 h1a8c8d9_0 conda-forge libedit 3.1.20191231 hc8eb9b7_2 conda-forge libev 4.33 h642e427_1 conda-forge libffi 3.4.2 h3422bc3_5 conda-forge libgd 2.3.3 hede1055_3 conda-forge libgfortran 5.0.0 11_3_0_hd922786_26 conda-forge libgfortran5 11.3.0 hdaf2cc0_26 conda-forge libgit2 1.5.0 hce1fc7d_1 conda-forge libglib 2.74.1 h4646484_1 conda-forge libiconv 1.17 he4db4b2_0 conda-forge liblapack 3.9.0 16_osxarm64_openblas conda-forge libllvm11 11.1.0 hfa12f05_5 conda-forge libnghttp2 1.47.0 h519802c_1 conda-forge libopenblas 0.3.21 openmp_hc731615_3 conda-forge libpng 1.6.39 h76d750c_0 conda-forge librsvg 2.54.4 ha2634a2_0 conda-forge libsodium 1.0.18 h27ca646_1 conda-forge libsqlite 3.40.0 h76d750c_0 conda-forge libssh2 1.10.0 h7a5bd25_3 conda-forge libtiff 4.4.0 hfa0b094_4 conda-forge libtool 2.4.6 hbdafb3b_1008 conda-forge libwebp 1.2.4 h328b37c_0 conda-forge libwebp-base 1.2.4 h57fd34a_0 conda-forge libxcb 1.13 h9b22ae9_1004 conda-forge libxml2 2.10.3 h87b0503_0 conda-forge libxslt 1.1.37 h1bd8bc4_0 conda-forge libzlib 1.2.13 h03a7124_4 conda-forge libzopfli 1.0.3 h9f76cd9_0 conda-forge llvm-openmp 15.0.6 h7cfbb63_0 conda-forge llvmlite 0.39.1 py38he6bf707_1 conda-forge locket 1.0.0 pyhd8ed1ab_0 conda-forge lxml 4.9.1 py38h43d3e64_1 conda-forge lz4-c 1.9.3 hbdafb3b_1 conda-forge markupsafe 2.1.1 py38hb991d35_2 conda-forge matplotlib 3.6.2 py38h150bfb4_0 conda-forge matplotlib-base 3.6.2 py38hbbe890c_0 conda-forge matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge mistune 2.0.4 pyhd8ed1ab_0 conda-forge mizani 0.7.3 pyhd8ed1ab_0 conda-forge multidict 6.0.2 py38hb991d35_2 conda-forge munkres 1.1.4 pyh9f0ad1d_0 conda-forge mypy_extensions 0.4.3 py38h10201cd_6 conda-forge nanotime 0.5.2 py_0 conda-forge nb_conda_kernels 2.3.1 py38h10201cd_2 conda-forge nbclassic 0.4.8 pyhd8ed1ab_0 conda-forge nbclient 0.7.2 pyhd8ed1ab_0 conda-forge nbconvert 7.2.6 pyhd8ed1ab_0 conda-forge nbconvert-core 7.2.6 pyhd8ed1ab_0 conda-forge nbconvert-pandoc 7.2.6 pyhd8ed1ab_0 conda-forge nbformat 5.7.0 pyhd8ed1ab_0 conda-forge ncurses 6.3 h07bb92c_1 conda-forge nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge networkx 2.8.8 pyhd8ed1ab_0 conda-forge notebook 6.5.2 pyha770c72_1 conda-forge notebook-shim 0.2.2 pyhd8ed1ab_0 conda-forge numba 0.56.4 py38h947a10e_0 conda-forge numpy 1.23.5 py38h09ac2d9_0 conda-forge omegaconf 2.2.3 py38h10201cd_0 conda-forge openjpeg 2.5.0 h5d4e404_1 conda-forge openssl 3.0.7 h03a7124_1 conda-forge packaging 22.0 pyhd8ed1ab_0 conda-forge palettable 3.3.0 py_0 conda-forge pandas 1.2.0 py38h9b9bf68_1 conda-forge pandoc 2.19.2 hce30654_1 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge pango 1.50.12 h8dd4791_0 conda-forge parso 0.8.3 pyhd8ed1ab_0 conda-forge partd 1.3.0 pyhd8ed1ab_0 conda-forge pathlib 1.0.1 py_1 conda-forge pathlib2 2.3.7.post1 py38h10201cd_2 conda-forge pathspec 0.9.0 pyhd8ed1ab_0 conda-forge patsy 0.5.3 pyhd8ed1ab_0 conda-forge pcre2 10.40 hb34f9b4_0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 9.2.0 py38hf86a106_3 conda-forge pip 22.3.1 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h27ca646_0 conda-forge pkgutil-resolve-name 1.3.10 pyhd8ed1ab_0 conda-forge platformdirs 2.6.0 pyhd8ed1ab_0 conda-forge plotnine 0.8.0 pyhd8ed1ab_0 conda-forge prometheus_client 0.15.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.36 pyha770c72_0 conda-forge prompt_toolkit 3.0.36 hd8ed1ab_0 conda-forge psutil 5.9.4 py38hb991d35_0 conda-forge pthread-stubs 0.4 h27ca646_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge pycparser 2.21 pyhd8ed1ab_0 conda-forge pycytominer 0.1 pypi_0 pypi pydot 1.4.2 py38h10201cd_3 conda-forge pygit2 1.11.1 py38hb991d35_1 conda-forge pygments 2.13.0 pyhd8ed1ab_0 conda-forge pygtrie 2.5.0 pyhd8ed1ab_0 conda-forge pynndescent 0.5.8 pyh1a96a4e_0 conda-forge pyopenssl 22.1.0 pyhd8ed1ab_0 conda-forge pyparsing 3.0.9 pyhd8ed1ab_0 conda-forge pyrsistent 0.19.2 py38hb991d35_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.8.15 h3ba56d0_0_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python-fastjsonschema 2.16.2 pyhd8ed1ab_0 conda-forge python-gssapi 1.8.2 py38h3d91cf0_1 conda-forge python-json-logger 2.0.1 pyh9f0ad1d_0 conda-forge python_abi 3.8 3_cp38 conda-forge pytz 2022.6 pyhd8ed1ab_0 conda-forge pywavelets 1.3.0 py38hb39dbe9_2 conda-forge pywin32-on-windows 0.1.0 pyh1179c8e_3 conda-forge pyyaml 6.0 py38hb991d35_5 conda-forge pyzmq 24.0.1 py38hb72be9f_1 conda-forge readline 8.1.2 h46ed386_0 conda-forge requests 2.28.1 pyhd8ed1ab_1 conda-forge rich 12.6.0 pyhd8ed1ab_0 conda-forge ruamel.yaml 0.17.21 py38hb991d35_2 conda-forge ruamel.yaml.clib 0.2.7 py38hb991d35_1 conda-forge s3fs 2022.11.0 pyhd8ed1ab_0 conda-forge s3transfer 0.6.0 pyhd8ed1ab_0 conda-forge scikit-image 0.19.3 py38h61dac83_2 conda-forge scikit-learn 1.1.3 py38h0c332d6_1 conda-forge scipy 1.9.3 py38h7b4f323_2 conda-forge scmrepo 0.1.4 pyhd8ed1ab_0 conda-forge seaborn 0.12.1 hd8ed1ab_0 conda-forge seaborn-base 0.12.1 pyhd8ed1ab_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 59.8.0 py38h10201cd_1 conda-forge shortuuid 1.0.11 pyhd8ed1ab_0 conda-forge shtab 1.5.8 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge smmap 3.0.5 pyh44b312d_0 conda-forge snappy 1.1.9 h17c5cce_2 conda-forge sniffio 1.3.0 pyhd8ed1ab_0 conda-forge soupsieve 2.3.2.post1 pyhd8ed1ab_0 conda-forge sqlalchemy 1.4.45 pypi_0 pypi stack_data 0.6.2 pyhd8ed1ab_0 conda-forge statsmodels 0.13.5 py38hb39dbe9_2 conda-forge tabulate 0.9.0 pyhd8ed1ab_1 conda-forge tbb 2021.7.0 hffc8910_1 conda-forge terminado 0.17.1 pyhd1c38e8_0 conda-forge threadpoolctl 3.1.0 pyh8a188c0_0 conda-forge tifffile 2022.10.10 pyhd8ed1ab_0 conda-forge tinycss2 1.2.1 pyhd8ed1ab_0 conda-forge tk 8.6.12 he1e0b03_0 conda-forge tomli 2.0.1 pyhd8ed1ab_0 conda-forge tomlkit 0.11.6 pyha770c72_0 conda-forge toolz 0.12.0 pyhd8ed1ab_0 conda-forge tornado 6.2 py38hb991d35_1 conda-forge tqdm 4.64.1 pyhd8ed1ab_0 conda-forge traitlets 5.7.0 pyhd8ed1ab_0 conda-forge typing 3.10.0.0 pyhd8ed1ab_0 conda-forge typing-extensions 4.4.0 hd8ed1ab_0 conda-forge typing_extensions 4.4.0 pyha770c72_0 conda-forge umap-learn 0.5.3 py38h10201cd_0 conda-forge unicodedata2 15.0.0 py38hb991d35_0 conda-forge urllib3 1.26.13 pyhd8ed1ab_0 conda-forge vine 5.0.0 pyhd8ed1ab_1 conda-forge voluptuous 0.13.1 pyhd8ed1ab_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge websocket-client 1.4.2 pyhd8ed1ab_0 conda-forge wheel 0.38.4 pyhd8ed1ab_0 conda-forge widgetsnbextension 3.5.1 py38h10201cd_4 conda-forge wrapt 1.14.1 py38hb991d35_1 conda-forge xorg-libxau 1.0.9 h27ca646_0 conda-forge xorg-libxdmcp 1.1.3 h27ca646_0 conda-forge xz 5.2.6 h57fd34a_0 conda-forge yaml 0.2.5 h3422bc3_2 conda-forge yarl 1.8.1 py38hb991d35_0 conda-forge zc.lockfile 2.0 pyhd8ed1ab_1 conda-forge zeromq 4.3.4 hbdafb3b_1 conda-forge zfp 1.0.0 hb6e4faa_3 conda-forge zipp 3.11.0 pyhd8ed1ab_0 conda-forge zlib 1.2.13 h03a7124_4 conda-forge zlib-ng 2.0.6 he4db4b2_0 conda-forge zstd 1.5.2 h8128057_4 conda-forge ```

Caveat here - I came at this from the opposite angle you did, "minimal changes to working product", as opposed to "minimum code required for functional product". Yours is MUCH better - we definitely want to drop things we aren't using.

ErinWeisbart commented 1 year ago

Sorry, linked the wrong commit. Updated it in comment above.