fatiando / pooch

A friend to fetch your data files
https://www.fatiando.org/pooch
Other
611 stars 74 forks source link

Pooch 1.7.0 fails when filenames have apostrophes #357

Closed bmcfee closed 11 months ago

bmcfee commented 1 year ago

Description of the problem:

This just popped up in the librosa test suite after the environments upgraded from pooch 1.6 to 1.7.

We have a registry of data files that are fetched by pooch, as listed here: https://github.com/librosa/librosa/blob/c800e74f6a6ec5c27e0fa978d7355943cce04359/librosa/util/example_data/registry.txt Note that three of the files (lines 22-24) contain an apostrophe:

Karissa_Hobbs_-_Let's_Go_Fishin'.hq.ogg ...
Karissa_Hobbs_-_Let's_Go_Fishin'.ogg ...
Karissa_Hobbs_-_Let's_Go_Fishin'.txt ...

In pooch 1.6, this works as expected. However, after upgrading to 1.7, reading these files fails (see below). I believe this is a regression caused by #315 .

Full code that generated the error Minimal example via librosa:

import librosa
librosa.ex('fishin')

This will call out to our example loader: https://github.com/librosa/librosa/blob/c800e74f6a6ec5c27e0fa978d7355943cce04359/librosa/util/files.py#L43-L98

Full error message

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <module>:1                                                                                    │
│                                                                                                  │
│ /home/bmcfee/git/librosa/librosa/util/files.py:98 in example                                     │
│                                                                                                  │
│    95 │   else:                                                                                  │
│    96 │   │   ext = ".ogg"                                                                       │
│    97 │                                                                                          │
│ ❱  98 │   return str(__GOODBOY.fetch(__TRACKMAP[key]["path"] + ext))                             │
│    99                                                                                            │
│   100                                                                                            │
│   101 ex = example                                                                               │
│                                                                                                  │
│ /home/bmcfee/miniconda/envs/py39/lib/python3.9/site-packages/pooch/core.py:561 in fetch          │
│                                                                                                  │
│   558 │   │   │   local storage.                                                                 │
│   559 │   │                                                                                      │
│   560 │   │   """                                                                                │
│ ❱ 561 │   │   self._assert_file_in_registry(fname)                                               │
│   562 │   │                                                                                      │
│   563 │   │   # Create the local data directory if it doesn't already exist                      │
│   564 │   │   make_local_storage(str(self.abspath))                                              │
│                                                                                                  │
│ /home/bmcfee/miniconda/envs/py39/lib/python3.9/site-packages/pooch/core.py:608 in                │
│ _assert_file_in_registry                                                                         │
│                                                                                                  │
│   605 │   │   it's not.                                                                          │
│   606 │   │   """                                                                                │
│   607 │   │   if fname not in self.registry:                                                     │
│ ❱ 608 │   │   │   raise ValueError(f"File '{fname}' is not in the registry.")                    │
│   609 │                                                                                          │
│   610 │   def get_url(self, fname):                                                              │
│   611 │   │   """                                                                                │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: File 'Karissa_Hobbs_-_Let's_Go_Fishin'.ogg' is not in the registry.

System information

Output of conda list (it's bloated, but you asked for it :))

# packages in environment at /home/bmcfee/miniconda/envs/py39:
#
# Name                    Version                   Build  Channel
_ipython_minor_entry_point 8.7.0                h8cf3c4a_0    conda-forge
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                  2_kmp_llvm    conda-forge
abseil-cpp                20210324.2           h9c3ff4c_0    conda-forge
absl-py                   1.4.0              pyhd8ed1ab_0    conda-forge
aiobotocore               2.4.2              pyhd8ed1ab_0    conda-forge
aiofiles                  0.8.0                    pypi_0    pypi
aiohttp                   3.8.4            py39h72bdee0_0    conda-forge
aioitertools              0.11.0             pyhd8ed1ab_0    conda-forge
aiosignal                 1.3.1              pyhd8ed1ab_0    conda-forge
aiosqlite                 0.18.0             pyhd8ed1ab_0    conda-forge
alabaster                 0.7.13             pyhd8ed1ab_0    conda-forge
alembic                   1.10.2             pyhd8ed1ab_0    conda-forge
alsa-lib                  1.2.7.2              h166bdaf_0    conda-forge
altgraph                  0.17.3             pyhd8ed1ab_0    conda-forge
anyconfig                 0.13.0             pyhd8ed1ab_0    conda-forge
anyio                     3.6.2              pyhd8ed1ab_0    conda-forge
aom                       3.5.0                h27087fc_0    conda-forge
appdirs                   1.4.4              pyh9f0ad1d_0    conda-forge
argon2-cffi               21.3.0             pyhd8ed1ab_0    conda-forge
argon2-cffi-bindings      21.2.0           py39hb9d737c_3    conda-forge
arrow-cpp                 6.0.1           py39h01fd06f_8_cpu    conda-forge
asgiref                   3.5.1                    pypi_0    pypi
astroid                   2.15.0           py39hf3d152e_0    conda-forge
asttokens                 2.2.1              pyhd8ed1ab_0    conda-forge
astunparse                1.6.3              pyhd8ed1ab_0    conda-forge
asv                       0.5.1                    pypi_0    pypi
async-timeout             4.0.2              pyhd8ed1ab_0    conda-forge
async_generator           1.10                       py_0    conda-forge
atk-1.0                   2.38.0               hd4edc92_1    conda-forge
attr                      0.3.1                    pypi_0    pypi
attrs                     22.2.0             pyh71513ae_0    conda-forge
audioread                 3.0.0                     dev_0    
autopage                  0.5.1              pyhd8ed1ab_0    conda-forge
autopool                  0.2.0                    pypi_0    pypi
aws-c-cal                 0.5.11               h95a6274_0    conda-forge
aws-c-common              0.6.2                h7f98852_0    conda-forge
aws-c-event-stream        0.2.7               h3541f99_13    conda-forge
aws-c-io                  0.10.5               hfb6a706_0    conda-forge
aws-checksums             0.1.11               ha31a3da_7    conda-forge
aws-sdk-cpp               1.8.186              hecaee15_4    conda-forge
babel                     2.12.1             pyhd8ed1ab_1    conda-forge
backcall                  0.2.0              pyh9f0ad1d_0    conda-forge
backports                 1.0                pyhd8ed1ab_3    conda-forge
backports.functools_lru_cache 1.6.4              pyhd8ed1ab_0    conda-forge
bandit                    1.7.0                    pypi_0    pypi
beautifulsoup4            4.11.2             pyha770c72_0    conda-forge
binpickle                 0.3.4              pyhd8ed1ab_2    conda-forge
black                     23.1.0           py39hf3d152e_0    conda-forge
bleach                    6.0.0              pyhd8ed1ab_0    conda-forge
blinker                   1.5                pyhd8ed1ab_0    conda-forge
blosc                     1.21.3               hafa529b_0    conda-forge
bokeh                     3.1.0              pyhd8ed1ab_0    conda-forge
boolean.py                3.7                        py_0    conda-forge
boto                      2.49.0                   pypi_0    pypi
boto3                     1.24.59            pyhd8ed1ab_0    conda-forge
botocore                  1.27.59            pyhd8ed1ab_0    conda-forge
boxing                    0.1.4                    pypi_0    pypi
brotli                    1.0.9                h166bdaf_8    conda-forge
brotli-bin                1.0.9                h166bdaf_8    conda-forge
brotlipy                  0.7.0           py39hb9d737c_1005    conda-forge
brunsli                   0.1                  h9c3ff4c_0    conda-forge
build                     0.7.0              pyhd8ed1ab_0    conda-forge
bzip2                     1.0.8                h7f98852_4    conda-forge
c-ares                    1.18.1               h7f98852_0    conda-forge
c-blosc2                  2.7.1                hf91038e_0    conda-forge
ca-certificates           2022.12.7            ha878542_0    conda-forge
cached-property           1.5.2                hd8ed1ab_1    conda-forge
cached_property           1.5.2              pyha770c72_1    conda-forge
cachetools                4.2.4                    pypi_0    pypi
cairo                     1.16.0            ha12eb4b_1010    conda-forge
certifi                   2022.12.7          pyhd8ed1ab_0    conda-forge
cffi                      1.15.1           py39he91dace_3    conda-forge
cfitsio                   4.2.0                hd9d235c_0    conda-forge
chardet                   5.1.0            py39hf3d152e_0    conda-forge
charls                    2.4.1                hcb278e6_0    conda-forge
charset-normalizer        2.1.1              pyhd8ed1ab_0    conda-forge
click                     8.1.3           unix_pyhd8ed1ab_2    conda-forge
cliff                     4.2.0              pyhd8ed1ab_0    conda-forge
cloudpickle               2.2.1              pyhd8ed1ab_0    conda-forge
cmaes                     0.9.1              pyhd8ed1ab_0    conda-forge
cmarkgfm                  0.8.0            py39hb9d737c_2    conda-forge
cmd2                      2.4.3            py39hf3d152e_0    conda-forge
codespell                 2.2.4              pyhd8ed1ab_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
colorcet                  3.0.1              pyhd8ed1ab_0    conda-forge
colorlog                  6.7.0            py39hf3d152e_1    conda-forge
comm                      0.1.2              pyhd8ed1ab_0    conda-forge
commonmark                0.9.1                      py_0    conda-forge
conda                     23.1.0           py39hf3d152e_0    conda-forge
conda-build               3.23.3           py39hf3d152e_0    conda-forge
conda-forge-pinning       2023.03.16.11.45.19      hd8ed1ab_0    conda-forge
conda-package-handling    2.0.2              pyh38be061_0    conda-forge
conda-package-streaming   0.7.0              pyhd8ed1ab_1    conda-forge
conda-smithy              3.23.0             pyhd8ed1ab_0    conda-forge
conda-tree                1.1.0              pyhd8ed1ab_0    conda-forge
condastats                0.2.1              pyhd8ed1ab_0    conda-forge
contextlib2               21.6.0             pyhd8ed1ab_0    conda-forge
contourpy                 1.0.7            py39h4b4f3f3_0    conda-forge
coverage                  7.2.1            py39h72bdee0_0    conda-forge
cpuonly                   1.0                           0    pytorch
cramjam                   2.6.2            py39h4ef89ea_0    conda-forge
crema                     0.2.0                     dev_0    
cryptography              39.0.0           py39hd598818_0    conda-forge
csr                       0.4.3            py39h5a80b73_0    conda-forge
curl                      7.87.0               h6312ad2_0    conda-forge
cycler                    0.11.0             pyhd8ed1ab_0    conda-forge
cython                    0.29.33          py39h227be39_0    conda-forge
cytoolz                   0.12.0           py39hb9d737c_1    conda-forge
dali-dataset              1.1                      pypi_0    pypi
dask                      2022.6.1           pyhd8ed1ab_0    conda-forge
dask-core                 2022.6.1           pyhd8ed1ab_0    conda-forge
dask-labextension         5.2.0                    pypi_0    pypi
dataclasses               0.8                pyhc8e2a94_3    conda-forge
dataproperty              0.54.2                   pypi_0    pypi
datashader                0.14.4             pyh1a96a4e_0    conda-forge
datashape                 0.5.4                      py_1    conda-forge
datasketch                1.5.7                    pypi_0    pypi
dav1d                     1.0.0                h166bdaf_1    conda-forge
dawdreamer                0.6.5                    pypi_0    pypi
dbus                      1.13.6               h5008d03_3    conda-forge
debugpy                   1.6.6            py39h227be39_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1              pyhd8ed1ab_0    conda-forge
deprecated                1.2.13             pyh6c4a22f_0    conda-forge
deprecation               2.1.0                    pypi_0    pypi
dill                      0.3.6              pyhd8ed1ab_1    conda-forge
distributed               2022.6.1           pyhd8ed1ab_0    conda-forge
django-extensions         3.1.0                    pypi_0    pypi
django-ranged-fileresponse 0.1.2                    pypi_0    pypi
docutils                  0.18.1           py39hf3d152e_1    conda-forge
dominate                  2.6.0                    pypi_0    pypi
drf-dynamic-fields        0.3.0                    pypi_0    pypi
drf-flex-fields           0.9.5                    pypi_0    pypi
efficientnet              1.1.1                    pypi_0    pypi
efficientnet-pytorch      0.7.1                    pypi_0    pypi
entrypoints               0.4                pyhd8ed1ab_0    conda-forge
etils                     1.1.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.1.1              pyhd8ed1ab_0    conda-forge
execnet                   1.9.0              pyhd8ed1ab_0    conda-forge
executing                 1.2.0              pyhd8ed1ab_0    conda-forge
expat                     2.5.0                h27087fc_0    conda-forge
expiringdict              1.1.4                    pypi_0    pypi
eyed3                     0.9.6                    pypi_0    pypi
faker                     17.3.0                   pypi_0    pypi
fastparquet               2023.2.0         py39h389d5f1_0    conda-forge
filelock                  3.10.0             pyhd8ed1ab_0    conda-forge
filetype                  1.0.8                    pypi_0    pypi
flake8                    6.0.0              pyhd8ed1ab_0    conda-forge
flake8-polyfill           1.0.2                      py_0    conda-forge
flask                     2.2.3              pyhd8ed1ab_0    conda-forge
flatlatex                 0.15                     pypi_0    pypi
flit-core                 3.8.0              pyhd8ed1ab_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 hab24e00_0    conda-forge
fontconfig                2.14.2               h14ed4e7_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
fonttools                 4.39.0           py39h72bdee0_0    conda-forge
freetype                  2.12.1               hca18f0e_1    conda-forge
fribidi                   1.0.10               h36c2ea0_0    conda-forge
frozenlist                1.3.3            py39hb9d737c_0    conda-forge
fsspec                    2023.3.0           pyhd8ed1ab_1    conda-forge
future                    0.18.3             pyhd8ed1ab_0    conda-forge
gast                      0.4.0              pyh9f0ad1d_0    conda-forge
gdk-pixbuf                2.42.10              h05c8ddd_0    conda-forge
gettext                   0.21.1               h27087fc_0    conda-forge
gflags                    2.2.2             he1b5a44_1004    conda-forge
giflib                    5.2.1                h0b41bf4_3    conda-forge
git                       2.39.1          pl5321ha3eba64_0    conda-forge
gitdb                     4.0.10             pyhd8ed1ab_0    conda-forge
gitpython                 3.1.31             pyhd8ed1ab_0    conda-forge
glib                      2.74.1               h6239696_1    conda-forge
glib-tools                2.74.1               h6239696_1    conda-forge
glob2                     0.7                        py_0    conda-forge
glog                      0.5.0                h48cff8f_0    conda-forge
gmp                       6.2.1                h58526e2_0    conda-forge
gmpy2                     2.1.2            py39h376b7d2_1    conda-forge
gnutls                    3.7.8                hf3e180e_0    conda-forge
google-api-core           2.10.0             pyhd8ed1ab_0    conda-forge
google-api-python-client  2.81.0             pyhd8ed1ab_0    conda-forge
google-auth               1.35.0                   pypi_0    pypi
google-auth-httplib2      0.1.0              pyhd8ed1ab_1    conda-forge
google-auth-oauthlib      0.4.1                      py_2    conda-forge
google-cloud-audit-log    0.2.0                    pypi_0    pypi
google-pasta              0.2.0              pyh8c360ce_0    conda-forge
google-resumable-media    0.5.1                    pypi_0    pypi
googleapis-common-protos  1.57.0           py39hf3d152e_0    conda-forge
gprof2dot                 2021.2.21                pypi_0    pypi
graphite2                 1.3.13            h58526e2_1001    conda-forge
graphviz                  4.0.0                h5abf519_0    conda-forge
greenlet                  2.0.2            py39h227be39_0    conda-forge
grpc-cpp                  1.42.0               ha1441d3_1    conda-forge
grpcio                    1.42.0           py39hff7568b_0    conda-forge
gst-plugins-base          1.20.3               h57caac4_2    conda-forge
gstreamer                 1.20.3               hd4edc92_2    conda-forge
gtk2                      2.24.33              h90689f9_2    conda-forge
gts                       0.7.6                h64030ff_2    conda-forge
h11                       0.12.0                   pypi_0    pypi
h2                        4.1.0                    pypi_0    pypi
h5grove                   0.0.14                   pypi_0    pypi
h5py                      3.8.0           nompi_py39h89bf01e_101    conda-forge
harfbuzz                  4.2.0                h40b6f09_0    conda-forge
hdf5                      1.14.0          nompi_h5231ba7_103    conda-forge
heapdict                  1.0.1                      py_0    conda-forge
holoviews                 1.12.7                     py_0    conda-forge
hpack                     4.0.0                    pypi_0    pypi
htmlmin                   0.1.12                   pypi_0    pypi
httpcore                  0.14.3                   pypi_0    pypi
httplib2                  0.21.0             pyhd8ed1ab_0    conda-forge
httpx                     0.21.1                   pypi_0    pypi
hypercorn                 0.13.2                   pypi_0    pypi
hyperframe                6.0.1                    pypi_0    pypi
hyperopt                  0.2.7              pyhd8ed1ab_0    conda-forge
icdiff                    2.0.6              pyhd8ed1ab_0    conda-forge
icu                       69.1                 h9c3ff4c_0    conda-forge
idna                      3.4                pyhd8ed1ab_0    conda-forge
imagecodecs               2023.1.23        py39hd061359_0    conda-forge
imageio                   2.26.0             pyh24c5eb1_0    conda-forge
imagesize                 1.4.1              pyhd8ed1ab_0    conda-forge
importlib-metadata        6.0.0              pyha770c72_0    conda-forge
importlib-resources       5.12.0             pyhd8ed1ab_0    conda-forge
importlib_metadata        6.0.0                hd8ed1ab_0    conda-forge
importlib_resources       5.12.0             pyhd8ed1ab_0    conda-forge
inflection                0.5.1                    pypi_0    pypi
iniconfig                 2.0.0              pyhd8ed1ab_0    conda-forge
intel-openmp              2023.0.0         h9e868ea_25371  
ipykernel                 6.21.3             pyh210e3f2_0    conda-forge
ipympl                    0.9.3              pyhd8ed1ab_0    conda-forge
ipython                   8.7.0              pyh41d4057_0    conda-forge
ipython_genutils          0.2.0                      py_1    conda-forge
ipywidgets                8.0.4              pyhd8ed1ab_0    conda-forge
isodate                   0.6.1              pyhd8ed1ab_0    conda-forge
isort                     5.12.0             pyhd8ed1ab_1    conda-forge
itsdangerous              2.1.2              pyhd8ed1ab_0    conda-forge
itypes                    1.2.0                    pypi_0    pypi
jams                      0.3.4                    pypi_0    pypi
jaraco.classes            3.2.3              pyhd8ed1ab_0    conda-forge
jaxlib                    0.1.75           py39hde0f152_0    conda-forge
jbig                      2.1               h7f98852_2003    conda-forge
jedi                      0.18.2             pyhd8ed1ab_0    conda-forge
jeepney                   0.8.0              pyhd8ed1ab_0    conda-forge
jinja2                    3.0.3                    pypi_0    pypi
jmespath                  1.0.1              pyhd8ed1ab_0    conda-forge
joblib                    1.2.0              pyhd8ed1ab_0    conda-forge
jpeg                      9e                   h166bdaf_2    conda-forge
json5                     0.9.5              pyh9f0ad1d_0    conda-forge
jsonpickle                2.2.0              pyhd8ed1ab_0    conda-forge
jsonschema                4.17.3             pyhd8ed1ab_0    conda-forge
jupyter-cache             0.4.3                    pypi_0    pypi
jupyter-server-mathjax    0.2.5                    pypi_0    pypi
jupyter-server-proxy      3.2.1                    pypi_0    pypi
jupyter-sphinx            0.3.2                    pypi_0    pypi
jupyter_client            8.0.3              pyhd8ed1ab_0    conda-forge
jupyter_core              5.2.0            py39hf3d152e_0    conda-forge
jupyter_events            0.6.3              pyhd8ed1ab_0    conda-forge
jupyter_server            2.4.0              pyhd8ed1ab_0    conda-forge
jupyter_server_fileid     0.8.0              pyhd8ed1ab_0    conda-forge
jupyter_server_terminals  0.4.4              pyhd8ed1ab_1    conda-forge
jupyter_server_ydoc       0.6.1              pyhd8ed1ab_0    conda-forge
jupyter_ydoc              0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab                3.6.1              pyhd8ed1ab_0    conda-forge
jupyterlab-hdf            1.2.0                    pypi_0    pypi
jupyterlab-spellchecker   0.7.3              pyhd8ed1ab_0    conda-forge
jupyterlab_pygments       0.2.2              pyhd8ed1ab_0    conda-forge
jupyterlab_server         2.20.0             pyhd8ed1ab_0    conda-forge
jupyterlab_widgets        3.0.5              pyhd8ed1ab_0    conda-forge
jupytext                  1.14.5             pyhcff175f_0    conda-forge
jxrlib                    1.1                  h7f98852_2    conda-forge
kapre                     0.3.7                    pypi_0    pypi
keras                     2.7.0              pyhd8ed1ab_0    conda-forge
keras-applications        1.0.8                    pypi_0    pypi
keras-preprocessing       1.1.2              pyhd8ed1ab_0    conda-forge
keyring                   23.13.1          py39hf3d152e_0    conda-forge
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.4            py39hf939315_1    conda-forge
krb5                      1.20.1               hf9c8cef_0    conda-forge
label-studio-tools        0.0.0.dev11              pypi_0    pypi
labelimg                  1.8.6           linux_pyhd8ed1ab_3    conda-forge
lame                      3.100             h166bdaf_1003    conda-forge
launchdarkly-server-sdk   7.3.0                    pypi_0    pypi
lazy-loader               0.1                      pypi_0    pypi
lazy-object-proxy         1.9.0            py39h72bdee0_0    conda-forge
lazyweb                   0.1                      pypi_0    pypi
lcms2                     2.15                 hfd0df8a_0    conda-forge
ld_impl_linux-64          2.40                 h41732ed_0    conda-forge
lenskit                   0.14.2             pyhd8ed1ab_0    conda-forge
lerc                      4.0.0                h27087fc_0    conda-forge
libaec                    1.0.6                hcb278e6_1    conda-forge
libarchive                3.5.2                hb890918_3    conda-forge
libavif                   0.11.1               h5cdd6b5_0    conda-forge
libblas                   3.9.0            16_linux64_mkl    conda-forge
libbrotlicommon           1.0.9                h166bdaf_8    conda-forge
libbrotlidec              1.0.9                h166bdaf_8    conda-forge
libbrotlienc              1.0.9                h166bdaf_8    conda-forge
libcblas                  3.9.0            16_linux64_mkl    conda-forge
libclang                  13.0.1          default_had23c3d_1    conda-forge
libcrc32c                 1.1.2                h9c3ff4c_0    conda-forge
libcurl                   7.87.0               h6312ad2_0    conda-forge
libdeflate                1.17                 h0b41bf4_0    conda-forge
libdrm                    2.4.114              h166bdaf_0    conda-forge
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libev                     4.33                 h516909a_1    conda-forge
libevent                  2.1.10               h9b69904_4    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libflac                   1.4.2                h27087fc_0    conda-forge
libgcc-ng                 12.2.0              h65d4601_19    conda-forge
libgd                     2.3.3                h283352f_2    conda-forge
libgfortran-ng            12.2.0              h69a702a_19    conda-forge
libgfortran5              12.2.0              h337968e_19    conda-forge
libgirepository           1.74.0               ha2a38d2_1    conda-forge
libglib                   2.74.1               h606061b_1    conda-forge
libgomp                   12.2.0              h65d4601_19    conda-forge
libgoogle-cloud           1.35.0               h1b8b793_1    conda-forge
libhwloc                  2.8.0                h32351e8_1    conda-forge
libiconv                  1.17                 h166bdaf_0    conda-forge
libidn2                   2.3.4                h166bdaf_0    conda-forge
libimagequant             2.17.0               h7f98852_1    conda-forge
libjpeg-turbo             2.1.4                h166bdaf_0    conda-forge
liblapack                 3.9.0            16_linux64_mkl    conda-forge
liblief                   0.12.3               h27087fc_0    conda-forge
libllvm10                 10.0.1               he513fc3_3    conda-forge
libllvm11                 11.1.0               he0ac6c6_5    conda-forge
libllvm13                 13.0.1               hf817b99_2    conda-forge
libnghttp2                1.51.0               hdcd2b5c_0    conda-forge
libnsl                    2.0.0                h7f98852_0    conda-forge
libogg                    1.3.4                h7f98852_1    conda-forge
libopenblas               0.3.21          pthreads_h78a6416_3    conda-forge
libopus                   1.3.1                h7f98852_1    conda-forge
libpciaccess              0.17                 h166bdaf_0    conda-forge
libpng                    1.6.39               h753d276_0    conda-forge
libpq                     14.5                 h2baec63_5    conda-forge
libprotobuf               3.19.4               h780b84a_0    conda-forge
librosa                   0.10.0.post0             pypi_0    pypi
librsvg                   2.54.4               h7abd40a_0    conda-forge
libsndfile                1.2.0                hb75c966_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.40.0               h753d276_0    conda-forge
libssh2                   1.10.0               haa6b8db_3    conda-forge
libstdcxx-ng              12.2.0              h46fd767_19    conda-forge
libtasn1                  4.19.0               h166bdaf_0    conda-forge
libthrift                 0.15.0               he6d91bd_1    conda-forge
libtiff                   4.5.0                h6adf6a1_2    conda-forge
libtool                   2.4.7                h27087fc_0    conda-forge
libunistring              0.9.10               h7f98852_0    conda-forge
libunwind                 1.6.2                h9c3ff4c_0    conda-forge
libutf8proc               2.8.0                h166bdaf_0    conda-forge
libuuid                   2.32.1            h7f98852_1000    conda-forge
libuv                     1.43.0               h7f98852_0    conda-forge
libva                     2.17.0               h0b41bf4_0    conda-forge
libvorbis                 1.3.7                h9c3ff4c_0    conda-forge
libvpx                    1.12.0               hcb278e6_0    conda-forge
libwebp                   1.2.4                h1daa5a0_1    conda-forge
libwebp-base              1.2.4                h166bdaf_0    conda-forge
libxcb                    1.13              h7f98852_1004    conda-forge
libxkbcommon              1.0.3                he3ba5ed_0    conda-forge
libxml2                   2.9.14               haae042b_4    conda-forge
libxslt                   1.1.33               h0ef7038_3    conda-forge
libzlib                   1.2.13               h166bdaf_4    conda-forge
libzopfli                 1.0.3                h9c3ff4c_0    conda-forge
license-expression        1.2                        py_0    conda-forge
line-profiler             3.3.0                    pypi_0    pypi
llvm-openmp               15.0.7               h0cdce71_0    conda-forge
llvmlite                  0.39.1           py39h7d9a04d_1    conda-forge
locket                    1.0.0              pyhd8ed1ab_0    conda-forge
lockfile                  0.12.2                   pypi_0    pypi
lxml                      4.8.0            py39hb9d737c_3    conda-forge
lz4                       4.3.2            py39h724f13c_0    conda-forge
lz4-c                     1.9.3                h9c3ff4c_1    conda-forge
lzo                       2.10              h516909a_1000    conda-forge
madmom                    0.16.1                   pypi_0    pypi
makefun                   1.15.1             pyhd8ed1ab_0    conda-forge
mako                      1.2.4              pyhd8ed1ab_0    conda-forge
mando                     0.6.4                      py_0    conda-forge
markdown                  3.4.1              pyhd8ed1ab_0    conda-forge
markdown-it-py            1.1.0                    pypi_0    pypi
markupsafe                2.1.2            py39h72bdee0_0    conda-forge
matplotlib-base           3.7.1            py39he190548_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mbstrdecoder              1.1.0                    pypi_0    pypi
mccabe                    0.7.0              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.2.8                    pypi_0    pypi
mdurl                     0.1.0              pyhd8ed1ab_0    conda-forge
memory_profiler           0.61.0             pyhd8ed1ab_0    conda-forge
memray                    1.0.3                    pypi_0    pypi
mesalib                   18.3.1               h590aaf7_0    conda-forge
mido                      1.2.10                   pypi_0    pypi
mir-eval                  0.6                      pypi_0    pypi
mistune                   2.0.5              pyhd8ed1ab_0    conda-forge
mkl                       2022.2.1         h84fe81f_16997    conda-forge
mock                      4.0.3                    pypi_0    pypi
more-itertools            9.1.0              pyhd8ed1ab_0    conda-forge
mpc                       1.3.1                hfe3b2da_0    conda-forge
mpfr                      4.2.0                hb012696_0    conda-forge
mpg123                    1.31.2               hcb278e6_0    conda-forge
mpi                       1.0                       mpich    conda-forge
mpi4py                    3.1.4            py39h32b9844_0    conda-forge
mpich                     4.0.3              h846660c_100    conda-forge
mpmath                    1.3.0              pyhd8ed1ab_0    conda-forge
mrjob                     0.7.4              pyhd8ed1ab_2    conda-forge
msgpack-python            1.0.5            py39h4b4f3f3_0    conda-forge
msrest                    0.6.21             pyh44b312d_0    conda-forge
multidict                 6.0.4            py39h72bdee0_0    conda-forge
multipledispatch          0.6.0                      py_0    conda-forge
munkres                   1.1.4              pyh9f0ad1d_0    conda-forge
music21                   8.1.0              pyhd8ed1ab_0    conda-forge
musicbrainzngs            0.7.1                    pypi_0    pypi
mutagen                   1.46.0                   pypi_0    pypi
mypy                      1.1.1            py39h72bdee0_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
mysql-common              8.0.32               h14678bc_0    conda-forge
mysql-libs                8.0.32               h54cf53e_0    conda-forge
myst-nb                   0.13.2                   pypi_0    pypi
myst-parser               0.15.2                   pypi_0    pypi
nbclassic                 0.5.3              pyhb4ecaf3_3    conda-forge
nbclient                  0.5.13                   pypi_0    pypi
nbconvert                 7.2.9              pyhd8ed1ab_0    conda-forge
nbconvert-core            7.2.9              pyhd8ed1ab_0    conda-forge
nbconvert-pandoc          7.2.9              pyhd8ed1ab_0    conda-forge
nbdime                    3.1.1                    pypi_0    pypi
nbformat                  5.7.3              pyhd8ed1ab_0    conda-forge
nbstripout                0.6.1              pyhd8ed1ab_0    conda-forge
ncurses                   6.3                  h27087fc_1    conda-forge
nest-asyncio              1.5.6              pyhd8ed1ab_0    conda-forge
nettle                    3.8.1                hc379101_1    conda-forge
networkx                  3.0                pyhd8ed1ab_0    conda-forge
ninja                     1.11.1               h924138e_0    conda-forge
nltk                      3.8.1              pyhd8ed1ab_0    conda-forge
nodeenv                   1.7.0              pyhd8ed1ab_0    conda-forge
nodejs                    17.8.0               h784f1fd_0    conda-forge
notebook                  6.5.3              pyha770c72_0    conda-forge
notebook-shim             0.2.2              pyhd8ed1ab_0    conda-forge
nspr                      4.35                 h27087fc_0    conda-forge
nss                       3.89                 he45b914_0    conda-forge
numba                     0.56.4           py39h61ddf18_0    conda-forge
numcodecs                 0.11.0           py39h227be39_1    conda-forge
numexpr                   2.7.3            py39hde0f152_1    conda-forge
numpy                     1.23.5           py39h3d75532_0    conda-forge
numpydoc                  1.5.0              pyhd8ed1ab_0    conda-forge
oauthlib                  3.2.2              pyhd8ed1ab_0    conda-forge
olefile                   0.46               pyh9f0ad1d_1    conda-forge
openh264                  2.3.1                hcb278e6_2    conda-forge
openjpeg                  2.5.0                hfec8fc6_2    conda-forge
openssl                   1.1.1t               h0b41bf4_0    conda-forge
opt_einsum                3.3.0              pyhd8ed1ab_1    conda-forge
optuna                    3.1.0              pyhd8ed1ab_0    conda-forge
orc                       1.7.2                h1be678f_0    conda-forge
ordered-set               4.0.2                    pypi_0    pypi
orjson                    3.6.8                    pypi_0    pypi
outcome                   1.2.0                    pypi_0    pypi
p11-kit                   0.24.1               hc5aa10d_0    conda-forge
packaging                 23.0               pyhd8ed1ab_0    conda-forge
pandas                    1.5.3            py39h2ad29b5_0    conda-forge
pandoc                    3.1.1                h32600fe_0    conda-forge
pandocfilters             1.5.0              pyhd8ed1ab_0    conda-forge
pango                     1.50.7               hbd2fdc8_0    conda-forge
papyri                    0.0.7                    pypi_0    pypi
param                     1.13.0             pyh1a96a4e_0    conda-forge
parquet-cpp               1.5.1                         2    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
partd                     1.3.0              pyhd8ed1ab_0    conda-forge
patch                     2.7.6             h7f98852_1002    conda-forge
patchelf                  0.17.2               h58526e2_0    conda-forge
pathspec                  0.11.1             pyhd8ed1ab_0    conda-forge
pathvalidate              2.5.0                    pypi_0    pypi
patsy                     0.5.3              pyhd8ed1ab_0    conda-forge
pbr                       5.6.0                    pypi_0    pypi
pcre                      8.45                 h9c3ff4c_0    conda-forge
pcre2                     10.40                hc3806b6_0    conda-forge
pep517                    0.13.0             pyhd8ed1ab_0    conda-forge
pep8                      1.7.1                      py_0    conda-forge
perl                      5.32.1          2_h7f98852_perl5    conda-forge
pescador                  2.1.0                    pypi_0    pypi
pexpect                   4.8.0              pyh9f0ad1d_2    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    9.4.0            py39h2320bf1_1    conda-forge
pip                       23.0.1             pyhd8ed1ab_0    conda-forge
pixman                    0.40.0               h36c2ea0_0    conda-forge
pkginfo                   1.9.6              pyhd8ed1ab_0    conda-forge
pkgutil-resolve-name      1.3.10             pyhd8ed1ab_0    conda-forge
platformdirs              3.1.1              pyhd8ed1ab_0    conda-forge
pluggy                    1.0.0              pyhd8ed1ab_5    conda-forge
poisson-disc              0.2.1                    pypi_0    pypi
pooch                     1.7.0              pyhd8ed1ab_0    conda-forge
portaudio                 19.6.0               h8e90077_6    conda-forge
pprintpp                  0.4.0            py39hf3d152e_5    conda-forge
presets                   0.1.3                    pypi_0    pypi
pretty-midi               0.2.9                    pypi_0    pypi
prettytable               2.4.0                    pypi_0    pypi
priority                  2.0.0                    pypi_0    pypi
proglog                   0.1.9                      py_0    conda-forge
prometheus_client         0.16.0             pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.38             pyha770c72_0    conda-forge
proselint                 0.13.0                   pypi_0    pypi
proto-plus                1.20.3                   pypi_0    pypi
protobuf                  3.19.4           py39he80948d_0    conda-forge
psutil                    5.9.4            py39hb9d737c_0    conda-forge
psycopg2-binary           2.9.1                    pypi_0    pypi
pthread-stubs             0.4               h36c2ea0_1001    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pumpp                     0.6.0                     dev_0    
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
py                        1.11.0             pyh6c4a22f_0    conda-forge
py-cpuinfo                9.0.0              pyhd8ed1ab_0    conda-forge
py-lief                   0.12.3           py39h5a03fae_0    conda-forge
py4j                      0.10.9.5           pyhd8ed1ab_0    conda-forge
pyaml                     21.10.1            pyhd8ed1ab_0    conda-forge
pyarrow                   6.0.1           py39hff6fa39_8_cpu    conda-forge
pyasn1                    0.4.8                      py_0    conda-forge
pyasn1-modules            0.2.7                      py_0    conda-forge
pycairo                   1.23.0           py39h23c5bb2_0    conda-forge
pycodestyle               2.10.0             pyhd8ed1ab_0    conda-forge
pycompmusic               0.2+60.g8d07307           dev_0    
pycosat                   0.6.4            py39hb9d737c_1    conda-forge
pycparser                 2.21               pyhd8ed1ab_0    conda-forge
pycrypto                  2.6.1           py39h3811e60_1006    conda-forge
pycryptodome              3.16.0           py39h709f0c5_0    conda-forge
pyct                      0.4.6                      py_0    conda-forge
pyct-core                 0.4.6                      py_0    conda-forge
pydantic                  1.8.2                    pypi_0    pypi
pydeps                    1.10.12                  pypi_0    pypi
pydocstyle                6.3.0              pyhd8ed1ab_0    conda-forge
pyenchant                 3.2.2                    pypi_0    pypi
pyflakes                  3.0.1              pyhd8ed1ab_0    conda-forge
pygithub                  1.58.0             pyh1a96a4e_0    conda-forge
pygments                  2.14.0             pyhd8ed1ab_0    conda-forge
pygobject                 3.42.2           py39h4e09a1e_1    conda-forge
pyjwt                     2.6.0              pyhd8ed1ab_0    conda-forge
pylama                    7.7.1                      py_0    conda-forge
pylint                    2.17.0             pyhd8ed1ab_0    conda-forge
pynacl                    1.5.0            py39hb9d737c_2    conda-forge
pynndescent               0.5.8              pyh1a96a4e_0    conda-forge
pyopenssl                 23.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.0.9              pyhd8ed1ab_0    conda-forge
pyperclip                 1.8.2              pyhd8ed1ab_2    conda-forge
pypistats                 0.15.0                   pypi_0    pypi
pyqt                      5.12.3           py39hf3d152e_7    conda-forge
pyqt-impl                 5.12.3           py39h0fcd23e_7    conda-forge
pyqt5-sip                 4.19.18          py39he80948d_7    conda-forge
pyqtchart                 5.12             py39h0fcd23e_7    conda-forge
pyqtwebengine             5.12.1           py39h0fcd23e_7    conda-forge
pyrfc3339                 1.1                      pypi_0    pypi
pyrsistent                0.19.3           py39h72bdee0_0    conda-forge
pyrubberband              0.3.0                    pypi_0    pypi
pysocks                   1.7.1              pyha2e5f31_6    conda-forge
pysoundfile               0.12.1             pyhd8ed1ab_0    conda-forge
pyspark                   3.3.2              pyhd8ed1ab_0    conda-forge
pytables                  3.7.0            py39hf8baa48_4    conda-forge
pytablewriter             0.64.1                   pypi_0    pypi
pytest                    7.2.2              pyhd8ed1ab_0    conda-forge
pytest-benchmark          4.0.0              pyhd8ed1ab_0    conda-forge
pytest-cache              1.0                        py_2    conda-forge
pytest-cov                4.0.0              pyhd8ed1ab_0    conda-forge
pytest-doctestplus        0.12.1             pyhd8ed1ab_0    conda-forge
pytest-forked             1.6.0              pyhd8ed1ab_0    conda-forge
pytest-icdiff             0.6                pyhd8ed1ab_0    conda-forge
pytest-mock               3.10.0             pyhd8ed1ab_0    conda-forge
pytest-mpl                0.16.1             pyhd8ed1ab_0    conda-forge
pytest-pep8               1.0.6                      py_2    conda-forge
pytest-profiling          1.7.0                    pypi_0    pypi
pytest-pylint             0.18.0             pyhd3deb0d_0    conda-forge
pytest-runner             6.0.0              pyhd8ed1ab_0    conda-forge
pytest-sugar              0.9.6              pyhd8ed1ab_0    conda-forge
pytest-xdist              3.2.1              pyhd8ed1ab_0    conda-forge
python                    3.9.15          h47a2c10_0_cpython    conda-forge
python-blosc              1.10.6           py39h4661b88_1    conda-forge
python-dateutil           2.8.2              pyhd8ed1ab_0    conda-forge
python-fastjsonschema     2.16.3             pyhd8ed1ab_0    conda-forge
python-flatbuffers        2.0                pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7              pyhd8ed1ab_0    conda-forge
python-libarchive-c       4.0              py39hf3d152e_2    conda-forge
python-slugify            5.0.2                    pypi_0    pypi
python-snappy             0.6.1            py39he8e2bb5_0    conda-forge
python-sounddevice        0.4.1              pyh9f0ad1d_0    conda-forge
python_abi                3.9                      3_cp39    conda-forge
pytorch                   1.11.0          cpu_py39h7613f69_0    conda-forge
pytz                      2019.3                   pypi_0    pypi
pyu2f                     0.1.5              pyhd8ed1ab_0    conda-forge
pyviz_comms               2.2.1              pyhd8ed1ab_1    conda-forge
pywavelets                1.4.1            py39h389d5f1_0    conda-forge
pyyaml                    6.0              py39hb9d737c_5    conda-forge
pyzenodo3                 1.0.2                    pypi_0    pypi
pyzmq                     25.0.1           py39h0be026e_0    conda-forge
qt                        5.12.9               h1304e3e_6    conda-forge
quart                     0.13.1                   pypi_0    pypi
quart-trio                0.6.0                    pypi_0    pypi
rapidjson                 1.1.0             he1b5a44_1002    conda-forge
re2                       2021.11.01           h9c3ff4c_0    conda-forge
readline                  8.1.2                h0f457ee_0    conda-forge
readme_renderer           37.3               pyhd8ed1ab_0    conda-forge
redis                     4.2.2                    pypi_0    pypi
regex                     2022.10.31       py39hb9d737c_0    conda-forge
requests                  2.28.2             pyhd8ed1ab_0    conda-forge
requests-oauthlib         1.3.1              pyhd8ed1ab_0    conda-forge
requests-toolbelt         0.10.1             pyhd8ed1ab_0    conda-forge
requests-unixsocket       0.2.0                      py_0    conda-forge
resampy                   0.3.0                     dev_0    
rfc3339-validator         0.1.4              pyhd8ed1ab_0    conda-forge
rfc3986                   1.5.0                    pypi_0    pypi
rfc3986-validator         0.1.1              pyh9f0ad1d_0    conda-forge
rich                      10.9.0                   pypi_0    pypi
ripgrep                   13.0.0               h2f28480_2    conda-forge
rsa                       4.9                pyhd8ed1ab_0    conda-forge
ruamel.yaml               0.17.21          py39h72bdee0_3    conda-forge
ruamel.yaml.clib          0.2.7            py39h72bdee0_1    conda-forge
ruamel_yaml               0.15.80         py39hb9d737c_1008    conda-forge
rules                     2.2                      pypi_0    pypi
s2n                       1.0.10               h9b69904_0    conda-forge
s3fs                      2023.3.0           pyhd8ed1ab_0    conda-forge
s3transfer                0.6.0              pyhd8ed1ab_0    conda-forge
samplerate                0.1.0                    pypi_0    pypi
scikit-image              0.19.3           py39h4661b88_2    conda-forge
scikit-learn              1.2.2            py39h86b2a18_0    conda-forge
scikit-optimize           0.9.0              pyhd8ed1ab_1    conda-forge
scipy                     1.10.1           py39h7360e5f_0    conda-forge
scrypt                    0.8.18           py39h945232b_4    conda-forge
seaborn                   0.12.2               hd8ed1ab_0    conda-forge
seaborn-base              0.12.2             pyhd8ed1ab_0    conda-forge
secretstorage             3.3.3            py39hf3d152e_1    conda-forge
seedbank                  0.1.2              pyhd8ed1ab_0    conda-forge
semver                    2.13.0                   pypi_0    pypi
send2trash                1.8.0              pyhd8ed1ab_0    conda-forge
sentry-sdk                1.5.11                   pypi_0    pypi
setuptools                67.6.0             pyhd8ed1ab_0    conda-forge
simpervisor               0.4                      pypi_0    pypi
simplejson                3.17.6                   pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
skimpy                    0.0.2                    pypi_0    pypi
sleef                     3.5.1                h9b69904_2    conda-forge
smart_open                6.3.0              pyhd8ed1ab_1    conda-forge
smmap                     3.0.5              pyh44b312d_0    conda-forge
snakeviz                  2.1.1              pyhd8ed1ab_0    conda-forge
snappy                    1.1.10               h9fff704_0    conda-forge
sniffio                   1.3.0              pyhd8ed1ab_0    conda-forge
snowballstemmer           2.2.0              pyhd8ed1ab_0    conda-forge
sortedcontainers          2.4.0              pyhd8ed1ab_0    conda-forge
soupsieve                 2.3.2.post1        pyhd8ed1ab_0    conda-forge
soxr                      0.1.3                h0b41bf4_3    conda-forge
soxr-python               0.3.3            py39h2ae25f5_0    conda-forge
sphinx                    6.1.3              pyhd8ed1ab_0    conda-forge
sphinx-autodoc-typehints  1.21.8             pyhd8ed1ab_0    conda-forge
sphinx-gallery            0.12.2             pyhd8ed1ab_0    conda-forge
sphinx-multiversion       0.2.4                    pypi_0    pypi
sphinx-togglebutton       0.3.1                    pypi_0    pypi
sphinx_rtd_theme          1.2.0              pyha770c72_0    conda-forge
sphinxcontrib-applehelp   1.0.4              pyhd8ed1ab_0    conda-forge
sphinxcontrib-devhelp     1.0.2                      py_0    conda-forge
sphinxcontrib-htmlhelp    2.0.1              pyhd8ed1ab_0    conda-forge
sphinxcontrib-jquery      4.1                pyhd8ed1ab_0    conda-forge
sphinxcontrib-jsmath      1.0.1                      py_0    conda-forge
sphinxcontrib-qthelp      1.0.3                      py_0    conda-forge
sphinxcontrib-serializinghtml 1.1.5              pyhd8ed1ab_2    conda-forge
sphinxcontrib-spelling    7.7.0                    pypi_0    pypi
sphinxcontrib-svg2pdfconverter 1.1.1                    pypi_0    pypi
sqlalchemy                1.4.37                   pypi_0    pypi
sqlite                    3.40.0               h4ff8645_0    conda-forge
sqlparse                  0.4.2                    pypi_0    pypi
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
statsmodels               0.13.5           py39h2ae25f5_2    conda-forge
stdlib-list               0.8.0                    pypi_0    pypi
stevedore                 3.3.0                    pypi_0    pypi
svt-av1                   1.4.1                hcb278e6_0    conda-forge
sympy                     1.11.1           py39hf3d152e_2    conda-forge
tabledata                 1.3.0                    pypi_0    pypi
tbb                       2021.7.0             h924138e_1    conda-forge
tblib                     1.7.0              pyhd8ed1ab_0    conda-forge
tcolorpy                  0.1.1                    pypi_0    pypi
tensorboard               2.6.0                      py_1  
tensorboard-data-server   0.6.1            py39hd97740a_4    conda-forge
tensorboard-plugin-wit    1.8.1              pyhd8ed1ab_0    conda-forge
tensorflow                2.7.0           cpu_py39h4655687_0    conda-forge
tensorflow-base           2.7.0           cpu_py39hf4995fd_0    conda-forge
tensorflow-estimator      2.7.0           cpu_py39ha241409_0    conda-forge
termcolor                 2.2.0              pyhd8ed1ab_0    conda-forge
terminado                 0.17.1             pyh41d4057_0    conda-forge
testpath                  0.6.0              pyhd8ed1ab_0    conda-forge
text-unidecode            1.3                      pypi_0    pypi
there                     0.0.9                    pypi_0    pypi
threadpoolctl             3.1.0              pyh8a188c0_0    conda-forge
thrift                    0.18.1           py39h227be39_0    conda-forge
tifffile                  2022.5.4                 pypi_0    pypi
tinycss2                  1.2.1              pyhd8ed1ab_0    conda-forge
tk                        8.6.12               h27826a3_0    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tomlkit                   0.11.6             pyha770c72_0    conda-forge
toolz                     0.12.0             pyhd8ed1ab_0    conda-forge
torch-lr-finder           0.2.1                    pypi_0    pypi
tornado                   6.2              py39hb9d737c_1    conda-forge
tqdm                      4.65.0             pyhd8ed1ab_1    conda-forge
traitlets                 5.9.0              pyhd8ed1ab_0    conda-forge
trio                      0.21.0                   pypi_0    pypi
tuna                      0.5.11                   pypi_0    pypi
twine                     4.0.2              pyhd8ed1ab_0    conda-forge
typed-ast                 1.5.4            py39hb9d737c_1    conda-forge
typepy                    1.3.0                    pypi_0    pypi
typer                     0.6.1                    pypi_0    pypi
types-decorator           5.1.8.3            pyhd8ed1ab_0    conda-forge
types-docutils            0.19.1.6           pyhd8ed1ab_0    conda-forge
types-setuptools          67.6.0.0           pyhd8ed1ab_0    conda-forge
typing                    3.10.0.0           pyhd8ed1ab_0    conda-forge
typing-extensions         4.5.0                hd8ed1ab_0    conda-forge
typing_extensions         4.5.0              pyha770c72_0    conda-forge
tzdata                    2022g                h191b570_0    conda-forge
ua-parser                 0.10.0                   pypi_0    pypi
ujson                     4.0.2                    pypi_0    pypi
umap-learn                0.5.3            py39hf3d152e_0    conda-forge
unicodedata2              15.0.0           py39hb9d737c_0    conda-forge
uritemplate               4.1.1              pyhd8ed1ab_0    conda-forge
urllib3                   1.26.15            pyhd8ed1ab_0    conda-forge
urwid                     2.1.2                    pypi_0    pypi
user-agents               2.2.0                    pypi_0    pypi
velin                     0.0.11                   pypi_0    pypi
vsts-python-api           0.1.25             pyhd8ed1ab_1    conda-forge
waf                       2.0.25           py39h72bdee0_0    conda-forge
wcwidth                   0.2.6              pyhd8ed1ab_0    conda-forge
webcolors                 1.12               pyhd8ed1ab_0    conda-forge
webencodings              0.5.1                      py_1    conda-forge
websocket-client          1.5.1              pyhd8ed1ab_0    conda-forge
werkzeug                  2.2.3              pyhd8ed1ab_0    conda-forge
wheel                     0.40.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.5              pyhd8ed1ab_0    conda-forge
wrapt                     1.15.0           py39h72bdee0_0    conda-forge
wsproto                   1.1.0                    pypi_0    pypi
x264                      1!164.3095           h166bdaf_2    conda-forge
x265                      3.5                  h924138e_3    conda-forge
xarray                    2023.2.0           pyhd8ed1ab_0    conda-forge
xkcdpass                  1.19.3                   pypi_0    pypi
xmljson                   0.2.0                    pypi_0    pypi
xorg-fixesproto           5.0               h7f98852_1002    conda-forge
xorg-kbproto              1.0.7             h7f98852_1002    conda-forge
xorg-libice               1.0.10               h7f98852_0    conda-forge
xorg-libsm                1.2.3             hd9c2040_1000    conda-forge
xorg-libx11               1.8.4                h0b41bf4_0    conda-forge
xorg-libxau               1.0.9                h7f98852_0    conda-forge
xorg-libxdmcp             1.1.3                h7f98852_0    conda-forge
xorg-libxext              1.3.4                h0b41bf4_2    conda-forge
xorg-libxfixes            5.0.3             h7f98852_1004    conda-forge
xorg-libxrender           0.9.10            h7f98852_1003    conda-forge
xorg-renderproto          0.11.1            h7f98852_1002    conda-forge
xorg-xextproto            7.3.0             h0b41bf4_1003    conda-forge
xorg-xproto               7.0.31            h7f98852_1007    conda-forge
xyzservices               2023.2.0           pyhd8ed1ab_0    conda-forge
xz                        5.2.6                h166bdaf_0    conda-forge
y-py                      0.5.9            py39h50f1755_0    conda-forge
yaml                      0.2.5                h7f98852_2    conda-forge
yarl                      1.8.2            py39hb9d737c_0    conda-forge
youtube-dl                2021.12.17         pyhd8ed1ab_0    conda-forge
ypy-websocket             0.8.2              pyhd8ed1ab_0    conda-forge
zeromq                    4.3.4                h9c3ff4c_1    conda-forge
zfp                       1.0.0                h27087fc_3    conda-forge
zict                      2.2.0              pyhd8ed1ab_0    conda-forge
zipp                      3.15.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               h166bdaf_4    conda-forge
zlib-ng                   2.0.6                h166bdaf_0    conda-forge
zstandard                 0.19.0           py39h29414ee_1    conda-forge
zstd                      1.5.2                h3eb15da_6    conda-forge

bmcfee commented 1 year ago

Confirmed that this is due to this line:

https://github.com/fatiando/pooch/blob/a965902d26015453ac00269597a23b83d85db644/pooch/core.py#L660

For example, the 1.6.0 behavior would have been:

In [2]: line = "Karissa_Hobbs_-_Let's_Go_Fishin'.ogg"

In [3]: line.split()
Out[3]: ["Karissa_Hobbs_-_Let's_Go_Fishin'.ogg"]

In 1.7, it becomes:

In [11]: shlex.split(line)
Out[11]: ['Karissa_Hobbs_-_Lets_Go_Fishin.ogg']

(note the missing single-quotes)

A quick fix here is to use shlex.quote:

In [12]: shlex.split(shlex.quote(line))
Out[12]: ["Karissa_Hobbs_-_Let's_Go_Fishin'.ogg"]

I don't know if this will be safe across platforms, but it would at least fix the bug I'm seeing.

dokempf commented 1 year ago

For the reference: The original bug that let to using shlex is https://github.com/fatiando/pooch/issues/313.

bmcfee commented 1 year ago

I think the use of shlex.quote would still fix #313:

In [2]: shlex.split(shlex.quote("foo' bar baz'.txt"))
Out[2]: ["foo' bar baz'.txt"]
bmcfee commented 1 year ago

Ok, I started prepping a PR for this and realized that it's a bit more subtle than just sticking shlex.quote in there, since it will apply to the entire line. If a filename ends with a quote, it will muck up the space parsing, and the whole parser falls apart.

The problem as I see it boils down to some ambiguity in the format of the registry file, in particular due to the option of custom urls as a third token. We don't know in advance how many spaces there will be in a line, and that makes splitting tricky.

If we didn't have custom urls, it would be enough to do:

filename, hash = line.rsplit(' ', maxsplit=2)

where rsplit will group all but the last space separator into filename.

If we always have a custom URL, it would be equally simple:

filename, hash, url = line.rsplit(' ', maxsplit=3)

but this will be incorrect if the url is absent and the filename contains a space.

What we really need is a way to first check if the final token is a hash or a url, and then act accordingly:

# Assuming URLs in registry files are properly URL-encoded, i.e. ' ' → '%20'
prefix, last = line.rsplit(' ', maxsplit=2)
if is_url(last):
    url = last
    filename, hash = prefix.split(' ', maxsplit=2)
else:
    filename, hash, url = prefix, last, None

Now, there's a problem that URLs (per RFC 3986) if not fully formed can look an awful lot like hashes. We can't rely on detecting just a : because this is also valid in hashes (md5:abcdef...).

We also can't get away with detecting :/, or even :// because this would rule out DOI urls.

I suspect the simplest solution would be to explicitly match URLs against a subset of supported schemes (http:, https:, ftp:, sftp:, doi:) and treat anything else as if it's a hash. If it's not actually a hash, this will eventually fail later on.

Does this sound reasonable? If so, I'd be happy to prep a PR implementing the above strategy.

santisoler commented 1 year ago

Hi @bmcfee, thanks for opening this issue.

I'm sorry that the last release broke backward compatibility. This is something we always try to avoid through exhaustive testing, but as you could tell we weren't considering the case where the filenames contain apostrophes.

I think the issue you are facing is part of a larger discussion: what type of standard do we want to use for filenames within Pooch. After we merged #315 I think we inadvertently established that we would follow the POSIX standard (see Parsing rules in shlex when POSIX is True).

Therefore, if a filename contains a special character, like a space or an apostrophe, we can always use the backslash to escape it. In your particular case, you can modify your registry file to:

Karissa_Hobbs_-_Let\'s_Go_Fishin\'.hq.ogg ...
Karissa_Hobbs_-_Let\'s_Go_Fishin\'.ogg ...
Karissa_Hobbs_-_Let\'s_Go_Fishin\'.txt ...

and it should work. We use the same strategy for spaces, as was introduced in #315.

If we try to modify the load_registry() method after every time we encounter a corner case that isn't supported, I think we'll be modifying its code after every release, which isn't sustainable and it's prone to introduce more backward compatibility breaks.

So, in my opinion, special characters like spaces, double quotes or apostrophes in filenames should be handled by inserting a backslash before them.

What do you think? Does it sounds like a good compromise?

bmcfee commented 1 year ago

Thanks @santisoler - I agree with that assessment. I'm totally happy if pooch wants to explicitly adopt POSIX-style syntax for filenames, and I think the confusion only arose because the syntax was implicitly defined (1.7) or under-specified (<1.6). I opened the PR because it wasn't clear if this was intended behavior or a bug to be fixed, but if it's intended behavior we can close it out.

I think what this means for my project in particular is that we need to put out a post-release to upper-bound the supported pooch versions, and then revise our registry file with an updated lower bound.

One question though: how does this formatting work when creating registry files from a pooch object? I don't see any logic for escaping filenames on write (make_registry) in #315 , and from a glance at https://github.com/fatiando/pooch/blob/a965902d26015453ac00269597a23b83d85db644/pooch/hashes.py#L218-L222 I expect it will not work as expected currently.

leouieda commented 11 months ago

@bmcfee thanks for being very understanding with this. I think it's best to assume the POSIX syntax is intended (though it was originally an oversight). make_registry wasn't updated to work with the new syntax, though, as you pointed out. That still needs to be done.

bmcfee commented 11 months ago

No worries - sounds like a good plan to me!

leouieda commented 11 months ago

Alright, closing this one since we have #374 and #369 to handle the issues identified here. Thanks again for reporting @bmcfee!