search

moldyn / MoSAIC

Correlation-based feature selection of Molecular Dynamics simulations
https://moldyn.github.io/MoSAIC/
MIT License
25 stars 2 forks source link

matplotlib artist error in CI plotting #28

Open mmfarrugia opened 2 months ago

mmfarrugia commented 2 months ago

I have run into an error while using the command line version of mosaic which seem isolated to a particular usage of matplotlib to plot the matrix data. I do successfully get the output of the calculations, just no plot. The only thing I thought could cause this is that I only gave the method calculation of a subset of the possible contacts in the system, else it would be trying to calculate with nearly a TB of RAM due to the size of the system and the length of the trajectories.

(mosaic_3_10) $ python -m mosaic clustering -i contacts_000_similarity.out -o contacts_000_clustering.out --plot -v

MoSAIC CLUSTERING
~~~ Initialize clustering class
~~~ Load file contacts_000_similarity.out
~~~ Fit input
~~~ Store output
~~~ Plot matrix
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/mosaic/__main__.py", line 361, in <module>
    main()
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/mosaic/__main__.py", line 332, in clustering
    im = ax.pcolormesh(
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/__init__.py", line 1473, in inner
    return func(
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/axes/_axes.py", line 6434, in pcolormesh
    collection = mcoll.QuadMesh(
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/collections.py", line 2146, in __init__
    Collection.__init__(self, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/collections.py", line 206, in __init__
    self._internal_update(kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/artist.py", line 1216, in _internal_update
    return self._update_props(
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/artist.py", line 1190, in _update_props
    raise AttributeError(
AttributeError: QuadMesh.set() got an unexpected keyword argument 'vim'

I have had some odd issues with the kernel dying due to undefined errors and blank reasons in jupyter notebook after about 30 minutes of calculation trying to fit the correlation matrix, else I would just use the python interface for this.

Currently digging to try to see if I can just load in the clustering matrix and clusters from the command line calculation to plot in jupyter notebook.

Thank you for your help!

braniii commented 2 months ago

@mmfarrugia Hi thx for opening this issue. I guess this issue is caused by a simple typo. Once @gegabo has tested the fix, we will release a new version.

braniii commented 2 months ago

@mmfarrugia Using the current dev version, the issue should be (hopefully!) fixed

python3 -m pip install git+https://github.com/moldyn/MoSAIC.git
gegabo commented 2 months ago

@mmfarrugia concerning loading the clustering results directly and then playing around in jupyter notebook for the plotting: take a look at the tutorial (https://moldyn.github.io/MoSAIC/tutorial/t4l_tutorial_github/).

There, we use the results of the clustering to sort the clusters by their size and reshuffle them accordingly:

idxs = np.argsort(
    [len(cluster) for cluster in clustering.clusters_],
)[::-1]
clusters_sorted = clustering.clusters_[idxs]
clusters_sorted_flattened = np.concatenate(clustering.clusters_[idxs])

# sort the matrix accordingly
matrix_sorted = correlation_matrix[
    np.ix_(clusters_sorted_flattened, clusters_sorted_flattened)
]
ticks = np.cumsum([len(cluster) for cluster in clustering.clusters_[idxs]])
ticks = [0, *ticks[:-1]]  # ticks start with 0

This way you only have to perform the clustering once as you can save the clustering results afterwards which makes it way easier to play around with the plotting settings. If you have any more questions, we're happy to help! :)

mmfarrugia commented 2 months ago

Thank you! I reran the clustering via the click CI but it seems to have run into a new matplotlib issue:

(mosaic_3_10) [mfarrugi@d32cepyc210 analysis]$ python -m mosaic clustering -i contacts1_similarity.out -o contacts1_clustering_retry.out --plot -v

MoSAIC CLUSTERING


~~~ Load file contacts1_similarity.out
~~~ Fit input
~~~ Store output
~~~ Plot matrix
Traceback (most recent call last):
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/mosaic/__main__.py", line 363, in <module>
    main()
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/mosaic/__main__.py", line 358, in clustering
    plt.colorbar(im, width='3%')
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/pyplot.py", line 2516, in colorbar
    ret = gcf().colorbar(mappable, cax=cax, ax=ax, **kwargs)
  File "/afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10/lib/python3.10/site-packages/matplotlib/figure.py", line 1253, in colorbar
    cb = cbar.Colorbar(cax, mappable, **{
TypeError: Colorbar.__init__() got an unexpected keyword argument 'width'

At first I thought this could be matplotlib version issue but it looks like it could be related how matplotlib draws colorbars when it is drawing the image since it seems like there is an odd order which must be followed based on this [stackoverflow example](https://stackoverflow.com/questions/69727902/matplotlib-colorbar-init-got-an-unexpected-keyword-argument-location) else it will conflict.

Please let me know if this does work on your system so I know if it is a version issue.

My current environment is based on what a collaborator has found to work:

# packages in environment at /afs/crc.nd.edu/user/m/mfarrugi/.conda/envs/mosaic_3_10:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
anyio                     4.4.0                    pypi_0    pypi
argon2-cffi               23.1.0                   pypi_0    pypi
argon2-cffi-bindings      21.2.0                   pypi_0    pypi
arrow                     1.3.0                    pypi_0    pypi
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
async-lru                 2.0.4                    pypi_0    pypi
attrs                     24.2.0                   pypi_0    pypi
babel                     2.16.0                   pypi_0    pypi
beartype                  0.18.5                   pypi_0    pypi
beautifulsoup4            4.12.3                   pypi_0    pypi
bleach                    6.1.0                    pypi_0    pypi
bzip2                     1.0.8                h4bc722e_7    conda-forge
ca-certificates           2024.8.30            hbcca054_0    conda-forge
certifi                   2024.8.30                pypi_0    pypi
cffi                      1.17.0                   pypi_0    pypi
charset-normalizer        3.3.2                    pypi_0    pypi
click                     8.1.7                    pypi_0    pypi
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
contourpy                 1.3.0                    pypi_0    pypi
cycler                    0.12.1                   pypi_0    pypi
debugpy                   1.8.5           py310hf71b8c6_1    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
defusedxml                0.7.1                    pypi_0    pypi
exceptiongroup            1.2.2              pyhd8ed1ab_0    conda-forge
executing                 2.1.0              pyhd8ed1ab_0    conda-forge
fastjsonschema            2.20.0                   pypi_0    pypi
fonttools                 4.53.1                   pypi_0    pypi
fqdn                      1.5.1                    pypi_0    pypi
h11                       0.14.0                   pypi_0    pypi
httpcore                  1.0.5                    pypi_0    pypi
httpx                     0.27.2                   pypi_0    pypi
idna                      3.8                      pypi_0    pypi
igraph                    0.11.6                   pypi_0    pypi
importlib-metadata        8.4.0              pyha770c72_0    conda-forge
importlib_metadata        8.4.0                hd8ed1ab_0    conda-forge
ipykernel                 6.29.5             pyh3099207_0    conda-forge
ipython                   8.27.0             pyh707e725_0    conda-forge
ipywidgets                8.1.5                    pypi_0    pypi
isoduration               20.11.0                  pypi_0    pypi
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4                    pypi_0    pypi
joblib                    1.4.2                    pypi_0    pypi
json5                     0.9.25                   pypi_0    pypi
jsonpointer               3.0.0                    pypi_0    pypi
jsonschema                4.23.0                   pypi_0    pypi
jsonschema-specifications 2023.12.1                pypi_0    pypi
jupyter                   1.1.1                    pypi_0    pypi
jupyter-console           6.6.3                    pypi_0    pypi
jupyter-events            0.10.0                   pypi_0    pypi
jupyter-lsp               2.2.5                    pypi_0    pypi
jupyter-server            2.14.2                   pypi_0    pypi
jupyter-server-terminals  0.5.3                    pypi_0    pypi
jupyter_client            8.6.2              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py310hff52083_0    conda-forge
jupyterlab                4.2.5                    pypi_0    pypi
jupyterlab-pygments       0.3.0                    pypi_0    pypi
jupyterlab-server         2.27.3                   pypi_0    pypi
jupyterlab-widgets        3.0.13                   pypi_0    pypi
keyutils                  1.6.1                h166bdaf_0    conda-forge
kiwisolver                1.4.5                    pypi_0    pypi
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.40                 hf3520f5_7    conda-forge
leidenalg                 0.10.2                   pypi_0    pypi
libedit                   3.1.20191231         he28a2e2_2    conda-forge
libffi                    3.4.2                h7f98852_5    conda-forge
libgcc                    14.1.0               h77fa898_1    conda-forge
libgcc-ng                 14.1.0               h69a702a_1    conda-forge
libgomp                   14.1.0               h77fa898_1    conda-forge
libnsl                    2.0.1                hd590300_0    conda-forge
libsodium                 1.0.18               h36c2ea0_1    conda-forge
libsqlite                 3.46.1               hadc24fc_0    conda-forge
libstdcxx                 14.1.0               hc0a3c3a_1    conda-forge
libstdcxx-ng              14.1.0               h4852527_1    conda-forge
libuuid                   2.38.1               h0b41bf4_0    conda-forge
libzlib                   1.3.1                h4ab18f5_1    conda-forge
linkify-it-py             2.0.3                    pypi_0    pypi
markdown-it-py            3.0.0                    pypi_0    pypi
markupsafe                2.1.5                    pypi_0    pypi
matplotlib                3.9.2                    pypi_0    pypi
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
mdit-py-plugins           0.4.1                    pypi_0    pypi
mdurl                     0.1.2                    pypi_0    pypi
mistune                   3.0.2                    pypi_0    pypi
mosaic-clustering         0.4.1                    pypi_0    pypi
nbclient                  0.10.0                   pypi_0    pypi
nbconvert                 7.16.4                   pypi_0    pypi
nbformat                  5.10.4                   pypi_0    pypi
ncurses                   6.5                  he02047a_1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
notebook                  7.2.2                    pypi_0    pypi
notebook-shim             0.2.4                    pypi_0    pypi
numpy                     1.26.0                   pypi_0    pypi
openssl                   3.3.1                hb9d3cd8_3    conda-forge
overrides                 7.7.0                    pypi_0    pypi
packaging                 24.1               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2                    pypi_0    pypi
pandocfilters             1.5.1                    pypi_0    pypi
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
pexpect                   4.9.0              pyhd8ed1ab_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pillow                    10.4.0                   pypi_0    pypi
pip                       24.2               pyh8b19718_1    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
prometheus-client         0.20.0                   pypi_0    pypi
prompt-toolkit            3.0.47             pyha770c72_0    conda-forge
psutil                    6.0.0           py310hc51659f_0    conda-forge
ptyprocess                0.7.0              pyhd3deb0d_0    conda-forge
pure_eval                 0.2.3              pyhd8ed1ab_0    conda-forge
pycparser                 2.22                     pypi_0    pypi
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.4                    pypi_0    pypi
python                    3.10.0          h543edf9_3_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-json-logger        2.0.7                    pypi_0    pypi
python_abi                3.10                    5_cp310    conda-forge
pytz                      2024.1                   pypi_0    pypi
pyyaml                    6.0.2                    pypi_0    pypi
pyzmq                     26.2.0          py310h71f11fc_0    conda-forge
readline                  8.2                  h8228510_1    conda-forge
referencing               0.35.1                   pypi_0    pypi
requests                  2.32.3                   pypi_0    pypi
rfc3339-validator         0.1.4                    pypi_0    pypi
rfc3986-validator         0.1.1                    pypi_0    pypi
rich                      13.8.0                   pypi_0    pypi
rpds-py                   0.20.0                   pypi_0    pypi
scikit-learn              1.5.1                    pypi_0    pypi
scikit-learn-extra        0.3.0                    pypi_0    pypi
scipy                     1.14.1                   pypi_0    pypi
send2trash                1.8.3                    pypi_0    pypi
setuptools                73.0.1             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.1                    pypi_0    pypi
soupsieve                 2.6                      pypi_0    pypi
sqlite                    3.46.1               h9eae976_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
terminado                 0.18.1                   pypi_0    pypi
texttable                 1.7.0                    pypi_0    pypi
textual                   0.79.1                   pypi_0    pypi
threadpoolctl             3.5.0                    pypi_0    pypi
tinycss2                  1.3.0                    pypi_0    pypi
tk                        8.6.13          noxft_h4845f30_101    conda-forge
tomli                     2.0.1                    pypi_0    pypi
tornado                   6.4.1           py310ha75aee5_1    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
trogon                    0.5.0                    pypi_0    pypi
types-python-dateutil     2.9.0.20240821           pypi_0    pypi
typing_extensions         4.12.2             pyha770c72_0    conda-forge
tzdata                    2024.1                   pypi_0    pypi
uc-micro-py               1.0.3                    pypi_0    pypi
uri-template              1.3.0                    pypi_0    pypi
urllib3                   2.2.2                    pypi_0    pypi
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
webcolors                 24.8.0                   pypi_0    pypi
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.8.0                    pypi_0    pypi
wheel                     0.44.0             pyhd8ed1ab_0    conda-forge
widgetsnbextension        4.0.13                   pypi_0    pypi
xz                        5.2.6                h166bdaf_0    conda-forge
zeromq                    4.3.5                h75354e8_4    conda-forge
zipp                      3.20.1             pyhd8ed1ab_0    conda-forge

Thank you again!
braniii commented 2 months ago

@mmfarrugia thx for the helpful log. @gegabo it seems that when switching from pplt to matplotlib, the syntax was not adjusted accordingly. Probably, one could add a test for running the command line interface.

gegabo commented 2 months ago

@mmfarrugia the issue youve encountered should be fixed now when using the current dev version.

@mmfarrugia Using the current dev version, the issue should be (hopefully!) fixed

python3 -m pip install git+https://github.com/moldyn/MoSAIC.git