Open cbeck22 opened 2 months ago
Hi cbeck,
Thanks for bringing these new changes to my attention. In principle, the yaml
and pkl
formats contain the same information and it should be quite straight forward to convert between them. I will look into this tomorrow.
Converting the yaml
to pkl
may not be enough however... I tested this before cryodrgn v3.3.3
existed and so I can't guarantee that WIGGLE
will work with the models output by newer cryodrgn
. I will install the most up to date cryodrgn
tomorrow and train a quick model and investigate. Hopefully I can make some quick changes to the dev branch and push those to the repo so you can use it.
Best, Charles
You were right, pickling the config.yaml
file to config.pkl
was straightforward, and Wiggle was able to read it successfully.
However, there was another unexpected problem - cryoDRGN v1.0 implemented Gaussian Fourier featurization in the encoder, and gaussian
is the new default for the enc_type parameter in the config.pkl
file. The config file also has a new parameter, feat_sigma
, which is necessary for the Gaussian Fourier featurization logic. I was able to make the necessary changes to miniDRGN.py
, and I can now generate volumes in wiggle when I click on the PCA plot. Yay!
However, I ran into a new issue when I tried to push these changes to my own repository. As a sanity check, I cloned the updated repository to my local machine and tried to reinstall wiggle into ChimeraX:
running build
running build_py
running bdist_wheel
installing to build/bdist.linux-x86_64/wheel
running install
running install_lib
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/wheel
creating build/bdist.linux-x86_64/wheel/chimerax
creating build/bdist.linux-x86_64/wheel/chimerax/wiggle
creating build/bdist.linux-x86_64/wheel/chimerax/wiggle/resources
copying build/lib/chimerax/wiggle/resources/Wiggle2.PNG -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/resources
copying build/lib/chimerax/wiggle/resources/Wiggle.PNG -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/resources
copying build/lib/chimerax/wiggle/tool.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle
creating build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/ui_compiled.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/miniSPARC.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/dijkstra_modified.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/clustering.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/data_handler.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/wiggle_packager.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/mrc.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/embedding.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/low_pass.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/volumiser.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/deps/path_inference_dijkstra.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
creating build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps/__pycache__
copying build/lib/chimerax/wiggle/deps/__pycache__/ui_compiled.cpython-37.pyc -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps/__pycache__
copying build/lib/chimerax/wiggle/deps/miniDRGN.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle/deps
copying build/lib/chimerax/wiggle/__init__.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle
copying build/lib/chimerax/wiggle/wiggle.py -> build/bdist.linux-x86_64/wheel/chimerax/wiggle
running install_egg_info
running egg_info
creating wiggle.egg-info
writing wiggle.egg-info/PKG-INFO
writing dependency_links to wiggle.egg-info/dependency_links.txt
writing requirements to wiggle.egg-info/requires.txt
writing top-level names to wiggle.egg-info/top_level.txt
writing manifest file 'wiggle.egg-info/SOURCES.txt'
reading manifest file 'wiggle.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'wiggle.egg-info/SOURCES.txt'
Copying wiggle.egg-info to build/bdist.linux-x86_64/wheel/wiggle-0.2.2-py3.9.egg-info
running install_scripts
adding license file "LICENSE" (matched pattern "LICEN[CS]E*")
creating build/bdist.linux-x86_64/wheel/wiggle-0.2.2.dist-info/WHEEL
creating 'dist/wiggle-0.2.2-py3-none-any.whl' and adding 'build/bdist.linux-x86_64/wheel' to it
adding 'chimerax/wiggle/__init__.py'
adding 'chimerax/wiggle/tool.py'
adding 'chimerax/wiggle/wiggle.py'
adding 'chimerax/wiggle/deps/clustering.py'
adding 'chimerax/wiggle/deps/data_handler.py'
adding 'chimerax/wiggle/deps/dijkstra_modified.py'
adding 'chimerax/wiggle/deps/embedding.py'
adding 'chimerax/wiggle/deps/low_pass.py'
adding 'chimerax/wiggle/deps/miniDRGN.py'
adding 'chimerax/wiggle/deps/miniSPARC.py'
adding 'chimerax/wiggle/deps/mrc.py'
adding 'chimerax/wiggle/deps/path_inference_dijkstra.py'
adding 'chimerax/wiggle/deps/ui_compiled.py'
adding 'chimerax/wiggle/deps/volumiser.py'
adding 'chimerax/wiggle/deps/wiggle_packager.py'
adding 'chimerax/wiggle/deps/__pycache__/ui_compiled.cpython-37.pyc'
adding 'chimerax/wiggle/resources/Wiggle.PNG'
adding 'chimerax/wiggle/resources/Wiggle2.PNG'
adding 'wiggle-0.2.2.dist-info/LICENSE'
adding 'wiggle-0.2.2.dist-info/METADATA'
adding 'wiggle-0.2.2.dist-info/WHEEL'
adding 'wiggle-0.2.2.dist-info/top_level.txt'
adding 'wiggle-0.2.2.dist-info/RECORD'
removing build/bdist.linux-x86_64/wheel
Distribution is in wiggle/dist/wiggle-0.2.2-py3-none-any.whl
[toolshed install](help:user/commands/toolshed.html#install)
wiggle/dist/wiggle-0.2.2-py3-none-any.whl
[toolshed install wiggle/dist/wiggle-0.2.2-py3-none-any.whl](cxcmd:toolshed
install wiggle/dist/wiggle-0.2.2-py3-none-any.whl)
Errors may have occurred when running pip:
pip standard error:
---
WARNING: Value for scheme.platlib does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX/1.3/site-packages
sysconfig: /home/cbeck/.local/lib/python3.9/site-packages
WARNING: Value for scheme.purelib does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX/1.3/site-packages
sysconfig: /home/cbeck/.local/lib/python3.9/site-packages
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX/include/python3.9/wiggle
sysconfig: /home/cbeck/.local/include/python3.9/wiggle
WARNING: Value for scheme.scripts does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX/bin
sysconfig: /home/cbeck/.local/bin
WARNING: Value for scheme.data does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX
sysconfig: /home/cbeck/.local
WARNING: Additional context:
user = True
home = None
root = None
prefix = None
WARNING: Value for scheme.headers does not match. Please report this to <https://github.com/pypa/pip/issues/10151>
distutils: /home/cbeck/.local/share/ChimeraX/include/python3.9/UNKNOWN
sysconfig: /home/cbeck/.local/include/python3.9/UNKNOWN
WARNING: You are using pip version 21.2.4; however, version 24.2 is available.
You should consider upgrading via the '/usr/bin/chimerax -m pip install --upgrade pip' command.
---
pip standard output:
---
Looking in indexes: https://pypi.org/simple, https://cxtoolshed.rbvi.ucsf.edu/pypi/
Processing ./wiggle/dist/wiggle-0.2.2-py3-none-any.whl
Requirement already satisfied: ChimeraX-UI~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from wiggle==0.2.2) (1.13.7)
Requirement already satisfied: ChimeraX-Core~=1.1 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from wiggle==0.2.2) (1.3)
Requirement already satisfied: ChimeraX-SelInspector~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-UI~=1.0->wiggle==0.2.2) (1.0)
Requirement already satisfied: ChimeraX-Graphics~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-UI~=1.0->wiggle==0.2.2) (1.1)
Requirement already satisfied: ChimeraX-MouseModes~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-UI~=1.0->wiggle==0.2.2) (1.1)
Requirement already satisfied: ChimeraX-Arrays~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-Graphics~=1.0->ChimeraX-UI~=1.0->wiggle==0.2.2) (1.0)
Requirement already satisfied: ChimeraX-Geometry~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-Graphics~=1.0->ChimeraX-UI~=1.0->wiggle==0.2.2) (1.1)
Requirement already satisfied: ChimeraX-ItemsInspection~=1.0 in /usr/lib/ucsf-chimerax/lib/python3.9/site-packages (from ChimeraX-SelInspector~=1.0->ChimeraX-UI~=1.0->wiggle==0.2.2) (1.0)
Installing collected packages: wiggle
Successfully installed wiggle-0.2.2
---
Successfully installed wiggle-0.2.2
Installed wiggle (0.2.2)
However, when I tried to launch wiggle, I got this error:
Traceback (most recent call last):
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/toolshed/info.py", line 554, in start_tool
ti = api._api_caller.start_tool(api, session, self, tool_info)
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/toolshed/__init__.py", line 1323, in start_tool
return cls._get_func(api, "start_tool")(session, bi, ti)
File "/home/cbeck/.local/share/ChimeraX/1.3/site-packages/chimerax/wiggle/__init__.py", line 31, in start_tool
raise ValueError("trying to start unknown tool: %s" % ti.name)
ValueError: trying to start unknown tool: wiggle
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/ui/gui.py", line 1659, in <lambda>
run(ses, "ui tool show %s" % StringArg.unparse(tool_name)))
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/commands/run.py", line 36, in run
results = command.run(text, log=log, return_json=return_json)
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/commands/cli.py", line 2856, in run
result = ci.function(session, **kw_args)
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/ui/cmd.py", line 219, in ui_tool_show
bi.start_tool(session, name)
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/toolshed/info.py", line 559, in start_tool
raise ToolshedError(
chimerax.core.toolshed.ToolshedError: start_tool() failed for tool wiggle in bundle wiggle:
trying to start unknown tool: wiggle
chimerax.core.toolshed.ToolshedError: start_tool() failed for tool wiggle in bundle wiggle:
trying to start unknown tool: wiggle
File "/usr/lib/ucsf-chimerax/lib/python3.9/site-packages/chimerax/core/toolshed/info.py", line 559, in start_tool
raise ToolshedError(
See log for complete Python traceback.
I saw in the installation instructions that the command devel clean /path/to/wiggle/' can fix some issues, so I ran it in ChimeraX. This removed the build/ and dist/ directories. After running
devel build /path/to/wiggle/and
devel install /path/to/wiggle/', the build/ and dist/ directories were rebuilt, and Wiggle would successfully launch. However, the `devel build /path/to/wiggle/ command seems to have restored the original version of miniDRGN.py, and not the one that I had changed. Do you have any idea what might be happening?
I can only get Wiggle to work properly if I launch ChimeraX using your original Wiggle repository and build wiggle using the miniDRGN.py script that I manually edited. I've attached the edited miniDRGN.py file, if you would like to reference it. miniDRGN.txt
Cheers, cbeck
It looks like this is a ChimeraX problem, but will probably be hard for me to help with if you are using the old ChimeraX 1.3. But two things would help. First you send "tried to reinstall wiggle" and then you list the output but you don't show the command you used to do the reinstall. When you run into errors you need to provide both the command you ran and the output you got. You could also attach your modified wiggle .whl file which would allow others to try to reproduce your error and understand it. A wild gues is that the problem is caused by you changing Wiggle without updating the wiggle version number from 0.2.2 to 0.2.3 so ChimeraX was confused by you trying to reinstall the same version. The .whl install code in ChimeraX has changed so much since ChimeraX 1.3 to fix problems like this it is hard to guess what the problem is.
The ChimeraX command "toolshed list" will list the installed tools so you can see if wiggle is installed. And if it is there but you get the error "trying to start unknown tool: wiggle" you might try ChimeraX command "toolshed uninstall
If you continue to have trouble with it you might try ChimeraX menu entry Help / Report a Bug... or press the Report a Bug button that is shown when you get the traceback trying to start wiggle. That will put you in touch with the actual ChimeraX developer who works on the toolshed (Greg Couch) who can probably give better advice than I can. I work on other aspects of ChimeraX (cryoEM, AlphaFold, VR).
Hi!
Clicking on a data point in the PCA plot doesn't generate a volume and instead gives the following error:
It's probably relevant that I wasn't able to load a
config.pkl
file, because as described in this post in March 2023, cryoDRGN now creates aconfig.yaml
file instead ofconfig.pkl
. I'm using cryoDRGN v3.3.3. Is there any way to recreate the originalconfig.pkl
file? Since it contains information on things like the neural network architecture, I assume it's required for the volume generator?Cheers, cbeck