Closed bgruening closed 5 years ago
Mh, it seems even worse. I have now fixed all our tools. This means all tools are updated and upgraded. The automatic installation, however, reset repositories. This is because we do have non-installable revisions in our YAML file. Revisions have been added in the past, they do not need to be strictly the latest installable one. That is not optimal, but I think those revisions should map indirectly to the next installable-revision. What happens currently, I think, is that this repositories crash with the below stacktrace and are then in "new" state. I can then indeed install that old revision - the repo is then shown as "there is an update available" ... and I can do the update (I do not mean the upgrade here).
@natefoo how do you handle that? Have you seen similar things?
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: galaxy.web.framework.decorators ERROR 2019-11-10 23:53:09,788 [p:46171,w:1,m:0] [uWSGIWorker1Core0] Uncaught exception in exposed API method:
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: Traceback (most recent call last):
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/galaxy/web/framework/decorators.py", line 282, in decorator
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: rval = func(self, trans, *args, **kwargs)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/galaxy/webapps/galaxy/api/tool_shed_repositories.py", line 531, in install_repository_revision
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: payload)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/install_manager.py", line 700, in install
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: install_options
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/install_manager.py", line 799, in __initiate_and_install_repositories
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: return self.install_repositories(tsr_ids, decoded_kwd, reinstalling=False, install_options=install_options)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/install_manager.py", line 846, in install_repositories
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: tool_panel_section_mapping=tool_panel_section_mapping)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/install_manager.py", line 874, in install_tool_shed_repository
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: install_options=install_options)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/install_manager.py", line 976, in update_tool_shed_repository
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: persist=True)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/galaxy_install/metadata/installed_repository_metadata_manager.py", line 29, in __init__
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: metadata_dict=metadata_dict, user=None)
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: File "lib/tool_shed/metadata/metadata_generator.py", line 64, in __init__
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: self.metadata_dict = {'shed_config_filename': self.shed_config_dict.get('config_filename', None)}
Nov 10 23:53:09 sn04.bi.uni-freiburg.de uwsgi[3615164]: AttributeError: 'NoneType' object has no attribute 'get'
Looking into it now, I see a couple of things we can do to make this more robust.
OK, I think I have fixed all of the issues in https://github.com/galaxyproject/galaxy/pull/8959, except the syntax error issue when parsing the integrated_tool_panel.xml file, but that shouldn't block installation. I've tested this manually and I think it works. I'll see if I can also come up with some integration tests for non-installable revision updates.
Should be fixed by https://github.com/galaxyproject/galaxy/pull/8959. Hope the next round of updates will go smoothly @bgruening!
I updated today the branch and currently it looks too good to be true: https://build.galaxyproject.eu/job/usegalaxy-eu/job/install-tools/161/console
:heart:
I'm trying to understand whats happening, but it seems that newly installed tools are not added to the shed_tool_conf.yml ...
I swear I have seen it in the integrated_tool_conf ... but it's not in the shed_tool_conf ... :(
I will find time to investigate, but @natefoo maybe hold of running anything against main.
Just installing snippy, guess we will see soon. https://jenkins.galaxyproject.org/job/usegalaxy-tools/111/console
1: All repositories with a minor update, like Alevin (908a8d400fa2)
will be turned into the
new
state during the update with ephemeris2: Going from
new
state toinstall
Here it would help now to also install the repo if it is either in
new
state or inerror
state I think.The above seems just to be temporary ... as the file looks ok all the time I lint it.