Closed github-actions[bot] closed 5 months ago
GemmaTuron
commented
1 year ago With the latest modifications @miquelduranfrigola , when I try to fetch the model from repo_path, it tries anyway to go via docker and the fetching fails. It works though just fetching directly from docker, which is a bit puzzling
DhanshreeA
commented
6 months ago Hi @GemmaTuron is this still happening?
With the latest modifications @miquelduranfrigola , when I try to fetch the model from
repo_path, it tries anyway to go via docker and the fetching fails. It works though just fetching directly from docker, which is a bit puzzling
GemmaTuron
commented
6 months ago I get the following error on my Mac M2 when fetching from repopath. I need to try from an ubuntu machine
12:07:43 | INFO | Deleting conda environment eos4qda
12:07:44 | DEBUG | Attempting temporary folder delete
12:07:44 | DEBUG | Attempting lake delete (local)
12:07:44 | DEBUG | Deleting /Users/gemmaturon/eos/isaura/lake/eos4qda_local.h5
12:07:44 | DEBUG | Attempting lake delete (public)
12:07:44 | DEBUG | Deleting /Users/gemmaturon/eos/isaura/lake/eos4qda_public.h5
12:07:44 | INFO | Removing docker images and stopping containers related to eos4qda
12:07:44 | DEBUG | Model entry eos4qda was not available in the fetched models registry
12:07:44 | SUCCESS | Model eos4qda deleted successfully
12:07:44 | DEBUG | Preparing inner template if necessary
12:07:44 | DEBUG | The pack.py file already exists
12:07:44 | DEBUG | The Dockerfile file already exists
12:07:44 | DEBUG | The src folder already exists
12:07:44 | DEBUG | Changing python version if necessary
12:07:44 | DEBUG | None
12:07:44 | DEBUG | User is not root
12:07:44 | DEBUG | Example file exists
Traceback (most recent call last):
File "/Users/gemmaturon/miniconda3/envs/ersilia/bin/ersilia", line 8, in <module>
sys.exit(cli())
^^^^^
File "/Users/gemmaturon/miniconda3/envs/ersilia/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/miniconda3/envs/ersilia/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/miniconda3/envs/ersilia/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/miniconda3/envs/ersilia/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/miniconda3/envs/ersilia/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/cli/commands/__init__.py", line 22, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 89, in fetch
_fetch(mf, model_id)
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 12, in _fetch
mf.fetch(model_id)
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 228, in fetch
self._fetch(model_id)
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 225, in _fetch
self._fetch_not_from_dockerhub(model_id=model_id)
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 135, in _fetch_not_from_dockerhub
self._pack()
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 99, in _pack
mp.pack()
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 61, in pack
self._setup()
File "/Users/gemmaturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 30, in _setup
raise Exception
ExceptionHi @DhanshreeA Same issue as with the Mac M2 on an Ubuntu Machine. I have pulled the latest Ersilia code which had a lot of changes from the previous version, so just to confirm I am using the latest version locally
Traceback (most recent call last):
File "/home/gturon/miniconda3/envs/ersilia/bin/ersilia", line 8, in <module>
sys.exit(cli())
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/__init__.py", line 22, in wrapper
return func(*args, **kwargs)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 89, in fetch
_fetch(mf, model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 12, in _fetch
mf.fetch(model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 228, in fetch
self._fetch(model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 225, in _fetch
self._fetch_not_from_dockerhub(model_id=model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 135, in _fetch_not_from_dockerhub
self._pack()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 99, in _pack
mp.pack()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 61, in pack
self._setup()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 30, in _setup
raise Exception
Exception@GemmaTuron my bad - this model has been built the old way, and the workflow overwrote the model Dockerfile which is causing this error. It is similar to https://github.com/ersilia-os/eos3ae6/issues/12#issuecomment-2088189441. Fixing this now.
DhanshreeA
commented
6 months ago With the latest modifications @miquelduranfrigola , when I try to fetch the model from
repo_path, it tries anyway to go via docker and the fetching fails. It works though just fetching directly from docker, which is a bit puzzling
@GemmaTuron regarding this - it happens, I am assuming only on Mac M1 and above, because the model is based on Python 3.8 which is not supported on ARM machines (only Python 3.10 and above are supported). This is why Ersilia tries to build Docker anyway. I am running into this too now when I run it on my mac with repo_path.
GemmaTuron
commented
6 months ago @DhanshreeA
This model is still trying to access Docker when being fetched from repo path (in an Ubuntu machine)
docker: Cannot connect to the Docker daemon at unix:///home/gturon/.docker/desktop/docker.sock. Is the docker daemon running?.
See 'docker run --help'.
12:38:35 | DEBUG | Executing container 3c68e039-d4c8-45d3-929d-aeacfa4fc83c
Cannot connect to the Docker daemon at unix:///home/gturon/.docker/desktop/docker.sock. Is the docker daemon running?
12:38:35 | DEBUG | Copying bundle from docker image to host
12:38:35 | DEBUG | Using this temporary directory: /tmp/ersilia-b5grw_bz
12:38:35 | DEBUG | Loading bentoml
Traceback (most recent call last):
File "/home/gturon/miniconda3/envs/ersilia/bin/ersilia", line 8, in <module>
sys.exit(cli())
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/gturon/miniconda3/envs/ersilia/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/__init__.py", line 22, in wrapper
return func(*args, **kwargs)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 99, in fetch
_fetch(mf, model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/cli/commands/fetch.py", line 12, in _fetch
mf.fetch(model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 228, in fetch
self._fetch(model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 225, in _fetch
self._fetch_not_from_dockerhub(model_id=model_id)
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 135, in _fetch_not_from_dockerhub
self._pack()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/fetch.py", line 99, in _pack
mp.pack()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 63, in pack
self._run()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/actions/pack.py", line 58, in _run
runner.run()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/pack/runners.py", line 216, in run
self._run()
File "/home/gturon/github/ersilia-os/ersilia/ersilia/hub/fetch/pack/runners.py", line 209, in _run
mdl.save()
AttributeError: 'NoneType' object has no attribute 'save'
Cannot connect to the Docker daemon at unix:///home/gturon/.docker/desktop/docker.sock. Is the docker daemon running?fyi
If I activate docker it tries to fetch the model via Docker not repo_path even though it is specified, and it fails
DhanshreeA
commented
6 months ago @GemmaTuron do you have a newer version of Ersilia? And do you have conda installed on this machine? I am unable to reproduce this - could you try running this on Codespaces and see if Ersilia still tries to use docker instead of conda?
GemmaTuron
commented
6 months ago Hi @DhanshreeA I have the latest ersilia code pulled and conda of course with the appropriate ersilia environment. I do not see the same issue in codespaces but I do not know what is different in my local system. I have everything freshly installed. Unless it is related to where Ersilia was looking for some packages, if you recall we ran into these issues a while ago.
Potentially there might have been issues with all the Ersilia changes and legacy code and installs remaining in my system that are causing trouble, I have deleted and created the ersilia env again but with same results so far. Any ideas aside from reinstalling miniconda which I am not keen to do?
DhanshreeA
commented
6 months ago Let me think about this and get back to you by EOD
DhanshreeA
commented
6 months ago Hi @GemmaTuron we've added modified this repository to force ersilia to build a conda environment from repo_path instead of trying to build a docker image. Could you pull the latest code from this repository, and try again?
Also please share the full logs produced from a verbose run so we can debug better. Thanks!
GemmaTuron
commented
5 months ago Hello @DhanshreeA
This is what I get now:
(ersilia) gturon@pujarnol:~/github/ersilia-os$ ersilia -v fetch eos4qda --repo_path eos4qda > out.log 2>&1
[sudo] password for gturon: It is asking for my sudo password, what is it trying to install that requires this? It should not work like that in any model
DhanshreeA
commented
5 months ago Hi @GemmaTuron I looked at this model again and have a few questions/comments:
GemmaTuron
commented
5 months ago Hi @DhanshreeA
I'll close this issue as we have decided there is no way around the requirement of sudo credentials. We have also decided to look into how to pass keys in general (for example API keys) to models, as we might need them in the near future. Maybe this pertains to a whole individual issue on the main repo
This model is ready for testing. If you are assigned to this issue, please try it out using the CLI, Google Colab and DockerHub and let us know if it works!