🐛 Bug: Test on Colab workflow keeps timing out and ends up failing.

[DhanshreeA]

Describe the bug.

GitHub runners by default a time limit of 6 hours. A job that takes longer than that fails because it gets canceled. The Test on Colab workflow seems to be stuck(?) and keeps running until the job automatically gets canceled. We will need to investigate this.

Describe the steps to reproduce the behavior

No response

Expected behavior.

No response

Screenshots.

No response

Operating environment

NA

Additional context

No response

[miquelduranfrigola]

I don't know what is happening to be honest. This is annoying and I am unsure whether it is related to Ersilia or not. Actually, it is not testing "Colab" per se, it is just testing the notebook. In my opinion, this test may eventually disappear since it periodically fails for some reason.

[DhanshreeA]

In the last discussion we came to the conclusion that currently the workflow is not representative of actually testing anything on Colab. It is also likely that the action times out because within the notebook we are fetching a model that hasn't been rebuilt and fails to fetch, which leaves ersilia hanging and therefore the entire workflow times out.

In any case, we have decided to redo this workflow to test a notebook in an actual Colab environment, something like this SO post: https://stackoverflow.com/questions/77437999/google-colab-environment-container-testing-in-github-action

[GemmaTuron]

Hi @DhanshreeA

There is a couple issues related to colab still open as well. Can you mayeb summarise if this is now working or we are discontinuing support?

[DhanshreeA]

@GemmaTuron This workflow itself presently works. In the current implementation we are using the Ersilia CLI to run a model and we are executing this notebook using jupyter's nbconvert. This approach is acting as a proxy for actually running a notebook. Additionally, at present, we are skipping testing the Python API in this way because of some 'unresolved issue' (I will have to test the Python API to understand what it is as it has been a while).

In the discussion above, we decided to use official Colab image from Google to actually run an Ersilia usage tutorial on a notebook.

This is unrelated to the other Colab related issue (#789) which is concerned with figuring out a way to run docker containers within Colab and supporting that functionality from within the Ersilia CLI.

As a concluding remark, a reimplementation of this test would be a nice to have however it is not a priority.

[GemmaTuron]

Hi @DhanshreeA ,

Sorry, I do not understand. From the issue you mention, Ersilia does not work in Colab, so then how come can we have a GitHub Action that testes it? Or Ersilia works in Colab?

[DhanshreeA]

@GemmaTuron Let me explain:

There are two Colab related issues:

1. Docker containers do not easily run on Colab (#789), but Ersilia does because anytime you try to fetch a model in Colab, at least with the CLI, the functionality within Ersilia which resolves how to fetch a model does not find the docker engine running and therefore ends up fetching the model from GitHub.
2. The second issue is that the current implementation of testing a Colab notebook, that contains a Ersilia tutorial and setup instructions etc, is in itself not good for two reasons:
   • We use a roundabout way of using nbconvert to save a notebook after executing it. However, we don't really check the saved notebook for errors/warnings after this step.
   • It is not actually testing the notebook in a "Colab" environment, since the ask here is to make sure a notebook with Ersilia works in Colab. Based on the discussion above, the idea is to use a Colab docker image and run the notebook in that container to really test it within Colab.