🐛 Bug: Error while fetching models eos1579 & eos2thm

[Femme-js]

Describe the bug.

While fetching the ersilia model eos2thm and eos1579, I am getting the following error:

''' EmptyOutputError

Error message: Model API eos1579:predict did not produce an output

'''

Describe the steps to reproduce the behavior

ersilia -v fetch eos1579

Expected behavior.

Model fetched successfully!

Screenshots.

Operating environment

Ubuntu 18.04.3 LTS

Additional context

No response

[Zainab-ik]

@Femme-js kindly add your log file for easy debugging.

[UChisom]

@Femme-js also cross-check and confirm all the required dependencies are installed.

• git-lfs (installed and activated)
• ersilia --help
• isaura
• and if it isn't this trivial, then your log file would help 👍

[Femme-js]

Thanks @Zainab-ik @UChisom

git-lfs is installed and activated. Still getting the same error.

Here is the log file. eos1579_fetch.log

[GemmaTuron]

Hi @Femme-js ,

Do you collect the log file with the |tee command?

[GemmaTuron]

I think I am missing part of the errors logs when we use the|tee command, as I have noted in other issues #343 and #355 Can you please run without it and paste the log error again?

[Femme-js]

Sure @GemmaTuron

[Femme-js]

@GemmaTuron eos1579.log

[GemmaTuron]

Thanks @Femme-js !

Let's try to understand what is happening, I can't reproduce the error, but let's see: On line 344 I see an error due to sqalchemy versions, but on line 409 the versions seem to be updated. Can you confirm which version of Sqalchemy package you have in the model conda environment? conda activate eos1579 and then I'd simply enter python and import sqlalchemy sqlalchemy.__version__

There is also the chardet urlib conflict but this seems only a warning

[GemmaTuron]

It would also be useful to run it in google colab, using the -v as well and try to see if you get the same error!

[GemmaTuron]

@miquelduranfrigola Let's see for this model I can't reproduce the error, see my log file eos1579.log

@Femme-js do the tests for sqlalchemy to make sure we have the right version and go on to test them on colab meanwhile ;)

[Femme-js]

Hi @GemmaTuron

THe sqlalchemy model I have is version 1.3.24'

[GemmaTuron]

Thanks @Femme-js !

• Do you have this error on Google Colab?
• Does eos2thm (as per the bug title) gives the same error?

[Femme-js]

Yes, I have same error with model eos1579 on google colab, with same log ouput.

However, model eos2thm shows an empty output error in CLI, but it successfully ran onto the collab and prediction was also successful.

One thing to note, the sqlalchemy version on my colab is diff (1.4.41) from the CLI

[GemmaTuron]

Ok @Femme-js That is interesting, I will need some time to dig deeper onto what is going on, I could reproduce the error on Google Colab as well...

See issue #369 from @sayantani11 and lets add all the information we have to try and debug it, but don't get stuck here, mark it in the excel as work in progress can you change the issue title to "eos1579 fetching does not produce output"

[Femme-js]

eos2thm.log Also attaching the log for eos2htm on CLI

@GemmaTuron Should I open a different issue for eos2thm, as it is producing the same error on CLI as eos1579, but successfully ran over collab?

[GemmaTuron]

Hello @Femme-js !

Can I ask you to:

1. Update pip and conda
2. Try fetching eos1579 again Meanwhile we are working on the chardet urllib conflict, to solve it and see if it was the cause of the error

[Femme-js]

Older Conda Version: 4.12.0 Older Pip Version: pip 22.2.2

New Conda Version: pip 22.2.2

New PiP Version: pip 22.3

Still having the same EmptyOutputError

eos15789UPDATED.log

However, the charset warning is not there anymore.

[GemmaTuron]

Thanks @Femme-js !

This might be a memory issue, specially because eos2thm is also failing... we need to do some work to implement memory errors, let's pause this until we have a better idea of how to solve it, I'll get back to you.

[miquelduranfrigola]

Agree - Memory issues will be difficult to catch. In any case, please try again because we have been doing quite a lot of interesting work on our "exceptions" module in Ersilia.

[GemmaTuron]

Hello @Femme-js,

An update from my side followig today's meeting:

• Model eos2thm does not give me any issues at fetch time in my WSL ¡probably it is a memory issue if you are encountering problems
• Model eos1579 does fail at fetch time with Cannot choose from an empty sequence in the WSL. Can you please update the errors you encounter with these models?

Thanks!

[Femme-js]

Hi @GemmaTuron

Model eos1579 fails to fetch in both colab and CLI. Going through the logs, I can see the error sniff package

eos157922.log

[Femme-js]

For some reason, API is not being called while fetching both of these models.

eos2thm222.log eos157922.log

[GemmaTuron]

Hi @Femme-js ,

As I mention above, I did not encounter errors with eos2thm and I had a different error for eos1579 so please can you:

• Specify which system you are using
• Ensure Ersilia is working in your system by testing another model, for example eos4e40
• Try to understand where might be the errors for the models, and try to give a more detailed explanation than above.
• Test the models in Google Colab as well and report if they have the same errors.

Thanks

[Femme-js]

Hi @GemmaTuron

I have ensured ersilia is working on my local system.

I was able to run eos2thm successfully on my colab, but on CLI the fetch failed. eos2thm222.log

eos1579 failed both with CLI and colab. It is giving different errors on CLI and colab. eos1579colab2.log (Colab Log) eos157922.log (CLI log)

With both the models on CLI, there is an issue with the sniff package that I am still figuring out.

On CLI: Ubuntu 18.04.6 LTS Python 3.7.13 conda version - conda 22.9.0 pip version - pip 22.2.2 SQLAlchemy==1.3.24 BentoML==0.11.0

On Colab: Python 3.7.13 conda 4.12.0 SQLAlchemy==1.3.24 bentoml==0.11.0

[Femme-js]

Currently, I am testing both models with updating the conda version on colab.

[GemmaTuron]

Hi, quick update to do some cleaning on the open github issues:

• eos2thm fetches right in the colab and wsl but sometimes it fails due to its large size in WSL, depending on user internet and available space @miquelduranfrigola anything we could do here?
• eos1579 the conversation will be kept in #369