'Database busy with too many connections' warning when importing pyiron

[pkruzikova]

When trying to from pyiron/pyiron_base/pyiron_atomistics import Project in a jupyter notebook connected to my hpc pyiron installation, I'm getting a 'Database busy with too many connections' warning - shows a couple times, then going into an error (full output attached). The strange thing is, this used to work just fine, now today suddenly throwing errors. Also, when I try the same with a local (Windows) installation of pyrion (exact same versions), it works fine.

Would appreciate ideas about what could be wrong, thanks!

pyiron_project_import_error.txt Versions used: pyiron 0.5.2, pyiron_base 0.8.2, pyiron_atomistics 0.5.2

### Tasks

[jan-janssen]

Hi @pkruzikova

can you remind me which database you are using? Can you share your ~/.pyiron configuration and please make sure to remove any passwords inside the configuration file before posting.

Best,

Jan

[pkruzikova]

Hi, the ~/.pyiron file is as follows:

[DEFAULT]
RESOURCE_PATHS = /path/to/my/home/pyiron_resources, /path/to/my/home/mambaforge/envs/pyiron_rdm/share/pyiron (and paths to other environments I'm using)
#PROJECT_CHECK_ENABLED = False
#DISABLE_DATABASE = True
FILE = ~/pyiron.db

What I've found (and is most likely a problem) is that in /path/to/my/home/mambaforge/envs/pyiron_rdm/share (and the other environment folders), there currently isn't a pyiron folder at all. (Frankly, I never checked this before as things worked fine but I'm assuming it did use to be there.)

Best, Pavlina

[jan-janssen]

Can you try to modify the ~/.pyiron file in the following way?

[DEFAULT]
RESOURCE_PATHS = /path/to/my/home/pyiron_resources, /path/to/my/home/mambaforge/envs/pyiron_rdm/share/pyiron (and paths to other environments I'm using)
PROJECT_CHECK_ENABLED = False
DISABLE_DATABASE = True
# FILE = ~/pyiron.db

The SQLite database ~/pyiron.db can only be accessed by a single compute node at the time. So there are primarily two options, either to disable the database as I suggest it here or install a central SQL database like PostgreSQL or MySQL but that might require special privileges on the cluster.

[pkruzikova]

The first suggestion helped, the import works now - thanks!

[ligerzero-ai]

This results in a mendeleev diskio import error when trying to load old jobs from a project - any idea?

[pkruzikova]

Attaching the whole error (after calling pr.load) if that helps in any way.

pyiron_job_load_error.txt

[pkruzikova]

Hi @jan-janssen, would you mind taking a look at this, please? Thanks!

[jan-janssen]

Hi @pkruzikova ,

it is a bit hard to say just based on the information in the issue. can you email me and then we arrange a meeting at the beginning of next week?

Best,

jan

[jan-janssen]

It seems to be an issue with a broken mendeleev package. To fix the database you can use the following steps:

Figure out where Mendeleev is located, in a python shell do:

import mendeleev
mendeleev.__file__

For me this returns:

'/srv/conda/envs/notebook/lib/python3.12/site-packages/mendeleev/__init__.py'

On the command line:

wget https://files.pythonhosted.org/packages/53/7e/d5feb5ca1be737be0fc670262851e98d3c580a0f5fb6e6d245137e362e31/mendeleev-0.16.2.tar.gz
tar -xf mendeleev-0.16.2.tar.gz 
cd mendeleev-0.16.2/mendeleev
cp elements.db /srv/conda/envs/notebook/lib/python3.12/site-packages/mendeleev

Unfortunately this also did not work ...

[jan-janssen]

After one more meeting we were able to resolve the issue. mendeleev uses an SQLite database and SQLite databases are not supported on network file systems, at least when security enhanced linux is enabled as it blocks the option to lock files. This was resolved by switching mendeleev to a read-only connection to the SQLite database. The corresponding changes were pushed up-stream https://github.com/lmmentel/mendeleev/pull/165 and get merged in pyiron_atomistics as soon as the conda package is available https://github.com/pyiron/pyiron_atomistics/pull/1442