search

aiida-vasp / aiida-vasp

A plugin to AiiDA for running simulations with VASP
https://aiida-vasp-plugin.readthedocs.org/en/latest
Other
48 stars 45 forks source link

Possible issue with POTCARs #728

Closed nkeilbart closed 1 month ago

nkeilbart commented 1 month ago

Hello everyone,

I am having an issue with submitting calculations with the aiida-vasp plugin. I don't remember what I originally did to make this happen but I believe it was something to do with the POTCARs. I also had issues with my local repository and that might be causing the issue. I might have been attempting to delete them to upload the newer potentials at some point and then I am unable to submit any jobs. I am currently on the most recent aiida-core and aiida-vasp versions.

I am simply attempting to run the run_relax.py example and have modified the script to load my codes that are saved in AiiDA. The report that gets generated from the VaspCalculation is:

report 2736494 2736494: None Scheduler output: N/A Scheduler errors: N/A 1 LOG MESSAGES: +-> REPORT at 2024-09-10 01:25:33.525704+00:00 [2736494 VaspCalculation on_except]: Traceback (most recent call last): File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/tasks.py", line 91, in do_upload calc_info = process.presubmit(folder) File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/calcjob.py", line 866, in presubmit calc_info = self.prepare_for_submission(folder) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/calcs/vasp.py", line 329, in prepare_for_submission calcinfo = super().prepare_for_submission(folder) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/calcs/base.py", line 96, in prepare_for_submission self.write_potcar(potentials) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/calcs/vasp.py", line 518, in write_potcar multi_potcar.write(dst) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/parsers/content_parsers/potcar.py", line 161, in write dest_fo.write(potcar.content) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/parsers/content_parsers/potcar.py", line 115, in content return self.file_node.get_content() File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/data/potcar.py", line 558, in get_content with self.get_file_obj() as potcar_fo: File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/data/potcar.py", line 490, in get_file_obj with self.get_archive() as archive: File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/WS1/keilbart/AiiDA/git/aiida-vasp/src/aiida_vasp/data/archive.py", line 26, in get_archive with self.base.repository.open('archive.tar.gz', mode='rb') as fobj: # pylint: disable=not-context-manager File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/orm/nodes/repository.py", line 198, in open with self._repository.open(path) as handle: File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/repository/repository.py", line 394, in open with self.backend.open(key) as handle: File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 135, in enter return next(self.gen) File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/repository/backend/disk_object_store.py", line 109, in open super().open(key) File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 281, in helper return _GeneratorContextManager(func, args, kwds) File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/contextlib.py", line 103, in init self.gen = func(*args, **kwds) File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/repository/backend/abstract.py", line 161, in open raise FileNotFoundError(f'object with key {key} does not exist.') FileNotFoundError: object with key a9b9d3678fc170d6b66fedf6a5d05471e44d5ff79d012aeb57ff577c320f3232 does not exist.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/workspace/keilbart/envs/aiida/lib/python3.10/site-packages/plumpy/processes.py", line 1221, in step
next_state = await self._run_task(self._state.execute)
File "/usr/workspace/keilbart/envs/aiida/lib/python3.10/site-packages/plumpy/processes.py", line 570, in _run_task
result = await coro(*args, **kwargs)
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/tasks.py", line 503, in execute
skip_submit = await self._launch_task(task_upload_job, self.process, transport_queue)
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/tasks.py", line 623, in _launch_task
result = await self._task
File "/collab/usr/gapps/python/build/spack-toss4.2/var/spack/environments/python310/._view/i74j3perqsso644fvgbrjnzhyuvpn42l/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/utils.py", line 137, in execute_coroutine
result = await coro(future)
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/tasks.py", line 105, in task_upload_job
skip_submit = await exponential_backoff_retry(
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/utils.py", line 205, in exponential_backoff_retry
result = await coro()
File "/usr/WS1/keilbart/AiiDA/aiida-core/src/aiida/engine/processes/calcjobs/tasks.py", line 93, in do_upload
raise PreSubmitException('exception occurred in presubmit call') from exception
aiida.engine.processes.calcjobs.tasks.PreSubmitException: exception occurred in presubmit call

Things I have attempted to remedy this is delete all previous instances of POTCARs in the group listing when typing verdi group list --all and deleting the appropriate PKs with the --delete-nodes option. This was able to clear the POTCAR families from the group list as well as the verdi data vasp-potcar listfamilies.

I'm wondering if there is some kind of caching going on that is detecting the original files that were uploaded and causing issues? Any assistance with this would be appreciated. Thanks.

Nathan

zhubonan commented 1 month ago

In aiida-vasp there is PotcarData and PotcarFileData. The intention is to hide the actual POTCAR file (PotcarFileData) in the provenance graph so when one exports the calculations, the actual POTCAR files are not shared (due to licensing issue).

From your stack trace, it seems the problem is in the storage of your AiiDA installation - the repository file of the PotcarFileDatais missing which make it not possible to construct the actual POTCAR files of the calculation.

When you delete a family, only the PotcarData is deleted. When you re-upload, aiida-vasp finds the existing PotcarFileDataand link PotcarData with it without making new PotcarFileData. The problem is that there is no check of the PotcarFileData's completeness.

To solve this, you need to manually delete all PotcarFileData then re-upload your potcar family. You can find all PotcarFileData node using this in verdi shell:

from aiida_vasp.data.potcar import PotcarFileData
q = QueryBuilder().append(PotcarFileData)
q.all()

However, I would suggest investigating why your storage is broken in the first place. Have you accidentally delete the loose folder of your storage?

Finally, the run_relax.py is out-dated and may not be compatible with the latest code (aiida-vasp 4.0). It will be removed/updated in the future.

nkeilbart commented 1 month ago

Thank you for clarifying this issue. By deleting those instances in the database I was able to reupload the files and get the simulations working again.

Yes I had naively delete some of the files at some point as I was having an issue with storage space when one of the other plugins I was developing started download cube files. Needless to say those take a lot of space and I was trying to clean that up but I do remember messing a lot of things up at that time. I'll make sure not to make the same mistake.