search

aiidateam / aiida-core

The official repository for the AiiDA code
https://aiida-core.readthedocs.io
Other
434 stars 186 forks source link

Improve error message when daemon cannot import process class #2410

Open ltalirz opened 5 years ago

ltalirz commented 5 years ago

I'm trying to run my first workchain on provenance_redesign.

But I get the following types of errors:

-/Users/leopold/Applications/miniconda3/envs/aiida_rmq/lib/python2.7/site-packages/psycopg2/__init__.py:144: UserWarning: The psycopg2 wheel package will be renamed from release 2.8; in order to keep installing from binary please use "pip install psycopg2-binary" instead. For details see: <http://initd.org/psycopg/docs/install.html#binary-install-from-pypi>.
  """)
01/22/2019 11:11:10 AM <10858> plumpy.processes: [ERROR] Exception trying to save checkpoint, this means you will not be able to restart in case of a crash until the next successful checkpoint.
Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/work/processes.py", line 198, in _save_checkpoint
    self.runner.persister.save_checkpoint(self)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/work/persistence.py", line 67, in save_checkpoint
    process, traceback.format_exc()))
PersistenceError: Failed to create a bundle for '<ListEcho> (ProcessState.CREATED)': Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/work/persistence.py", line 63, in save_checkpoint
    bundle = plumpy.Bundle(process, plumpy.LoadSaveContext(loader=get_object_loader()))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq/lib/python2.7/site-packages/plumpy/persistence.py", line 46, in __init__
    self.update(savable.save(save_context))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq/lib/python2.7/site-packages/plumpy/persistence.py", line 508, in save
    Savable._set_class_name(out_state, loader.identify_object(self.__class__))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq/lib/python2.7/site-packages/plumpy/loaders.py", line 62, in identify_object
    self.load_object(identifier)
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq/lib/python2.7/site-packages/plumpy/loaders.py", line 57, in load_object
    raise ValueError("object '{}' form identifier '{}' could not be loaded".format(name, identifier))
ValueError: object 'ListEcho' form identifier '__main__:ListEcho' could not be loaded

The code I'm trying to run is the following (copied from .ci/workflows.py):

# run simple workchain
from aiida.work.launch import run_get_node
from aiida.work.workchain import WorkChain, ToContext, append_
from aiida.work import run
from aiida.orm.data.base import Int, List

class ListEcho(WorkChain):
    @classmethod
    def define(cls, spec):
        super(ListEcho, cls).define(spec)

        spec.input('list', valid_type=List)
        spec.output('output', valid_type=List)

        spec.outline(cls.do_echo)

    def do_echo(self):
        self.out('output', self.inputs.list)

expected_results = {}

list_value = List()
list_value.extend([1, 2, 3])
pk = run(ListEcho, list=list_value).pk
expected_results[pk] = list_value
ltalirz commented 5 years ago

It turns out this error arises because the workchain was defined in the same file from where it was run. If the workchain definition is put into a separate file and imported from there, the problem disappears.

Independent of whether this is really necessary, I think the error message should be a bit more helpful in this regard. How could this be improved?

sphuber commented 5 years ago

The same error also appears when the workchain cannot be found in the python path. It being in the same file might have the same origin as that problem. If these are two separate causes I am not sure the loader can determine the reason. But at least we could maybe update the exception message to give a suggestion to make sure the workchain is in the python path

ltalirz commented 4 years ago

Just to mention that when adapting the script to aiida 1.0

# run simple workchain
from aiida.engine import run
from aiida.engine import WorkChain, ToContext, append_
from aiida.orm import Int, List

class ListEcho(WorkChain):
    @classmethod
    def define(cls, spec):
        super(ListEcho, cls).define(spec)

        spec.input('list', valid_type=List)
        spec.output('output', valid_type=List)

        spec.outline(cls.do_echo)

    def do_echo(self):
        self.out('output', self.inputs.list)

expected_results = {}

my_list = List()
my_list.extend([1, 2, 3])
result = run(ListEcho, list=my_list)
print(result)

the error message is still thrown, although it seems like the script now actually manages to run properly with verdi run.


$ verdi run wc.py
11/11/2019 12:16:29 PM <83083> plumpy.processes: [ERROR] Exception trying to save checkpoint, this means you will not be able to restart in case of a crash until the next successful checkpoint.
Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 50, in load_object
    return getattr(module, name)
AttributeError: module '__main__' has no attribute 'ListEcho'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 83, in save_checkpoint
    bundle = plumpy.Bundle(process, plumpy.LoadSaveContext(loader=get_object_loader()))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 40, in __init__
    self.update(savable.save(save_context))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 504, in save
    Savable._set_class_name(out_state, loader.identify_object(self.__class__))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/loaders.py", line 62, in identify_object
    self.load_object(identifier)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 52, in load_object
    raise ImportError("object '{}' from identifier '{}' could not be loaded".format(name, identifier))
ImportError: object 'ListEcho' from identifier '__main__:ListEcho' could not be loaded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/processes/process.py", line 207, in _save_checkpoint
    self.runner.persister.save_checkpoint(self)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 87, in save_checkpoint
    "Failed to create a bundle for '{}': {}".format(process, traceback.format_exc())
plumpy.exceptions.PersistenceError: Failed to create a bundle for '<ListEcho> (ProcessState.CREATED)': Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 50, in load_object
    return getattr(module, name)
AttributeError: module '__main__' has no attribute 'ListEcho'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 83, in save_checkpoint
    bundle = plumpy.Bundle(process, plumpy.LoadSaveContext(loader=get_object_loader()))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 40, in __init__
    self.update(savable.save(save_context))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 504, in save
    Savable._set_class_name(out_state, loader.identify_object(self.__class__))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/loaders.py", line 62, in identify_object
    self.load_object(identifier)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 52, in load_object
    raise ImportError("object '{}' from identifier '{}' could not be loaded".format(name, identifier))
ImportError: object 'ListEcho' from identifier '__main__:ListEcho' could not be loaded

11/11/2019 12:16:29 PM <83083> aiida.orm.nodes.process.workflow.workchain.WorkChainNode: [ERROR] Exception trying to save checkpoint, this means you will not be able to restart in case of a crash until the next successful checkpoint.
Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 50, in load_object
    return getattr(module, name)
AttributeError: module '__main__' has no attribute 'ListEcho'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 83, in save_checkpoint
    bundle = plumpy.Bundle(process, plumpy.LoadSaveContext(loader=get_object_loader()))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 40, in __init__
    self.update(savable.save(save_context))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 504, in save
    Savable._set_class_name(out_state, loader.identify_object(self.__class__))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/loaders.py", line 62, in identify_object
    self.load_object(identifier)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 52, in load_object
    raise ImportError("object '{}' from identifier '{}' could not be loaded".format(name, identifier))
ImportError: object 'ListEcho' from identifier '__main__:ListEcho' could not be loaded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/processes/process.py", line 207, in _save_checkpoint
    self.runner.persister.save_checkpoint(self)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 87, in save_checkpoint
    "Failed to create a bundle for '{}': {}".format(process, traceback.format_exc())
plumpy.exceptions.PersistenceError: Failed to create a bundle for '<ListEcho> (ProcessState.RUNNING)': Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 50, in load_object
    return getattr(module, name)
AttributeError: module '__main__' has no attribute 'ListEcho'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 83, in save_checkpoint
    bundle = plumpy.Bundle(process, plumpy.LoadSaveContext(loader=get_object_loader()))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 40, in __init__
    self.update(savable.save(save_context))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/persistence.py", line 504, in save
    Savable._set_class_name(out_state, loader.identify_object(self.__class__))
  File "/Users/leopold/Applications/miniconda3/envs/aiida_rmq_py3/lib/python3.7/site-packages/plumpy/loaders.py", line 62, in identify_object
    self.load_object(identifier)
  File "/Users/leopold/Personal/Postdoc-MARVEL/repos/aiida/aiida_rmq/aiida/engine/persistence.py", line 52, in load_object
    raise ImportError("object '{}' from identifier '{}' could not be loaded".format(name, identifier))
ImportError: object 'ListEcho' from identifier '__main__:ListEcho' could not be loaded

{'output': <List: uuid: b0b0dc09-dc8f-4ed6-920a-8b36e5096593 (pk: 1835) value: [1, 2, 3]>}

@sphuber I'm not quite sure at which point this should be improved.

Perhaps somewhere down the line, there should be a check whether the module name is __main__, and then point out that the process needs to be defined in a separate script?