alephdata / memorious

Lightweight web scraping toolkit for documents and structured data.
https://docs.alephdata.org/developers/memorious
MIT License
311 stars 59 forks source link

malformed yaml results to an error in listing crawlers #217

Open zufanka opened 1 year ago

zufanka commented 1 year ago

I got this error when one of my yaml files was not formatted properly. Not completely sure why it happens like this, also if this is the only time this happens, but there is an empty object being passed on, hence the NoneType. This error message is quite confusing, can it be something like "One of your yaml files is probably malformatted." ?

$ memorious list
Traceback (most recent call last):
  File "/usr/local/bin/memorious", line 33, in <module>
    sys.exit(load_entry_point('memorious', 'console_scripts', 'memorious')())
  File "/memorious/memorious/cli.py", line 225, in main
    cli(obj={})
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1137, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1062, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1668, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.8/dist-packages/click/core.py", line 763, in invoke
    return __callback(*args, **kwargs)
  File "/memorious/memorious/cli.py", line 174, in index
    crawler.name,
AttributeError: 'NoneType' object has no attribute 'name'