Open curita opened 8 months ago
Same exception, it works fine locally, but not when deployed on Zyte's cloud.
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/spidermon/core/actions.py", line 39, in run
self.run_action()
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/email/__init__.py", line 110, in run_action
message = self.get_message()
^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/email/__init__.py", line 143, in get_message
body_html = self.get_body_html()
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/email/__init__.py", line 137, in get_body_html
html = transform(self.render_template(self.body_html_template))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/templates.py", line 57, in render_template
return template.render(self.get_template_context())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
self.environment.handle_exception()
File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
raise rewrite_traceback_stack(source=source)
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/reports/templates/reports/email/monitors/result.jinja", line 81, in top-level template code
{% macro render_header_data_separator() %}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/reports/templates/reports/email/bases/report/medium.jinja", line 1, in top-level template code
{% extends 'reports/email/bases/report/base.jinja' %}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/reports/templates/reports/email/bases/report/base.jinja", line 16, in top-level template code
{% block page_content %}{% endblock %}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/spidermon/contrib/actions/reports/templates/reports/email/monitors/result.jinja", line 136, in block 'page_content'
{% set is_script = data.job.metadata['spider'].startswith('py:') %}
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 485, in getattr
return getattr(obj, attribute)
^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'scrapinghub.client.jobs.JobMeta object' has no attribute 'spider'
Issue
data.job.metadata
(an instance ofscrapinghub.client.jobs:JobMeta
) cannot be subscribed. There are multiple places in Spidermon's templates where job meta keys are accessed viadata.job.metadata[x]
. Those cases fail because of it.It's unclear when
job.metadata
stopped being subscribable or if this is a change in the latest Jinja versions (see: https://jinja.palletsprojects.com/en/3.0.x/templates/#variables), but it doesn't seem to work now.Locally can be partially reproduced via:
Proposal
Replace all instances of
data.job.metadata[x]
in the templates withdata.job.metadata.get(x)
.