search

inveniosoftware / cookiecutter-invenio-rdm

Cookiecutter template for a new InvenioRDM instance.
MIT License
4 stars 33 forks source link

investigate lxml issue with uwsgi multiple interpreters #89

Closed zzacharo closed 3 years ago

zzacharo commented 4 years ago

The issue is referring to the invenio-cli containerize command. See https://github.com/onelogin/python3-saml/issues/121.

Summary: It turned out, we are using uwsgi with threads and each thread is creating its own interpreter. and that causes lxml to fail with an error

[2020-06-03 12:23:09,813] ERROR in app: Failed to initialize entry point: invenio_pidstore = invenio_pidstore:InvenioPIDStore
Traceback (most recent call last):
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_app/wsgi_rest.py", line 16, in <module>
    application = create_api()
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 113, in _create_app
    modules=extensions,
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 174, in app_loader
    modules=modules)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 233, in _loader
    init_func(ep.load())
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_base/app.py", line 173, in <lambda>
    _loader(app, lambda ext: ext(app), entry_points=entry_points,
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_pidstore/ext.py", line 100, in __init__
    fetchers_entry_point_group=fetchers_entry_point_group
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_pidstore/ext.py", line 154, in init_app
    fetchers_entry_point_group=fetchers_entry_point_group,
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_pidstore/ext.py", line 45, in __init__
    self.load_minters_entry_point_group(minters_entry_point_group)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_pidstore/ext.py", line 73, in load_minters_entry_point_group
    self.register_minter(ep.name, ep.load())
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2462, in load
    return self.resolve()
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2468, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_oaiserver/minters.py", line 19, in <module>
    from .utils import datetime_to_datestamp
  File "/opt/rh/rh-python36/root/usr/lib/python3.6/site-packages/invenio_oaiserver/utils.py", line 17, in <module>
    from lxml import etree
ImportError: Interpreter change detected - this module can only be loaded into one interpreter per process.
ppanero commented 4 years ago

Recovering the blogpost mentioned in this issue (https://github.com/inveniosoftware/troubleshooting/issues/51) we can see that we actually went for a single-interpreter. The only downside it has is that, AFAIU, the number of threads is 1 per process, so no multi threading is used. Some benchmarking would be needed, but I would be okay with that.

Sadly the uWSGI docs give 0-to-no information about the flag/feature. Many other places (e.g. bloomberg tech above, newrelic) mention that many C extensions do not work well with multi-threading. Therefore:

WDYT? @lnielsen @zzacharo @fenekku (Also @slint @ntarocco cuz it might affect future migrations to RDM)

fenekku commented 4 years ago

We had to disable it (use single-interpreter=true) in our instance: https://github.com/galterlibrary/InvenioRDM-at-NU/blob/master/docker/uwsgi/uwsgi_ui.ini#L17 . So I am in favour of that. I wouldn't worry about not being able to use multi-threading now.