matrix-org / synapse-s3-storage-provider

Synapse storage provider to fetch and store media in Amazon S3
Apache License 2.0
118 stars 33 forks source link

Unable to run with Synapse 1.96.1 #105

Closed MomentQYC closed 9 months ago

MomentQYC commented 9 months ago

Error at startup:

Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/app/homeserver.py", line 399, in <module>
    main()
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/app/homeserver.py", line 389, in main
    hs = setup(sys.argv[1:])
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/app/homeserver.py", line 298, in setup
    config = HomeServerConfig.load_or_generate_config(
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/config/_base.py", line 802, in load_or_generate_config
    obj.parse_config_dict(
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/config/_base.py", line 823, in parse_config_dict
    self.invoke_all(
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/config/_base.py", line 422, in invoke_all
    res[config_class.section] = getattr(config, func_name)(*args, **kwargs)
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/config/repository.py", line 193, in read_config
    provider_class, parsed_config = load_module(
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/synapse/util/module_loader.py", line 47, in load_module
    module = importlib.import_module(module_name)
  File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/s3_storage_provider.py", line 23, in <module>
    import boto3
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/boto3/__init__.py", line 17, in <module>
    from boto3.session import Session
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
    import botocore.session
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
    import botocore.client
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
    from botocore import waiter, xform_name
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
    from botocore.docs.docstring import WaiterDocstring
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/docs/__init__.py", line 15, in <module>
    from botocore.docs.service import ServiceDocumenter
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
    from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/docs/client.py", line 14, in <module>
    from botocore.docs.example import ResponseExampleDocumenter
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
    from botocore.docs.shape import ShapeDocumenter
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
    from botocore.utils import is_json_value_header
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/utils.py", line 37, in <module>
    import botocore.httpsession
  File "/opt/venvs/matrix-synapse/lib/python3.10/site-packages/botocore/httpsession.py", line 22, in <module>
    from urllib3.util.ssl_ import (
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/opt/venvs/matrix-synapse/lib/python3.10/site-packages/urllib3/util/ssl_.py)

This synapse version uses urllib3 version 2.0.7, and as of this version, DEFAULT_CIPHERS has been deprecated.

clokep commented 9 months ago

How do you have Synapse installed? Can you double check the version of botocore / boto3 in use? (pip freeze | grep -i boto)

MomentQYC commented 9 months ago

How do you have Synapse installed? Can you double check the version of botocore / boto3 in use? (pip freeze | grep -i boto)

Installation using apt.

boto3==1.26.50
botocore==1.29.50

synapse-s3-storage-provider is installed via pip within the venv

clokep commented 9 months ago

We seem to be using:

boto3==1.28.75
botocore==1.31.75

Maybe upgrading those will fix the issue? We might need to up the minimums on our pinned versions.

MomentQYC commented 9 months ago

Maybe upgrading those will fix the issue?

Upgraded to the specified version and it's running fine.

clokep commented 9 months ago

Looks like you need a version that has https://github.com/boto/botocore/pull/2922 at least? Maybe?

The changelog includes the following for 1.31.62: "enhancement:Dependencies: Add support for urllib3 2.0 for Python 3.10+"

I guess we should make the minimum version 1.31.62 then.

clokep commented 9 months ago

I've published a 1.3.0 build with this fix.