[22.03] Python3-boto3 package incompatible with python3-light

[ncul777]

Describe the bug

I have installed python3-light (version 3.10.9) and python3-boto3 (version 1.9.135), however running a sample code i get the following error:

  File "/root/awstest.py", line 1, in <module>
    import boto3
  File "/usr/lib/python3.10/site-packages/boto3/__init__.py", line 16, in <module>
  File "/usr/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/session.py", line 27, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/configloader.py", line 19, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/compat.py", line 25, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/exceptions.py", line 15, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/__init__.py", line 58, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/utils.py", line 26, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/compat.py", line 7, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/__init__.py", line 3, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/__init__.py", line 10, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 38, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 9, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py", line 1, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.pyc)

I found the following issue which indicates the incompatibility and that we need an updated version of python3-boto3 (1.10) for OpenWRT See https://github.com/boto/boto3/issues/3766

OpenWrt version

r20123-38ccc47687

OpenWrt release

22.03.4

OpenWrt target/subtarget

ath79/nand

Device

GL.iNet GL-E750

Image kind

Official downloaded image

Steps to reproduce

Run the following script (change the region and source/recipient emails accordingly)

import boto3
from botocore.exceptions import ClientError

def send_email(subject, body, recipient):
    # Create a new SES client
    ses_client = boto3.client('ses', region_name='us-east-1')

    # Try to send the email
    try:
        response = ses_client.send_email(
            Destination={
                'ToAddresses': [
                    recipient,
                ],
            },
            Message={
                'Body': {
                    'Text': {
                        'Charset': 'UTF-8',
                        'Data': body,
                    },
                },
                'Subject': {
                    'Charset': 'UTF-8',
                    'Data': subject,
                },
            },
            Source='recipient@test.com',  # Sender email address
        )
    except ClientError as e:
        print("Failed to send email:", e.response['Error']['Message'])
    else:
        print("Email sent! Message ID:", response['MessageId'])

# Example usage
send_email(
    subject="Test Email",
    body="This is a test email sent via AWS SES.",
    recipient="recipient@test.com"
)

Actual behaviour

No response

Expected behaviour

No response

Additional info

No response

Diffconfig

No response

Terms

• [X] I am reporting an issue for OpenWrt, not an unsupported fork.

[brada4]

23.05 contains 1.28

[ncul777]

Ah fair enough. Just the GL-iNet device (Mudi 750) still officially uses OpenWrt 22.03, so wondering is it possible to update the package on 22.03?

[brada4]

You can officially install official openwrt. https://firmware-selector.openwrt.org/?version=23.05.2&target=ath79%2Fnand&id=glinet_gl-e750 And stop reporting unrelated vendor fork package issues here.

[ncul777]

In fact I had tried to install OpenWRT 23.05 on the MUDI, which bricked it. So that doesn't help either. I will take it up with GL-iNet. I am relatively new to OpenWRT so finding my way where to report issues, so apologies!!

[brada4]

They send users post-warranty to use openwrt, blessing from support is nice to have.

[ncul777]

Python version is specified here: 3.10 https://github.com/openwrt/packages/blob/openwrt-22.03/lang/python/python3-version.mk

Python-boto3 version is 1.9.135 https://github.com/openwrt/packages/blob/openwrt-22.03/lang/python/python-boto3/Makefile

So in my opinion this is an incompatibility between packages on 22.03. The python packages was updated several times over the last few years, but last update for boto3 was April 2020.

There was some discussion about collections here on this PR but looks like it was never fully resolved https://github.com/openwrt/packages/pull/17121#issuecomment-974261167

Boto3 was updated in 23.05 8 months ago https://github.com/openwrt/packages/pull/21662

@jefferyto since you created the PR for 23.05, would it be much effort to update boto3 in 22.03 ? I'm more than willing to try to create a PR / test.

Really appreciate any guidance

[brada4]

It is pure python module, you can try installing pypi module with pip if you have space.

[brada4]

python3 -m pip install --user boto3 it will install in ~/.local Or if you have ton of space then you can use python virtualenv which is a copy of most of python files never shaken by system upgrades.

[BKPepe]

Reproduced on Turris Omnia, mvebu/cortexa9, running OpenWrt 22.03.6. Thanks for reporting.

root@turris:~# python3
Python 3.10.13 (main, Nov 21 2023, 00:45:41) [GCC 11.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/boto3/__init__.py", line 16, in <module>
  File "/usr/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/session.py", line 27, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/configloader.py", line 19, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/compat.py", line 25, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/exceptions.py", line 15, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/__init__.py", line 58, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/utils.py", line 26, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/compat.py", line 7, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/__init__.py", line 3, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/__init__.py", line 10, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/connectionpool.py", line 38, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/response.py", line 9, in <module>
  File "/usr/lib/python3.10/site-packages/botocore/vendored/requests/packages/urllib3/_collections.py", line 1, in <module>
ImportError: cannot import name 'Mapping' from 'collections' (/usr/lib/python3.10/collections/__init__.pyc)

I will look into it.

[ncul777]

@BKPepe just wondering if you got a chance to investigate any further? Thanks

[BKPepe]

https://github.com/openwrt/packages/commit/f185287f13b6e5894da214175f4b48539d8d859a this one should help. This is the latest version, which we do have in newer versions of OpenWrt.

[ncul777]

I can confirm that this works! Thanks very much!

For reference: add the following line to the end of etc/opkg/distfeed.conf (change the version and architecture as necessary) src/gz openwrt_packages https://downloads.openwrt.org/releases/22.03.6/packages/mips_24kc/packages