boto / botocore

The low-level, core functionality of boto3 and the AWS CLI.
Apache License 2.0
1.44k stars 1.06k forks source link

Start testing on 3.13 beta build #3185

Open nateprewitt opened 1 month ago

nateprewitt commented 1 month ago

This PR will start provisional testing on the recently released beta for Python 3.13. This will be used to catch any issues early before the official release in October.

nateprewitt commented 1 month ago

It looks like all 3.13 builds are failing on these two tests due to increased memory usage with paginators:

FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_paginator_memory_constant
FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_waiter_memory_constant

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

Example Stacktrace Memory increases ~10.8MB which exceeds our arbitrary 10MB threshold. I'd like to see what our change looks like in 3.12 to determine if we barely crossed a threshold we were already approaching or if this is a significant increase.


self = <tests.functional.leak.test_resource_leaks.TestDoesNotLeakMemory testMethod=test_create_single_paginator_memory_constant>

    def test_create_single_paginator_memory_constant(self):
        self.cmd('create_paginator', 's3', 'list_objects')
        self.cmd('free_paginators')
        self.record_memory()
        for _ in range(100):
            self.cmd('create_paginator', 's3', 'list_objects')
            self.cmd('free_paginators')
        self.record_memory()
        start, end = self.memory_samples
>       self.assertTrue((end - start) < self.MAX_GROWTH_BYTES, (end - start))
E       AssertionError: False is not true : 10878976
jonathan343 commented 3 weeks ago

We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.

I had a few minutes to compare the delta values between 3.12 a 3.13. The results are shown below: Test Python 3.12 Memory Delta (B) Python 3.13 Memory Delta (B) % Increase
test_create_memory_clients_in_loop -9,052,160 176,128 -101.95 ??
test_create_memory_paginators_in_loop 1,503,232 1,937,408 28.88
test_create_memory_waiters_in_loop 1,675,264 1,794,048 7.09
test_create_single_client_memory_constant 1,785,856 10,338,304 478.90
test_create_single_paginator_memory_constant 2,551,808 11,886,592 365.81
test_create_single_waiter_memory_constant 2,490,368 11,870,208 376.64

There is definitely weird something going on with the 3.13 interpreter. I don't have to time to do a deeper dive now, but wanted to add this information here for future reference.

Notes:

  1. What is going on with test_create_memory_clients_in_loop??