Open nateprewitt opened 1 month ago
It looks like all 3.13 builds are failing on these two tests due to increased memory usage with paginators:
FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_paginator_memory_constant
FAILED functional/leak/test_resource_leaks.py::TestDoesNotLeakMemory::test_create_single_waiter_memory_constant
We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.
Example Stacktrace Memory increases ~10.8MB which exceeds our arbitrary 10MB threshold. I'd like to see what our change looks like in 3.12 to determine if we barely crossed a threshold we were already approaching or if this is a significant increase.
self = <tests.functional.leak.test_resource_leaks.TestDoesNotLeakMemory testMethod=test_create_single_paginator_memory_constant>
def test_create_single_paginator_memory_constant(self):
self.cmd('create_paginator', 's3', 'list_objects')
self.cmd('free_paginators')
self.record_memory()
for _ in range(100):
self.cmd('create_paginator', 's3', 'list_objects')
self.cmd('free_paginators')
self.record_memory()
start, end = self.memory_samples
> self.assertTrue((end - start) < self.MAX_GROWTH_BYTES, (end - start))
E AssertionError: False is not true : 10878976
We'll need to figure out what the delta is between 3.12 and 3.13, then see if we have an issue or there's some form of new memory leak happening in the 3.13 interpreter.
I had a few minutes to compare the delta values between 3.12 a 3.13. The results are shown below: | Test | Python 3.12 Memory Delta (B) | Python 3.13 Memory Delta (B) | % Increase |
---|---|---|---|---|
test_create_memory_clients_in_loop |
-9,052,160 | 176,128 | -101.95 ?? | |
test_create_memory_paginators_in_loop |
1,503,232 | 1,937,408 | 28.88 | |
test_create_memory_waiters_in_loop |
1,675,264 | 1,794,048 | 7.09 | |
test_create_single_client_memory_constant |
1,785,856 | 10,338,304 | 478.90 | |
test_create_single_paginator_memory_constant |
2,551,808 | 11,886,592 | 365.81 | |
test_create_single_waiter_memory_constant |
2,490,368 | 11,870,208 | 376.64 |
There is definitely weird something going on with the 3.13 interpreter. I don't have to time to do a deeper dive now, but wanted to add this information here for future reference.
Notes:
test_create_memory_clients_in_loop
??
This PR will start provisional testing on the recently released beta for Python 3.13. This will be used to catch any issues early before the official release in October.