matrix-org / synapse-s3-storage-provider

Synapse storage provider to fetch and store media in Amazon S3
Apache License 2.0
132 stars 36 forks source link

Fix a CI flake #94

Closed squahtx closed 1 year ago

squahtx commented 1 year ago

We need to consume the Failure in test_error, otherwise subsequent tests will fail if the garbage collector runs during them.


An example flake can be found at https://github.com/matrix-org/synapse-s3-storage-provider/actions/runs/4417512983/jobs/7743354660.

py: commands[0]> coverage run --include=s3_storage_provider.py -m twisted.trial test_s3
test_s3
  StreamingProducerTestCase
    test_error ...                                                         [OK]
    test_pause_produce ...                                                 [OK]
Error: t_simple_produce ...                                             [ERROR]

===============================================================================
Error: 
Traceback (most recent call last):
  File "/home/runner/work/synapse-s3-storage-provider/synapse-s3-storage-provider/s3_storage_provider.py", line 271, in _stream_to_producer
    chunk = body.read(READ_CHUNK_SIZE)
  File "/home/runner/work/synapse-s3-storage-provider/synapse-s3-storage-provider/test_s3.py", line 171, in read
    raise val
builtins.Exception: Test Exception

test_s3.StreamingProducerTestCase.test_simple_produce
-------------------------------------------------------------------------------
Ran 3 tests in 0.026s

FAILED (errors=1, successes=2)
py: exit 1 (3.[25](https://github.com/matrix-org/synapse-s3-storage-provider/actions/runs/4417512983/jobs/7743354660#step:5:26) seconds) /home/runner/work/synapse-s3-storage-provider/synapse-s3-storage-provider> coverage run --include=s3_storage_provider.py -m twisted.trial test_s3 pid=2051
.pkg: _exit> python /opt/hostedtoolcache/Python/3.10.10/x64/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
  py: FAIL code 1 (53.00=setup[49.75]+cmd[3.25] seconds)
  evaluation failed :( (53.[34](https://github.com/matrix-org/synapse-s3-storage-provider/actions/runs/4417512983/jobs/7743354660#step:5:35) seconds)

Inserting a gc.collect() at the start of t_simple_produce will reproduce the flake.

squahtx commented 1 year ago

Thanks for the fast review!