kostaleonard / mlops

A framework for conducting MLOps.
MIT License
3 stars 0 forks source link

Mocked AWS fixture scoping does not appear to be working properly #57

Open kostaleonard opened 2 years ago

kostaleonard commented 2 years ago

The mocked AWS fixture scoping is currently set to session, which prevents any tests from reaching out to S3, even those not requesting the fixture. This is not currently an issue because all of our S3 tests are mocked, but in the future they may not be. The mocked AWS fixture should have function level scope, which would cause a new mocked bucket to be created for every test. However, this does not appear to work for some reason. Tests will pass when run individually, but tests in and between modules will fail with PublicationPathAlreadyExistsError. Despite our debugging using fs.ls() within tests to verify that the mocked buckets are empty, moto believes otherwise (perhaps because it relies on boto3, which does not flush changes to buckets immediately after every operation).

If we introduce tests that cannot or should not use a mocked S3 bucket, we will need to decrease the fixture's scope to function level and devise strategies to repair failing tests.