As a general comment, I'm not a fan of having large if: blocks since they make it easier to miss bugs or inconsistent behaviour. The approach I've had in mind is basically to create a storage class which would support file enumeration, creation, etc. with a default local filesystem implementation and an optional S3 implementation so we could unit-test just that class and have a single code path through the main file.
This would also work nicely with optional dependencies: pip install bagit would get the traditional bagit.py but pip install bagit[s3] would pull in boto3 as well.
As a general comment, I'm not a fan of having large
if:blocks since they make it easier to miss bugs or inconsistent behaviour. The approach I've had in mind is basically to create a storage class which would support file enumeration, creation, etc. with a default local filesystem implementation and an optional S3 implementation so we could unit-test just that class and have a single code path through the main file.This would also work nicely with optional dependencies:
pip install bagitwould get the traditional bagit.py butpip install bagit[s3]would pull in boto3 as well.