As a Thoth developer, I'm frustrated when I need to delete data and "manually" sync between S3 and SQL.
High-level Goals
We should reference Object Storage (S3) inside SQL (alchemy models) and not interact directly with it.
Proposal description
Use something like sqlalchemy-file (#2691 )
Alternatives
N/A
Additional context
Born from frustation trying to work on #2657
We use S3 as some kind of FS storage (using object names to distinguish classes of objects), see all child Classes of ResultStorageBase) which IMO is an antipattern.
Problem statement
As a Thoth developer, I'm frustrated when I need to delete data and "manually" sync between S3 and SQL.
High-level Goals
We should reference Object Storage (S3) inside SQL (alchemy models) and not interact directly with it.
Proposal description
Use something like sqlalchemy-file (#2691 )
Alternatives
N/A
Additional context
Born from frustation trying to work on #2657
We use S3 as some kind of FS storage (using object names to distinguish classes of objects), see all child Classes of ResultStorageBase) which IMO is an antipattern.
Acceptance Criteria
TBD