Add new backends to seqrepo

[korikuzma]

Submitter Name

Reece Hart (@reece)

Submitter Affiliation

MyOme

Requested By

seqrepo community

Lead(s)

@reece

biocommons Repo

seqrepo

Project Details

Hackathon Project Slide

SeqRepo provides a non-redundant, compressed, journalled, file-based storage for biological sequences. SeqRepo also provides a simple interface to this storage. File-based storage is not convenient or efficient for many users. This project will generalize the interface to support other types of backends.

This project has several goals:

• Formalize an abstract interface to sequences and sequence slicing as described here and implement it
• Investigate and implement additional backends, such as redis or s3, which might be preferable for public access
• Consider how to create a fingerprint of an instance of seqrepo data, such as digesting all sequences and aliases to create a fingerprint for the collection

[korikuzma]

@reece plans on making an abstract interface before the hackathon. If this happens, then this project will work on implementing caching (Redis/Aurora) and federation (something else)

[reece]

https://github.com/biocommons/biocommons.seqrepo/issues/61#issuecomment-1663176331 for the intended plan for generalizing the interface and implementing backends.

[korikuzma]

@ahwagner strongly agrees don't do the advance work and have this be part of the hackathon