Why do we store the issue data in s3

[RobStallion]

Why do we store our issue data as JSON in S3?

What are the pros/cons of storing this kind of data in S3 over our own sql database? e.g.

• is it faster to load data
• is it cheaper
• does it take up less space as a json file on S3 than as a string in postgres. If so could storing the data as a binary help? (see quotes below)

quotes from https://github.com/dwyl/phoenix-ecto-encryption-example

The reason we are creating the encrypted/hashed fields as :binary is that the data stored in them will be encrypted and :binary is the most efficient Ecto/SQL data type for storing encrypted data; storing it as a String would take up more bytes for the same data. i.e. wasteful without any benefit to security or performance.

The best data type for storing encrypted data is :binary (it uses half the "space" of a :string for the same ciphertext)

[nelsonic]

@RobStallion The original reasoning for using S3 for storage was because it's (much) cheaper than Heroku's PostgreSQL, but I feel that S3 is not "fit for purpose" if we want the ability to search through issues.

Please assume that we are going to use PostgreSQL instead of S3 going forward. Because we are going to be using the data extracted by github-backup to power https://github.com/dwyl/tudo ... effectively tudo will be the "front-end" for "gitbu".

Thanks. 👍