hunter-meloche / infra-ebay-scraper

Terraform AWS IaC for ebay-scraper
MIT License
0 stars 0 forks source link

Implement low-cost db #7

Open hunter-meloche opened 1 year ago

hunter-meloche commented 1 year ago

It turns out that using RDS is expensive and overkill. A smarter solution would be serverless like the rest of the infra. I think i can put an sqlite db file in an EFS store to accomplish this. Why not s3? You would have to download the db file, modify it, and write it back into the bucket. This breaks concurrency. Sqlite is inherently anti-concurrent, but on EFS (or any other regular filesystem), a polite queue can be formed to write to the sqlite db.

hunter-meloche commented 1 year ago

It's unlikely that the DB size would exceed 1GB quickly, so it's reasonable to expect DB costs to be roughly $0.16 a month with this solution. I'm relegating it to the one-zone version because this is only for educational purposes. Ironically, this makes it ineligible for the 12 month free tier trial offering that only allows for the two-zone version. My account is way older than a year, and I feel like most other's will be too, so I'm going with one-zone.