search

Howdju / howdju

Monorepo for the Howdju crowdsourced fact checking and summarization platform
https://www.howdju.com
GNU Affero General Public License v3.0
5 stars 2 forks source link

Starting Feb 1 AWS started charging for attached public IPv4 addresses #657

Open carlgieringer opened 6 months ago

carlgieringer commented 6 months ago

Details are here.

Options:

⛔️ (bad) ✅ (good) ⚠️ (tossup)

Do nothing

Consideration Analysis
AWS bill ⛔️ The elastic IP costs $20/month.
Effort ✅ do nothing
Risk ✅ no changes mean no risks
Scraping accessibility ✅ We have our own IPv4 address

Move URL fetching to a public lambda

Consideration Analysis
AWS bill ✅ lambdas not in VPCs access the internet for free
Effort ⛔️ migrating the scraping to a new lambda and updating our message handler to call it would take a few days
Risk ✅ we can test the changes and easily revert
Scraping accessibility ⛔️ our lambda will share IPs with all other AWS customers in that region, making it more likely to be blocked

An interesting side benefit of this approach is that we could more easily expose the functionality to other AWS accounts.

Migrate our subnets to IPv6

Consideration Analysis
AWS bill ✅ IPv6 networks access the public internet from a VPC for free
Effort ⛔️ migrating existing infra to IPv6 is complicated and time-consuming
Risk ⛔️ there could be downtime during the changes if we mess up the network configuration
Scraping accessibility ⛔️ our lambda will be IPv6 only, preventing us from reaching IPv4 sites
carlgieringer commented 5 months ago

Based on the considerations above, I will leave the existing exclusive IPv4 approach in-place. I took other measures to reduce AWS costs.