search

hammad93 / hurricane-deploy

This creates email reports and hosts the REST API. The reports have tropical storm and hurricane forecasts from the hurricane-net model. The deployment repository for a hurricane forecasting system based on machine learning and deep learning methods.
MIT License
1 stars 0 forks source link

Remove Redis and transition back to SQL, notes on "data contingency" #21

Closed hammad93 closed 7 months ago

hammad93 commented 7 months ago

The Redis costs at one point was half of the total monthly operating costs. This wasn't a large amount, about $40/month on the managed instance for 1gb, but we encountered a cloud "data contingency" where either we pay up or never get access to the data.

Reference the original redis implementation here, https://github.com/hammad93/hurricane-deploy/issues/12

hammad93 commented 7 months ago

The comments made public on page 9 of the PDF of the document, https://www.courthousenews.com/wp-content/uploads/2024/02/musk-v-altman-openai-complaint-sf.pdf reinforce the characterization.

On a related note, other cloud providers allow limited access to the data even if there is a billing cap.

hammad93 commented 7 months ago

I changed to wording such that if anyone ever reads this other than me it will be more collaborative

hammad93 commented 7 months ago

I changed to wording such that if anyone ever reads this other than me it will be more collaborative

I gave ChatGPT the PDF document to read over and it suddenly had less of a problem calling it data hostage than before. Before, it kept on talking about reading the terms and services and that I agreed to this but pulled back when reading those comments.

hammad93 commented 7 months ago

The only thing that's really on redis is the audio.

hammad93 commented 7 months ago

This is done because it was already using SQL and redis was a backup but just needed to be reconfigured.