search

sveltejs / api.svelte.dev

The API worker source for https://api.svelte.dev
14 stars 4 forks source link

Use something other than Workers KV for todos storage #8

Closed Rich-Harris closed 2 years ago

Rich-Harris commented 3 years ago

Now that we have three demo apps deployed...

...it's clear that Workers KV isn't a viable solution for the Todos app. The CFW implementation works as expected, but the Vercel and Netlify ones both behave in a... somewhat absent-minded fashion. (Add/toggle/delete a few todos and refresh the page to see what I mean, or disable JS and try to interact with the app. It's chaos.)

Regardless of where the blame lies, Vercel and Netlify are massively popular and the demo app needs to work correctly with them. We can't expect people not to conclude that SvelteKit itself is the problem.

A few ideas for alternatives:

Durable Objects

Pros

Cons

Fauna

Pros

Cons

Supabase

Pros

Cons

I don't think cost should be a dealbreaker since we're sitting on unused opencollective funds. But it is a factor.

Thoughts on all this? Suggestions for alternatives? Do the issues we're encountering also apply to REPL apps (i.e. should we move storage generally away from KV?) or do we think we're in the clear? (The benchmark being 'a user should be able to save a REPL and refresh the page, reliably, without seeing stale data.)

armchair-traveller commented 3 years ago

Just going to add that Fauna has a GraphQL layer and for basic CRUD/MVPs it's basically all you need. You wouldn't have to touch its query language at all in that case. Don't quote me on this but I believe something that works in Workers KV should have no problems being adapted to its GraphQL API. Only when you need more powerful features (temporal, compound indexing, auth/ABAC, query optimizations, etc.) would you need to write FQL and possibly hook up to a resolver.

In fact, I recall a newsletter article a while ago stated more than 60% of Fauna users were using GraphQL in some capacity.

That said I did eventually spend enough time for FQL to click and because I haven't learned any other database query language syntax, I didn't have the confidence to outright call it fugly but it did feel awful writing let statements as a function with object entries and nesting functions deeply being a common pattern unless you create UDFs... It's manageable for me, but it's interesting to see comments on it.

The pricing con remains. I feel that Cloudflare generally gives a pricing advantage in most of their offerings, which is what I really like about them.

swyxio commented 3 years ago

i'd also try to check with @Kiwicopple to see if they can waive the supabase charges just for these demos since it might be in their interest...

kiwicopple commented 3 years ago

Hey @Rich-Harris we can certainly waive the fees - feel free to reach out on twitter (@kiwicopple) or email (copple at supabase)

minimum $300/yr

We also have a free tier and a very generous $25/m tier. I'm curious how you calculated this?

JS client is a bit of a mess, easier to use HTTP

😅. Yes you're welcome to use the REST endpoints or direct connection (use the connection pooler if it's serverless[1]). also happy to discuss JS improvements - we're always striving for the ultimate DX

latency for users who aren't near the instance

We'll solve this over time with edge replicas.

Final note - we have a lot of devs starting to use Supabase with sveltekit, so regardless of your decision here we'll build a Supabase+Svelte example so we'd love to work with you one this to make it awesome.

e: thanks for the mention @sw-yx [1]: you only need the pooler if you directly connect to the database from a serverless function. if you use our HTTP APIs then we handle everything (recommended)

GrygrFlzr commented 3 years ago

We also have a free tier and a very generous $25/m tier. I'm curious how you calculated this?

The todo demo is what's mentioned here, but the current REPL data usage on an existing database is already near the limits of the free 500MB tier listed on the site. $25/month * 12 months = $300/yr, and since this would be the long term storage for the REPL, future space usage needs to be taken into account.

Another consideration is that at the moment our todo API's use of Workers KV also makes use of its TTL feature to delete old todos after 30 days. We'd have to do that application-side if we move to either Fauna or Supabase. DO has the same lack of TTL though Cloudflare might add it in the future.

silentworks commented 3 years ago

Another consideration is that at the moment our todo API's use of Workers KV also makes use of its TTL feature to delete old todos after 30 days. We'd have to do that application-side if we move to either Fauna or Supabase.

This can be done on Supabase itself without doing it application-side. You can setup a cron job inside of Supabase itself and also their upcoming workflow feature should make this even easier when it’s released to the public.

leerob commented 3 years ago

Let me know if you need anything from the Vercel side 🙏🏼

kiwicopple commented 3 years ago

REPL data usage on an existing database is already near the limits of the free 500MB

Got it - we're happy to waive this cost, just let us know the database/size requirements.

delete old todos after 30 days

As @silentworks mentions, you can do this with pg-cron today, or you can use our upcoming workflow engine. I can help set up the cron

CaptainCodeman commented 3 years ago

Google Firestore or Datastore offer a generous daily free quota - I've run some fairly busy apps on them for an absolute pittance, often completely free. A cron task on AppEngine could also easily run for free to cleanup entities over a certain age.

https://firebase.google.com/docs/firestore/quotas#free-quota

Rich-Harris commented 2 years ago

As of now, todos are stored in a Supabase database (along with everything in https://svelte.dev/apps), and the demo is no longer glitching