infogulch / xtemplate

A html/template-based hypertext preprocessor and rapid application development web server written in Go.
Apache License 2.0
73 stars 1 forks source link

Feature request: Handle file upload from forms #8

Open taophp opened 5 months ago

taophp commented 5 months ago

May I missed something somewhere, but I can't figure how to handle file upload with xtemplate. If I'm wrong, tell me. If I'm right, well, we might figure out how to implement it. It seems a requirement for a modern web app framework.

infogulch commented 5 months ago

Yes I definitely want to have file upload support.

Here's an idea for a design:

Add another method onto the DotFS like

{{$dirName := .FS.ReceiveFiles "./dir" $maxfiles $maxbytes}}
{{range .FS.ListFiles $dirName}}
    ...

It would create a new directory under dir with a random name and save all multipart form files there as long as they are within the limits. This would guarantee that each request gets its own dir and can't clash with an existing directory.

Here are a few references:

I may work on this after the dot provider system is ready.

taophp commented 5 months ago

It seems a very good solution !

taophp commented 5 months ago

@infogulch I don't if that can be helpful, I've just discovered this Caddy module: https://github.com/git001/caddyv2-upload

gedw99 commented 5 months ago

We can use nats to also distribute files using nats object store.

It will mean that files will multi master sync. Exactly the same as we do with Marmot.

taophp commented 5 months ago

@gedw99 What uploading have to do with distributing ?

gedw99 commented 4 months ago

@gedw99 What uploading have to do with distributing ?

We already now have NATS KV and so adding NATS Object Store is a very easy extra to add.

It will make it easy to do file uploads into the NATS Object store. It's just another file store, like S3 but different. However because NATS is a distributed system, it wil mean that you can deploy many servers ( in many data centres) and have automatic redundancy and recovery.

https://docs.nats.io/using-nats/developer/develop_jetstream/object explains the NATS> object store.

https://docs.nats.io/nats-concepts/service_infrastructure/adaptive_edge_deployment explains the scale out aspects. Its just a config file.

It's easy to setup a worker, such that when a NATS Object store changes the worker is told and can then put the File anywhere it wants. S3, Minio, local file, etc.

NATS Object store is fault tolerant in that chunked uploads can be on the client and the server, so that if the connection fails it will pick up where it left off.

For a Web client, the way it would work is that you can do a traditional HTTP File Post to the Server and into the local NATS Object Store. Then depending on how ou configured NATS, that file will be synced to other servers etc.

The other way with a Web Client is to use the browser client: https://github.com/nats-io/nats.ws and upload straight into NATS Object store.


The other way is to use htmx and any standard file upload technique, and then out it into NATS Object store so that you get scale ut and redundancy. then on each server as the NATS Object is creates, it will put it to Disk using a worker. SO you have a CDN of sorts now.