search

Deadwood-ai / file-storage-api

FastAPI backend for the Deadwood-AI file storage server
GNU General Public License v3.0
0 stars 0 forks source link

Upload strategy #1

Closed mmaelicke closed 5 months ago

mmaelicke commented 5 months ago

Hey @JesJehle,

This is the most basic API function to manage large file uploads on the Storage Server for Deadwood-AI. Uploading works fine, even for GB-sized files. Question is now, how to handle meta-data upload. I see two ways here:

  1. The API requires the upload form to include a second file, which is the json containing the metadata. This is easy for the API, but as the users rarely have metadata files in JSON, it would mean to create the files in memory for the frontend. This is also possible, but not the most straightforward workflow.
  2. The API requires the metadata fields themselves to be present in the form. This is the way how forms are designed and meant to be used. It's easy on your side and easy on my side. Main downside is, that we need to change front- and backend, whenever we change the metadata (new fields etc.)

I personally would prefer option 2, specifically, because I like to build infrastructure that is designed along common patterns, but at the end of the day, you can make the decision, because it will have the larger impact on the frontend

JesJehle commented 5 months ago

I would suggest, The upload API takes the image together with a JSON object, the structure could be enforced or not. On the frontend, I build the JSON based on the form input. You make a simple python package as a wrapper for the storage API, which Clemens can use to programmatically upload images with corresponding metadata.

mmaelicke commented 5 months ago

You can checkout the save-metadata branch. It solves #1 and implemented the 'platform' of the image as metadata field. The correct type annotation renders a most-helpful documentation from the OpenAPI 3.0 docs, which you can find at http://localhost:8000/docs, then hit 'Try out' on the /upload path.

Query parameters works very similar. You can also upload any file through the docs and inspect the json, which will be created. Maybe things become a bit clearer then.

This would be my goto way, before we just send JSON objects from the frontend, which we can still decide to do, if fields turn out to be too much work on frontend implementation.