jonra1993 / fastapi-alembic-sqlmodel-async

This is a project template which uses FastAPI, Pydantic 2.0, Alembic and async SQLModel as ORM. It shows a complete async CRUD using authentication and role base access control.
MIT License
879 stars 143 forks source link

A short version for the image object in the schemas #44

Closed 8thgencore closed 11 months ago

8thgencore commented 1 year ago

Now we get the user like this

{
  "message": "Data got correctly",
  "meta": {},
  "data": {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "first_name": "string",
    "last_name": "string",
    .......
    "image": {
      "file_format": "string",
      "width": 0,
      "height": 0,
      "media": {
        "title": "string",
        "description": "string",
        "path": "string",
        "id": "string",
        "link": "string"
      }
    }
  }
}

But the image object takes up a lot of information.
This is especially clearly seen when we get a list of users and unnecessary fields are inserted.
it happens that need when only the url is needed from the image.

Example:

{
  "message": "Data got correctly",
  "meta": {},
  "data": {
    "id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "first_name": "string",
    "last_name": "string",
    .......
    "image_url": "string"
  }
}

It would be nice if there was an alternative scheme.

jonra1993 commented 1 year ago

Hello @8thgencore I agree it seems a good idea I am going to work on that the next days

8thgencore commented 1 year ago

I also noticed that if we get a list of users from the server (for example, 50 people). Then for each of them there is a request to s3 storage to form a link to the image. This greatly affects the speed of data acquisition. I am now also thinking how to speed up this process

jonra1993 commented 1 year ago

Hello, @8thgencore that is a good insight do you have a sample code to replicate it?

jonra1993 commented 11 months ago

Hello @8thgencore in this commit you can find how to combine pydantic models for what you are trying to do with image it uses root_validator https://github.com/jonra1993/fastapi-alembic-sqlmodel-async/commit/be0b363badd27976b0c6db147ebd4c24022d62b0

code here: https://github.com/jonra1993/fastapi-alembic-sqlmodel-async/blob/main/backend/app/app/schemas/image_media_schema.py

Image without media just link

image image
bazylhorsey commented 11 months ago

This still grabs a link for every single item, if you needed to grab 100 users, there would be 100 seperate requests to minio to grab fresh links. I was thinking perhaps this could be stored in redis for faster retrieval, and expiration between minio urls and redis could be synchronized to expire at the same time. Otherwise, is it possible to make some kind of schema validator that looks at all the file paths and asks minio for a link on all of them at once, which can then be mapped back across the responding schema?

jonra1993 commented 11 months ago

Hello @bazylhorsey if you want to get faster responses I see 2 options

  1. Using fastapi-cache could be an easier solution to avoid requiring more requests it will work just in get enpoints.
  2. Creating a dedicated cache system with Redis and minio as you suggest.