search

zoriya / Kyoo

A portable and vast media library solution.
https://kyoo.zoriya.dev
GNU General Public License v3.0
1.64k stars 38 forks source link

Add Diagrams #547

Closed acelinkio closed 3 months ago

acelinkio commented 3 months ago

Adds several diagrams:

acelinkio commented 3 months ago

Do have a couple of questions:

zoriya commented 3 months ago

Why does the frontend need to connect to backend? afaik frontend just serves static content

The frontend does SSR, if you request /movie/alien, the frontend will first fetch the backend and then give back an html page with all the content (you can see this if you disable the JS on your browser, most pages will work).

What does autosync do?

This is the service that will listen to watch status changes and update it on external website (for now only SIMKL)

What publishes/subscribes to each queue? Some of those queues might be internal to a system and others used to share between.

The scanner pushes to the scanner queue when new files are found in the library. This is read by the matcher. The scanner.rescan queue is read by the scanner (pushed via an API call to the backend) and triggers a whole library rescan.

Backend has a metadata volume. What exactly is inside of that?

This volume contains images (poster, thumbnails...). This gets downloaded from images URL saved in the database.

Transcoder has a metadata volume. What exactly is inside of that?

This contains:

acelinkio commented 3 months ago

Not really happy with the way Mermaid is rendering these diagrams. May look to moving away from their c4 https://mermaid.js.org/syntax/c4.html. Trying to avoid drawing this by hand.

Last commit included updates for:

zoriya commented 3 months ago

Just wrote my thought, keep in mind I had never seen a c4 diagram. Overall, I think those diagrams would make Kyoo more understandable for potential contributors. Thanks for taking the time to make those!

I feel arrows directions sometimes are wrong, mixing both consume/produce makes is hard to know how data flows.

On the scanner diagram:

For the transcoder, the API's only interaction with it is to proxy requests from the front (for example, when calling /api/movie/alien/info or /api/episode/got-s1e1/master.m3u8). The backend does not use anything else from the transcoder. The backend currently proxies requests to check JWT and permissions. I'd ideally want to move this logic elsewhere, but not sure how to proceed.

For the back diagram, there is no mention of postgres or meilisearch. Also, I don't know if mentioning all the others components (like scanner/front/autosync) makes sense. The backend is technically independent of those systems.

acelinkio commented 3 months ago

I feel arrows directions sometimes are wrong, mixing both consume/produce makes is hard to know how data flows.

fixed

scanner.resan is consumed by the scanner, not the matcher.

fixed

why are the scanner queue and the scanner.rescan queue not defined on the same ContainerBoundary? Not sure if this is an oversight or for another reason.

Middleware like messaging is hard to represent. Typically I would always assign ownership to the object producing the exchange/queue. kyoo_back creates the scanner.rescan ContainerQueue and is not a member of scanner. emb (EnterpriseMessageBus) CountainerBoundary is an entity used to represent those inter-container communication and also help with diagram formatting.

Anything intra-container communication is represented inside of the ContainerBoundary. scanner queue is internal to scanner ContainerBoundary.

For the back diagram, there is no mention of postgres or meilisearch. Also, I don't know if mentioning all the others components (like scanner/front/autosync) makes sense. The backend is technically independent of those systems.

postgres and meilisearch are the database icons in the diagram. Screencap to point out where. I'll try to see if I can customize that to make them pop more. image

The reason why scanner/front/autosync are mentioned is because their relationships. kyoo_front directly connects to kyoo_back. kyoo_back also produces messages that are consumed by kyoo_scanner and kyoo_autosync.

For the transcoder, the API's only interaction with it is to proxy requests from the front (for example, when calling /api/movie/alien/info or /api/episode/got-s1e1/master.m3u8). The backend does not use anything else from the transcoder. The backend currently proxies requests to check JWT and permissions. I'd ideally want to move this logic elsewhere, but not sure how to proceed.

Definitely down to talk that out. Diagram is meant to talk about the current state. There are a couple of things we can brainstorm about in the future: TranscoderMetadata, creating a kyoo_matcher image, and etc. Happy to do that through Discord, posts can be a little wordy.

acelinkio commented 3 months ago

Removing draft. Think the diagrams are in a good enough state to check in. I will definitely help maintain them in the future too.

acelinkio commented 3 months ago

Really happy this merge request is opening this dialog

Diagrams should be supplemental. More than happy to start working on Github pages docs with you. That would be a better place to describe each component in detail.

zoriya commented 3 months ago

Yes I agree with you but I don't want something too boilerplate-y. Having a doc too far from the code would either become outdated or not read.

What do you think about having a short document containing a one line description of each service and their relations to each other (diagrams) and a readme per service which will be code focused (with mermaid if necessary).

I started writing blogs for deep technical stuff about a service, I'll try to keep this going and linking them in the doc here!