search

tim-hat-die-hand-an-der-maus / architecture

MIT License
0 stars 0 forks source link

Create an architecture diagram #8

Open BjoernPetersen opened 2 years ago

torbencarstens commented 2 years ago

Look at Mr. Fancypants over here wanting an architecure diagram

I agree though, it'll become too complicated at some point. some more microservices in.

BjoernPetersen commented 2 years ago

I'm confused already! (at times)

torbencarstens commented 2 years ago

Getting old is tough.

I think we should have more than one architecure diagram:

I've created a rough overview, I left some parts as a blackbox to provide a better overview.

We can expand on this after we've discussed the scope of the different architecture diagrams:

image

BjoernPetersen commented 2 years ago

Nice! I think that covers the most important parts.

torbencarstens commented 2 years ago

Since I've recently ran into this problem I'll give a quick rundown of the services, I'll leave out the bus-frontend since it's a self-contained site. This is the current state of affairs, planned features can added later.

api

Has all the information (connects to postgres)

preliminary-frontend

read-only, shows all unwatched movies

Connects to: api

telegram-bot

Current frontend which provides basic functionality for the user (add, delete, ...)

Connects to: api

plex-resolver

Provides an api endpoint which returns all movies (plex library type: Movies) from the configured plex servers as json

Currently unused

plex-requester

Other services can send a request for a movie to be added to the download queue

Backend: to be decided

In development

streaming-provider-resolver

Checks which streaming service (info by werstreamt.es)has which movue and returns this information from an api endpoint

Currently unused

ImdB-resolver

Provides an api which returns information from ImdB about the title requested

metadata-updater

Checks for outdated Metadata (e.g. 404 for thumbnails) and tells the api to request new Metadata from the ImdB-resolver

Connects to: ImdB-resolver, api

rotating-docx-file-handler

Best logging impl out there

torbencarstens commented 1 year ago

I'm gonna take another stab at actually creating a diagram, I'll try out a few different libraries/services for doing this.

First up, Ilograph: https://app.ilograph.com/@torben/tim/Dependency

definition ``` resources: - name: Users subtitle: Users of the system color: Gray style: plural icon: Networking/user.svg - name: Prod subtitle: Environment color: navy children: - name: preliminary-frontend subtitle: Frontend icon: Networking/laptop.svg - name: telegram-bot subtitle: Frontend icon: Networking/laptop.svg - name: streamingprovider-resolver subtitle: Service icon: Networking/server.svg - name: imdb-resolver subtitle: Service icon: Networking/server.svg - name: metadata-updater subtitle: Service icon: Networking/server.svg - name: plex-resolver subtitle: Service icon: Networking/server.svg - name: plex-request subtitle: Service icon: Networking/server.svg - name: imdb subtitle: External Service icon: Networking/browser-search.svg - name: werstreamt.es subtitle: External Service icon: Networking/browser-search.svg - name: api subtitle: Backend color: Firebrick icon: Networking/cloud-server.svg - name: postgres subtitle: Database color: Purple description: This is a description icon: Networking/database.svg perspectives: - name: Dependency relations: - from: Users to: preliminary-frontend label: Use - from: Users to: telegram-bot label: Use - from: preliminary-frontend to: api label: Read - from: telegram-bot to: api label: Read/Write - from: telegram-bot to: streamingprovider-resolver label: Query - from: telegram-bot to: imdb-resolver label: Query - from: metadata-updater to: imdb label: Read - from: streamingprovider-resolver to: werstreamt.es label: Read - from: metadata-updater to: imdb-resolver label: Query - from: metadata-updater to: api label: Write - from: plex-resolver to: plex-request label: Write - from: streamingprovider-resolver to: plex-request label: Read - from: telegram-bot to: plex-request label: Request - from: imdb-resolver to: imdb label: Query - from: api to: postgres label: Read/Write notes: ```
torbencarstens commented 1 year ago

d2

example from d2:

d2lang code 
users -> preliminary-frontend
users -> telegram-bot

preliminary-frontend -> api: Read
telegram-bot -> api: Read/Write
telegram-bot -> imdb-resolver: Read
metadata-updater -> imdb: Read
streamingprovider-resolver -> werstreamtes: Read
metadata-updater -> imdb-resolver: Read
metadata-updater -> imdb: Read
metadata-updater -> api: Read/Write
plex-resolver -> plex-request: Write
streamingprovider-resolver -> plex-request: Read
telegram-bot -> plex-request: Write
imdb-resolver -> imdb: Read
api -> postgres: Read/Write
BjoernPetersen commented 1 year ago

Doesn't the API make requests to the imdb resolver?

Edit: yes it does. The metadata-updater just tells the API to refresh certain fields of a movie. I think the idea was somehow that the API needs to talk to the imdb-resolver anyway for new movies, so we didn't want to duplicate the imdb-resolver client code in another service.

BjoernPetersen commented 1 year ago

I'm impressed that no other service has snuck in direct access to the database yet. It's definitely a beautiful mess that we can only make worse.

torbencarstens commented 1 year ago

Doesn't the API make requests to the imdb resolver?

Edit: yes it does. The metadata-updater just tells the API to refresh certain fields of a movie. I think the idea was somehow that the API needs to talk to the imdb-resolver anyway for new movies, so we didn't want to duplicate the imdb-resolver client code in another service.

huh, didn't even know this was the case. I've added this as an additional dependency.

While ilographs pricing model is absolutely insane (especially for our use-case), it still has a very nice set of features; specifically being able to view different perspectives depending on service/parent resource/...

Which makes this part:

I think we should have more than one architecure diagram:

  • the group of stakeholders is probably more interested in a rough overview
  • a new frontend developer is mostly interested in the API connections
  • [...]

fairly trivial.

The export API (exporting standalone HTML) is even more ridiculous

Downside: We're dependent on their service continuation and can only update diagrams manually.

That being said, after the initial completion we'll proabably update this at most once a month (on average) and migrating to another language (such as d2) for the individual perspectives isn't that hard.

I've started this here: https://app.ilograph.com/@torben/kubernetes/

I'll play around a bit with the different concepts so this graph may change/misbehave at any time.

I'm impressed that no other service has snuck in direct access to the database yet. It's definitely a beautiful mess that we can only make worse.

Yeah, I'm amazed too. I think we only violate this (in all of our projects) in woog-life/graphing since the API doesn't expose every entry.

BjoernPetersen commented 1 year ago

huh, didn't even know this was the case. I've added this as an additional dependency.

You actually knew once: https://github.com/tim-hat-die-hand-an-der-maus/architecture/issues/8#issuecomment-1091945965

metadata-updater

Checks for outdated Metadata (e.g. 404 for thumbnails) and tells the api to request new Metadata from the ImdB-resolver

But I wouldn't have remembered that either, we're all getting old.

Downside: We're dependent on their service continuation and can only update diagrams manually.

I'm really hesitant to buy into a lock-in like that, but in the end, you're the one spending all that time on it now, so it's mostly your loss if we need to migrate at some point 😬.

torbencarstens commented 1 year ago

huh, didn't even know this was the case. I've added this as an additional dependency.

You actually knew once: #8 (comment)

Do you really expect me to r*ad?

I may share your opinion though.

metadata-updater

Checks for outdated Metadata (e.g. 404 for thumbnails) and tells the api to request new Metadata from the ImdB-resolver

But I wouldn't have remembered that either, we're all getting old.

I had to look at the code to check how it works since the last (code related) commit I've made was March 2022.

But there aren't any missing thumbnails (which was the usual problem) so I guess this works.

Downside: We're dependent on their service continuation and can only update diagrams manually.

I'm really hesitant to buy into a lock-in like that, but in the end, you're the one spending all that time on it now, so it's mostly your loss if we need to migrate at some point 😬.

Yeah, me too. I'll probably write a small tool which converts the ilograph structure to a basic d2-lang structure.

basically

from: x
to: y

simply converts to

x -> y

for each perspective.