search

transfem-org / Sharkey

🌎 A Sharkish microblogging platform 🚀
https://joinsharkey.org/
73 stars 19 forks source link

merge: upstream #73

Closed Mar0xy closed 11 months ago

Mar0xy commented 11 months ago

This merges all latest commits as well as the timeline change over to redis cache as it now keeps track of global and users messages again

dakkar commented 11 months ago

"interesting" bits you probably already know, but I think need thought:

Mar0xy commented 11 months ago

https://github.com/misskey-dev/misskey/issues/12019 does it mean I can't scroll back as far as I want?

As tastytea said this can easily be fixed by upping the maxcache out of the box or in instance settings

https://github.com/misskey-dev/misskey/issues/12028 and related issues: do they happen because the timeline in Redis is not populated?

Exactly this happens due to no population in redis and should affect all timelines afaik. Pinned notes aren't affected by this though as they still get directly pulled via the DB.

dakkar commented 11 months ago

ok, so after these changes:

please tell me I'm wrong…

Mar0xy commented 11 months ago
  1. is correct though you could inflate the cache to an extreme amount but i dont know what would happen
  2. yes though i am confused due to the fact that the update note states "temporarily"
  3. don't know what is meant with that

I also want to point out I was basically forced to pull this update cause otherwise keeping up with upstream would have become impossible as I spent about 2 hours on this just due to the timeline change was initially missing

Mar0xy commented 11 months ago

And the recent changes also really brought good things with them as well like a better CW button

image

Mar0xy commented 11 months ago

Update on user notes just checked them it seems they appear after a while again on a users profile which explains the "temporarily" statement:

image

dakkar commented 11 months ago

I also want to point out I was basically forced to pull this update cause otherwise keeping up with upstream would have become impossible as I spent about 2 hours on this just due to the timeline change was initially missing

apologies, I really didn't mean to sound antagonistic in any way, or like I was demeaning your work! I very much appreciate the effort and care you're putting into Sharkey!

let me try to clarify my last question:

Mar0xy commented 11 months ago

The user would not see the note in their home timeline initially due to the change to redis cache only if they view the user themself then they would see it.

dakkar commented 11 months ago

ok, so it is as bad as I thought.

I'm all in favour of caching, but a cache without a back-filling mechanism is… not a cache, it's a lossy filter.

(and yes, I understand that it's not something "we" can practically fix…)

Mar0xy commented 11 months ago

"we" can practically fix

Well if someone could figure out how to write a script that imports all notes from the DB into the redis cache then it could be fixed but I am not clever enough to do this stuff

dakkar commented 11 months ago

impressive that api/channel/timeline has a "fallback to database" https://github.com/transfem-org/Sharkey/pull/73/files#diff-9de864eae560189a50ee3137979b3b9a32baed7da582458b552096b5b465e5dfR129 when the cache doesn't contain the notes that the request asks for, but api/notes/timeline (and the other timelines under api/notes/) doesn't ☹ great job, upstream!

also, possible warnings: since all the timelines of all users and channels get stored in Redis, I suspect that this new code will overall take a lot more RAM… like, if you have 100 inactive users, the current code does nearly nothing about them, but the new code will populate their timelines in Redis

Mar0xy commented 11 months ago

if you have 100 inactive users

This will also depend on if they even follow anyone in the first place as most inactive users are accounts that got created and never used or people that really just reacted atleast once to a post.

dakkar commented 11 months ago

fair!