Closed phiresky closed 1 year ago
I've not rebased this on master yet because then it would depend on the time zone changes to be merged into lemmy.
The TZ stuff is merged now.
Clippy is failing in a few places.
Makes sense. I've removed the send_activity function, adjusted the docs and fixed clippy.
I'm not 100% happy with the structure of SendActivityTask::prepare, but it's hard to improve without making openssl pkey part of the public interface and kinda circumventing the Actor { private_key_pem() } trait.
The integrated in-memory activity queue does not scale well since it sends many activities in parallel to the same server, and sequentially to multiple servers. There's also no backpressure so memory keeps building up. It's also not reliable and gets destroyed on process restart. It's been causing lots of issues for lemmy, even though it's been tweaked to be improved a fair bit by e.g @cetra3 .
This PR adds a method to send activities directly, without any queueing or retrying logic. The error handling is kept the same.
It also removes the activity queue. The intention is to replace the queue (in lemmy) with this: https://github.com/LemmyNet/lemmy/pull/3605 . Parts of that code could probably be extracted to not be lemmy- and postgresql specific, but that's out of scope for now.
For applications that don't need reliability or large scale, you can still use
send_activity()
(without retries or parallelization) or implement your own parallel replacement by using a simple retry loop and a tokio JoinSet.Since the new queue always calls the
SendActivityTask::prepare
function with only a single inbox, this PR also adds caching to the private key instantiation - otherwise it would be instantiated once per target inbox. I added a config option for the size of this cache. It would be possible to move the cache outside this crate but then the openssl crate would have to become part of the public interface of this crate.