Need maintainers?

[ilkerkorkut]

I've noticed issues and PR's piling up. @hibiken do you need help maintaining this?

[kamikazechaser]

I've noticed issues and PR's piling up.

Yeah noticed this too :(

Some PRs look really important.

[hibiken]

Yes, it'd be great if I can get some help with reviewing PRs and managing releases. I don't have enough bandwidth to focus on this project recently.

[gedw99]

How do you want releases to work ?

[gedw99]

https://github.com/hibiken/asynq/blob/master/.github/workflows/build.yml

Want a release a room to build for all 3 OS and each ISA of each OS ?

Docs and change log too later ?

[gedw99]

Also the web yo could be moved into this repo to make CI etc easier.

mono repo is fine imho.

There is a way to make the Web UI 100% golang without any js too using htmx. It will make it much easier to maintain. There are many golang project doing this and even one project that imports your project doing it.

https://github.com/mikestefanello/pagoda/tree/main/pkg/htmx

https://github.com/mikestefanello/pagoda/blob/main/go.mod#L14

It means golang devs can maintain the backend and the frontend.

https://github.com/mikestefanello/pagoda/tree/main/templates

It’s very fast still . Updates are send via SSE or Web sockets To update the web gui.

[ioannidesalex]

Even though I don’t have much free time, I can also assist with maintnenance. It’s a great project and it has to keep going.

[ibukunoluwayomi]

Happy to assist with maintenance too. I've loved using Asynq. Would be great to give back.

[ibukunoluwayomi]

Also the web yo could be moved into this repo to make CI etc easier.

mono repo is fine imho.

There is a way to make the Web UI 100% golang without any js too using htmx. It will make it much easier to maintain. There are many golang project doing this and even one project that imports your project doing it.

https://github.com/mikestefanello/pagoda/tree/main/pkg/htmx

https://github.com/mikestefanello/pagoda/blob/main/go.mod#L14

It means golang devs can maintain the backend and the frontend.

https://github.com/mikestefanello/pagoda/tree/main/templates

It’s very fast still . Updates are send via SSE or Web sockets To update the web gui.

I'm a fan of a mono repo in this case too.

[gedw99]

Cool

@hibiken are you ok with mono repo and htmx ? Will make maintenance easier

[gedw99]

Also a docker that runs Redis, frontend and backend will make life easier .

Can run on fly.io machines for free and shows it all working.

Very easy to deploy as part of ci as a docker stored inside fly.io

then we can run all tagged versions easily with the docs so everyone can see old and latest

[vishrutkohli]

i can help too .

[sjiekak]

I've never contributed to this project but used celery in python, implemented own redis and rabbitmq distributed task queue for my job. I also see few quality insurance tools could help (linters like golangci, package manager like renovate, etc...).

[ilkerkorkut]

We may create a nice community on top this project. I don't have enough time to create discord channel etc. Can somebody be a volunteer via taking an action for coming together? May be we can consider about forking this project and contribute on on it. Any other ideas?

[ioannidesalex]

I can do a Discord server if we gather enough interest.

[acaloiaro]

I'm happy to help review PRs and issues.

But before anyone can do anything of substance on this repo I think contributors should seek @hibiken's blessing to become official maintainers with repository permissions. Otherwise, the only option to contribute without Ken becoming a bottleneck is forking the repo and organizing a group of contributors around a new repository. That can be seen as a hostile act without the original author's blessing.

@hibiken When you have some time to think about this, it would be helpful to hear how you'd like the maintenance of this project to move forward

1. Do you want to continue with the repository under hibiken/asynq? Or would you like the mental distance of moving it to an open source org like asynq (do you control this organization? If not, maybe go-asynq, which is available)?
2. Are there any past contributors who you trust to be responsible stewards and have sufficient asynq familiarity to guide the project in a good direction?

I do have time to help the project get organized around a new group of contributors and set some priorities. But I don't have a lot of asynq experience, and I can only imagine there's someone better suited to stewarding this project from a trust and knowledge standpoint.

[ioannidesalex]

I completely agree that we should talk with @hibiken . No hostile acts at all. We shall do whatever is best for the project, following his feedback, recommendations and blessings.

[benjaminclauss]

looks like conversation on this issue has slowed down

I would be very interested in helping with maintaining this project.

[benjaminclauss]

Would collaborators on this thread be opposed to waiting the entirety of July 2023 for an update? If others have input as to a more appropriate duration, feel free to comment.

If no update is received, I would be interested in creating an organization to manage a fork.

[ilkerkorkut]

Would collaborators on this thread be opposed to waiting the entirety of July 2023 for an update? If others have input as to a more appropriate duration, feel free to comment.

If no update is received, I would be interested in creating an organization to manage a fork.

Actually, @hibiken didn't response to here about forking. I think most of contributors waiting for his considerations.

[benjaminclauss]

Any modifications to a fork could be pushed upstream, right?

[acaloiaro]

In the interest of advancing currently pending issues/PRs, I went ahead and forked this project into https://github.com/go-asynq/asynq

To be clear, my intention is not to become a permanent maintainer of this project, or for go-asynq/asynq to become an "official" fork of asynq, unless that is @hibiken's desire. I don't have enough familiarity with the project to be anything like a lead maintainer, and I also don't have the desire to be that.

My hope is that this will be temporary, and @hibiken can declare their desire for asynq's future when they have the time to do so. Ideally, with any changes that happen on go-asynq/asynq being merged back into hibiken/asynq at that time.

If anyone wants to help facilitate development on go-asynq/asynq, I'm happy to add official contributors there. Since there are no current users of go-asynq/asynq, there is no risk of a supply chain attacks through the fork. Open an issue if you're interested in maintaining: https://github.com/go-asynq/asynq/issues

As for communications, I don't see a reason for a Discord, as the project already has an official Gitter: https://gitter.im/go-asynq/community

If this fork gains momentum, I'm happy to do my best with PR/Issue review, and will set up a release system, per @hibiken's comment here: https://github.com/hibiken/asynq/issues/626#issuecomment-1473040642. That way, if and hopefully when the fork merges back into this repository, Ken will get at least one thing that they hoped for.

So this is more or less a call for maintainers who can do the work of (at least temporarily) maintaining a project. If that's you, head over to the fork and bring your pet issues/PRs: https://github.com/go-asynq/asynq

[hibiken]

Sorry that I haven't responded to this thread for a while.

@acaloiaro and others Thank you so much for being very respectful and expressing interests in maintaining this project.

I think maintaining a separate fork in conjunction with the original project may cause more headaches in the long-run (I may reject some features that were added to the forked project or vice-versa, then the two projects will diverge). so my recommendation is to maintain this repository (both for my selfish reason + maintainability reason).

I have become the bottle neck in recent months and I will try my best to start reviewing incoming issues/PRs at least once a week. We do have quite a lengthy backlog of issues/PRs so I'll also try to go through them at much slower pace (so if you think there's an urgent/important issue/PR, please add a comment).

That being said, I have other higher priority responsibilities -- being an employee, a dad, a husband, etc, and I'll have limited bandwidth to work on this project Hopefully in the future, I can find a few maintainers, so we can increase the bus-factor for the project.

Lastly, if you are using this package for business, please consider sponsoring -- it really helps to justify my time spent on this project.

Thanks!

[benjaminclauss]

sounds like this issue can be closed then?

I would be interested but will discuss in Gitter.

[kamikazechaser]

Agree with @hibiken. Aside from outdated dependencies, the project is pretty stable and has enough features to cover most worker queue requirements.

That being said, I am open to reviewing/merging any dependency upgrades/security fixes/critical bug fixes.

[acaloiaro]

Awesome, thank you @hibiken! I'll go ahead and shut that org/fork down.

[benjaminclauss]

I would be very interested in supporting backends other than Redis (i.e. Mongo).

https://github.com/hibiken/asynq/issues/173

[gedw99]

hey @hibiken

From looking at the many issues relating to supporting other brokers than just Redis, I think https://github.com/streamdal/plumber might be a good abstraction layer.

https://github.com/streamdal/plumber?tab=readme-ov-file#supported-messaging-systems

Its all pure golang and also has a nice CLI. Its uses a basis for a high level pipelining system.

Supported Messaging Systems

Kafka RabbitMQ RabbitMQ Streams Google Cloud Platform PubSub MQTT Amazon Kinesis Streams Amazon SQS Amazon SNS (Publishing) ActiveMQ (STOMP protocol) Azure Service Bus Azure Event Hub NATS NATS Streaming (Jetstream) Redis-PubSub Redis-Streams Postgres CDC (Change Data Capture) MongoDB CDC (Change Data Capture) Apache Pulsar NSQ KubeMQ Memphis - NEW!