search

acaloiaro / neoq

Queue-agnostic background job library for Go, with a pleasant API and powerful features.
MIT License
270 stars 4 forks source link

Replace gocraft/work with neoq #70

Closed Vad1mo closed 1 year ago

Vad1mo commented 1 year ago

Hello, for our project Harbor Jobservice I am considering replacing it with neoq. But before jumping into days of code refactoring, I wanted to see if you have any experience of compatibility of those two.

acaloiaro commented 1 year ago

I'm somewhat familiar with gocraft/work. I've used it for small projects in the past, but I'm by no means an expert.

I'd start by

  1. Acknowledging that neoq is still pre-1.0. While I enjoy using it, neoq's pre-1.0 version is intended to signal some API-breaking changes may still occur in the future.
  2. Take stock of gocraft/work's feature set and determine which features you're using, and see how those overlap with https://github.com/acaloiaro/neoq#what-it-does
  3. Finally, if there's sufficient feature parity, don't go all-in on neoq. Pick a single job type to move off gocraft/work, implement it with neoq, and throw some work at it. I'd also put this new code path behind a feature flag or environment variable so you can easily roll back to gocraft work if neoq isn't meeting your expectations.
  4. If (3.) goes as planned, continue the migration at a comfortable pace, one job type/queue at a time. There's no reason not to use neoq and gocraft/work concurrently while you migrate.
elliotcourant commented 1 year ago

Also curious in replacing gocraft/work with this due to this primarily https://github.com/gocraft/work/issues/148

Is there anything that could be taken on by outside contributors to help get neoq closer to a 1.0 release?

acaloiaro commented 1 year ago

@elliotcourant @Vad1mo Up until now there have been two parts to my holding off on the 1.0 milestone.

API Instability

I wanted the freedom to evolve neoq's API in its early stages.

However, I think it's relatively safe to lock neoq's API in place at this point. I like the API and a few others I've talked to have said the same.

Until this point I've primarily held 1.0 off because I wanted the freedom to change the API. But because I'm pretty satisfied with it, that is much less of a reason today than it was 6 months ago.

So with that said, you have my guarantee that the API (i.e. neoq.Neoq) won't change between now and the 1.0 release.

More feedback / users / bug reports

The other component of my holding out on 1.0 is that I'd like more real world feedback for multiple backends. Primarily, memory and postgres, since those are neoq-first implementations. Whereas redis is simply asynq with Neoq's API in front of it -- so I'm less interested in redis feedback because asynq is good, and stable.

Since mentioning neoq in my blog post recently, neoq has gotten a lot more attention and usage. I expect that to turn into more bug reports, questions, and feedback. While I don't have any plans to add new major features before 1.0, I would be happy to accept some usability feedback and bug reports before then.

So I'd like to to target roughly a month from now so that feedback from new users can be gathered and acted upon.

How does that sound?

Is there anything that could be taken on by outside contributors to help get neoq closer to a 1.0 release?

I'm happy to accept any contributions. Top priority will be fixing any bugs. But I'd also be happy to accept improvements to test coverage, as increased coverage is likely to reveal some low hanging fruit in the bug department.

elliotcourant commented 1 year ago

This sounds excellent, I'm going to start trying to use it in my own projects and we'll see what comes of it. I'm happy to help fix any bugs I do find or in general help contribute upstream!

acaloiaro commented 1 year ago

I'm going to go ahead and close this issue since it's not about neoq functionality. Feel free to open a new issue if you have any more questions.