search

acidiney / bull-queue

AdonisJS package to work with bull queues
MIT License
4 stars 2 forks source link

Cannot remove/promote jobs or clean queues #8

Open royvanderzon opened 4 months ago

royvanderzon commented 4 months ago

Everything works great except actions made from the bull dashboard. Actions like "cleaning", "promoting" or opening "server status" don't work.

On the server there are no error logs. Also the dashboard itself (showing queues and jobs) is working.

Here are a few samples of responses I got from the Adonis server. Promote Screenshot 2024-05-21 at 07 37 59 Screenshot 2024-05-21 at 07 38 08

Stats Screenshot 2024-05-21 at 07 38 30

"TypeError: this.queue.client.info is not a function\n at BullAdapter.getRedisInfo (/Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/@bull-board/api/dist/src/queueAdapters/bull.js:12:34)\n at getStats (/Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/@bull-board/api/dist/src/handlers/redisStats.js:22:38)\n at redisStatsHandler (/Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/@bull-board/api/dist/src/handlers/redisStats.js:44:43)\n at Object.handler (/Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/@bull-board/h3/dist/H3Adapter.js:128:44)\n at Object.handler (/Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/h3/dist/index.cjs:2203:34)\n at Object.handler (file:///Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/h3/dist/index.mjs:1962:31)\n at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n at async Server.toNodeHandle (file:///Users/royzon/Documents/TranspaClean/Local/transpaclean-boilerplate/api/node_modules/h3/dist/index.mjs:2249:7)"

Clean Screenshot 2024-05-21 at 07 39 15

Current config:

export default defineConfig({
  connection: {
    host: env.get('REDIS_HOST'),
    port: env.get('REDIS_PORT'),
    password: env.get('REDIS_PASSWORD'),
    username: env.get('REDIS_USERNAME'),
    tls: {},
  },
  queue: {},
  worker: {},
  jobs: {
    attempts: 3,
    removeOnComplete: 100,
    removeOnFail: 100,
  },
  queueNames: ['default'],
  queuePrefix: '',
})

Some things I tried:

acidiney commented 4 months ago

Hi @royvanderzon,

Let me check it

acidiney commented 4 months ago

Hmm, looks like it have a bug on BullAdapter implementation, I tried switch to BullMQAdapter, and works fine

Ref: https://github.com/felixmosh/bull-board/issues/322#issuecomment-914033348

acidiney commented 4 months ago

I can't reproduce the "Job not found "

acidiney commented 4 months ago

@royvanderzon please update to 1.5.2, it should fix most of the problems, you're facing.

royvanderzon commented 4 months ago

@acidiney thank you very much, this indeed fixed most of the problems. The "Job not found" error still persists. The most important issues are resolved for me now becasue I can cancel "test crons" that where configured to start every minute.

I've connected a Redis without TLS but that also didn't fixed the issue. Just fore reference here's a example of the request that is sent from my browser:

curl 'http://localhost:9999/admin/api/queues/syncing_source/repeat%3A54a780d537b7c579e4ab0c619437d6ca%3A1716329410000/clean' \
  -X 'PUT' \
  -H 'Accept: application/json, text/plain, */*' \
  -H 'Accept-Language: en,nl;q=0.9,en-GB;q=0.8' \
  -H 'Cache-Control: no-cache' \
  -H 'Connection: keep-alive' \
  -H 'Content-Length: 0' \
  -H 'Cookie: fjo2uf389nr3u29e2y37=s%3AZKH6Ho-NzPaIsfdzkGb3Q4f32XoLqEej.HOkcnVRxq4z7R1I2O19spVInSnFTN8Gdyat2bOKH9ew; token=oat_MjQ.c3daWlV5RXFCc25faVBpZmY4ZFhiNFlTUUF2a25EYjJaeHBuM0NiQTUzNjU3NzAwNw; refresh_token=; set_googtrans=/nl; _csrf=9YIifdFLEoct0-2fyeuPRkcs; googtrans=/nl' \
  -H 'Origin: http://localhost:9999' \
  -H 'Pragma: no-cache' \
  -H 'Referer: http://localhost:9999/admin/queue/syncing_source?status=delayed' \
  -H 'Sec-Fetch-Dest: empty' \
  -H 'Sec-Fetch-Mode: cors' \
  -H 'Sec-Fetch-Site: same-origin' \
  -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36' \
  -H 'sec-ch-ua: "Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "macOS"'

Returning:

{
  "error": "Job not found"
}

The jobId in the url matches the jobId what I'm seeing in the bull dashboard (or what I think the id is).

encoded = http://localhost:9999/admin/api/queues/syncing_source/repeat%3A54a780d537b7c579e4ab0c619437d6ca%3A1716329410000/clean
decoded = http://localhost:9999/admin/api/queues/syncing_source/repeat:54a780d537b7c579e4ab0c619437d6ca:1716329410000/clean

I've another project that also is with bull but setup in a different way. It uses bull dashboard 2.1.3. The only differnce I see is that the id from my other (old version) bull has a number id in stead of a string that contains the word "repeat:".

Old bull:  /bull-queues/api/queues/Tendering_task_tender_sync_all/174485/clean
Adonis bull: /admin/api/queues/syncing_source/repeat%3A54a780d537b7c579e4ab0c619437d6ca%3A1716286210000/clean

Also another interesting thing I noticed is that in the newer Bull the "job" is also clickable (by clicking on the jobId). This is also not working. I tried various parts of the id repeat:54a780d537b7c579e4ab0c619437d6ca:1716329410000, but without success.

acidiney commented 4 months ago

@royvanderzon could you send me a repo with a reproducible example?

royvanderzon commented 4 months ago

@acidiney Repo

Env

TZ=UTC
PORT=3333
HOST=localhost
LOG_LEVEL=info
APP_KEY=QiHwKri3CuX4oKfwBLNRqiQSyMpQJkz_
NODE_ENV=development

REDIS_PORT=6379
REDIS_HOST=127.0.0.1
REDIS_USERNAME=
REDIS_PASSWORD=foobared
acidiney commented 4 months ago

Hey @royvanderzon, I've tried your repo, with same env settings and I still can't reproduce. I've attached a video, so, you can see what I've done.

https://github.com/acidiney/bull-queue/assets/24493328/7b805fa9-8c12-4e08-90c2-4cdb4e6460fe

royvanderzon commented 4 months ago

Hi @acidiney , so sorry, I tried to make it less complex for you to test by removing the cron from my example. But when I tested it, it also worked just the same way as in your video.

What it looks like is that the issue is related to a job that is generated from a cron. I'm very curious if the cron is working for you. I've updated the repo.

Opening a job that isn't related to a cron does work: http://localhost:9999/admin/queue/default/3?status=completed

Opening a job that is related to a cron doesn't work: http://localhost:9999/admin/queue/default/repeat:3d3793fb29fb2de31a3d22a4164ca90a:1716423010000?status=completed

acidiney commented 4 months ago

Hi @royvanderzon, I'm now able to reproduce the problem... I think it's related to the server (h3) I'm using... because the jobId in the URL isn't deserialised...

royvanderzon commented 4 months ago

Hi @acidiney , that makes total sense! Thank you for your help. For now (since your last version) I'm already helped a lot 👍🏻. Please let me know if I can do anything.