V3 not working with NestJS

[AgentSource]

I'm running into an issue where is appears the serverAdapter isn't handling req as expected — I just simply get a 404. I setup a basic example Stackblitz => https://stackblitz.com/edit/nestjs-starter-demo-gvfxkh

I was using v1.5.1 before - here is was my working main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { router } from 'bull-board';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.use('/admin/queues', router);
  await app.listen(3000);
}
bootstrap();

I migrated to v3.3.0 yesterday and have no very little luck with getting ExpressAdapter working correctly. Here is my current non-working main.ts

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { ExpressAdapter } from '@bull-board/express';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const serverAdapter = new ExpressAdapter();
  serverAdapter.setBasePath('/admin/queues');
  app.use('/admin/queues', serverAdapter.getRouter());  

  await app.listen(3000);
}
bootstrap();

Any help would be awesome!

[felixmosh]

You forgot to initiate the bullBoard it self :]

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { ExpressAdapter } from '@bull-board/express';
import { createBullBoard } from '@bull-board/api';
import { BullMQAdapter } from '@bull-board/api/bullMQAdapter';
import QueueMQ from 'bullmq';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const serverAdapter = new ExpressAdapter();
  serverAdapter.setBasePath('/admin/queues');

  const queueMQ = new QueueMQ('queueMQName')

  createBullBoard({
    queues: [
      new BullMQAdapter(queueMQ),
    ],
    serverAdapter
  });

  app.use('/admin/queues', serverAdapter.getRouter());  

  await app.listen(3000);
}
bootstrap();

[AgentSource]

@felixmosh thanks for the quick reply. I think I might be misunderstanding how to setup / init createBullBoard.

Currently I have 7 different queues — 3 are in a controller and the other 4 are in services...

My setup in v1.5.1 was

import { InjectQueue } from '@nestjs/bull';
import { Body, Controller, Get, Injectable, Logger, Param, Post, Put } from '@nestjs/common';
import { Queue } from 'bull';
import { setQueues, BullAdapter } from 'bull-board';

@Controller('base')
export class baseController {

    constructor(
        @InjectQueue('matches') public scoreQueue: Queue,
        @InjectQueue('opponents') public opponentQueue: Queue,
        @InjectQueue('progress') public progressQueue: Queue,
    ) {
        setQueues([
            new BullAdapter(this.scoreQueue),
            new BullAdapter(this.opponentQueue),
            new BullAdapter(this.progressQueue)
        ]);

}

Based on what is in the examples for v3.3.0 I just changed it to this

import { Body, Controller, Get, Injectable, Logger, Param, Post, Put } from '@nestjs/common';
import { Queue } from 'bull';
import { createBullBoard } from '@bull-board/api';
import { ExpressAdapter } from '@bull-board/express';
import { BullAdapter } from '@bull-board/api/bullAdapter';

@Controller('base')
export class baseController {
    public serverAdapter = new ExpressAdapter();

    constructor(
        @InjectQueue('matches') public scoreQueue: Queue,
        @InjectQueue('opponents') public opponentQueue: Queue,
        @InjectQueue('progress') public progressQueue: Queue,
    ) {
        createBullBoard({
            queues: [
                new BullAdapter(this.scoreQueue),
                new BullAdapter(this.opponentQueue),
                new BullAdapter(this.progressQueue),
            ],
            serverAdapter: this.serverAdapter
        });

}

Correct me if I'm wrong here but I want to createBullBoard in my main.ts then push my queues into that?

[felixmosh]

You are correct :]

[AgentSource]

Ok i'm completely stubbed on how to correctly access the queue...

There is setQueues on the ExpressAdapter but that doesn't appear to be what I want.

There is addQueue on createBullBoard but I have already init that but its in my main.ts which is boostrapped in...

I'm have to be missing something super simple here...

[felixmosh]

Can you access your queues in the main.ts file? If so, create an instance of serverAdapter, call createBullBoard with it & your queues, if they are not accessible there, createBullBoard returns addQueue method like previous versions did, share it somehow.

Hope it helps. If not, prepare a simple stackblitz, and I will try to help you.

[felixmosh]

Did you managed to solve this?

[AgentSource]

@felixmosh sorry busy weekend I'll kick that around today and let you know

[AgentSource]

@felixmosh finally got some time to flush this out, let me know what you think.

• Created a class to act as a pool for my queues

import { BullAdapter } from '@bull-board/api/bullAdapter';
import { BaseAdapter } from '@bull-board/api/dist/src/queueAdapters/base';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';

@Injectable()
export class BullBoardQueue { }

export const queuePool: Set<Queue> = new Set<Queue>();

export const getBullBoardQueues = (): BaseAdapter[] => {
    const bullBoardQueues = [...queuePool].reduce((acc: BaseAdapter[], val) => {
        acc.push(new BullAdapter(val))
        return acc
    }, []);

    return bullBoardQueues
}

• In my main.ts I get my queue pool and loop over it and call addQueue

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { ExpressAdapter } from '@bull-board/express';
import { createBullBoard } from '@bull-board/api';
import { getBullBoardQueues } from './bull-board-queue';
import { BaseAdapter } from '@bull-board/api/dist/src/queueAdapters/base';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const serverAdapter = new ExpressAdapter();
    const queues = getBullBoardQueues();

    serverAdapter.setBasePath('/admin/queues');
    app.use('/admin/queues', serverAdapter.getRouter());

    const { addQueue } = createBullBoard({
        queues: [],
        serverAdapter
    });

    queues.forEach((queue: BaseAdapter) => {
        addQueue(queue);
    });

  await app.listen(3000);
}
bootstrap();

I just import my class and add my queue pool => queuePool.add(updateQueue) — not perfect but gets the job done. Thanks for the help!

[felixmosh]

It looks good to me 👍

[asomethings]

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'queue1',
    }),
  ],
})
export class BullBoardModule implements NestModule {
  @Inject(getQueueToken('queue1'))
  private readonly queue: Queue

  configure(consumer: MiddlewareConsumer) {
    const serverAdapter = new ExpressAdapter()
    const { addQueue, removeQueue, setQueues, replaceQueues } = createBullBoard(
      { queues: [new BullAdapter(this.queue)], serverAdapter },
    )
    serverAdapter.setBasePath('/api/admin/queues')
    consumer.apply(serverAdapter.getRouter()).forRoutes('/admin/queues')
  }
}

This should work too.

[marcoacevey]

@Module({
  imports: [
    BullModule.registerQueue({
      name: 'queue1',
    }),
  ],
})
export class BullBoardModule implements NestModule {
  @Inject(getQueueToken('queue1'))
  private readonly queue: Queue

  configure(consumer: MiddlewareConsumer) {
    const serverAdapter = new ExpressAdapter()
    const { addQueue, removeQueue, setQueues, replaceQueues } = createBullBoard(
      { queues: [new BullAdapter(this.queue)], serverAdapter },
    )
    serverAdapter.setBasePath('/api/admin/queues')
    consumer.apply(serverAdapter.getRouter()).forRoutes('/admin/queues')
  }
}

This should work too.

You tried to add basic auth on this aprocah on nest js?

[mtdewulf]

I couldn't get the above examples to work with my setup, so I thought I'd paste what I got to work in case anyone else is in the same boat.

My versions:

$ egrep "nestjs|bull|bull-board" package.json 
    "@bull-board/api": "^3.7.0",
    "@bull-board/express": "^3.7.0",
    "@nestjs/bull": "^0.4.2",
    "@nestjs/common": "^8.1.2",
    "@nestjs/config": "^1.0.2",
    "@nestjs/core": "^8.1.2",
    "@nestjs/event-emitter": "^1.0.0",
    "@nestjs/jwt": "^8.0.0",
    "@nestjs/passport": "^8.0.1",
    "@nestjs/platform-express": "^8.1.2",
    "@nestjs/typeorm": "^8.0.2",
    "bull": "^3.3",
    "bullmq": "^1.51.1",
    "@nestjs/cli": "^8.1.4",
    "@nestjs/schematics": "^8.0.4",
    "@nestjs/testing": "^8.1.2",
    "@types/bull": "^3.15.1",

My Controller:

import { AuScope } from '@au/dtos';
import {
  Request,
  Response,
  All,
  Controller,
  Next,
  UseGuards,
  BadRequestException,
} from '@nestjs/common';
import express from 'express';
import { AuthGuard } from '../auth/auth.guard';
import { Scope } from '../auth/scope.decorator';
import { QueuesManagerService } from './queues-manager.service';

export const bullBoardPath = 'api/queues/admin';

@Controller(bullBoardPath)
@Scope(AuScope.QueuesAdmin)
@UseGuards(AuthGuard)
export class BullBoardController {
  constructor(private readonly service: QueuesManagerService) {}

  @All('*')
  admin(
    @Request() req: express.Request,
    @Response() res: express.Response,
    @Next() next: express.NextFunction,
  ) {
    const router = this.service.router;
    if (!router) {
      throw new BadRequestException('router not ready'); // Shouldn't happen.
    }

    const entryPointPath = '/' + bullBoardPath + '/';
    req.url = req.url.replace(entryPointPath, '/');

    router(req, res, next);
  }
}

And the relevant part of QueuesManagerService, which initializes bull-board. Note that initBullBoard is called as part of onModuleInit:

  readonly initBullBoard = () => {
    const funcPrefix = `${prefix}.initBullBoard: `;
    this.logger.log(`${funcPrefix}registering queues with UI`);
    const queues = this.queues.map((_) => new BullAdapter(_));

    const serverAdapter = new ExpressAdapter();
    const basePath = '/' + bullBoardPath;
    serverAdapter.setBasePath(basePath);

    createBullBoard({
      queues,
      serverAdapter,
    });

    this.router = serverAdapter.getRouter() as express.Express;
    this.logger.log(`${funcPrefix}registered queues with UI`);
  };

The odd thing is that serverAdapter is told the base path, and prefixes all the client requests with the base path, as appropriate. However, the express router that it returns does not use the given prefix to set its routes. Without the req.url = req.url.replace(entryPointPath, '/'); line, I was getting 404s since the router didn't match the path - it needs '/' instead of '/api/queues/admin/'.

[dotswing]

For basic auth I use the following code in my main.ts

import { NestFactory } from '@nestjs/core';
import * as passport from 'passport';
import { BasicStrategy } from 'passport-http';
import { BullMonitorBoardModule } from './bull-monitor-board.module';

async function bootstrap() {
  const bullBoardUserName = process.env.BULL_BOARD_USERNAME;
  const bullBoardPassword = process.env.BULL_BOARD_PASSWORD;
  const app = await NestFactory.create(BullMonitorBoardModule);
  passport.use(
    new BasicStrategy((username, password, done) => {
      if (username === bullBoardUserName && password === bullBoardPassword) {
        done(null, true);
      } else {
        done(null, false);
      }
    }),
  );
  const port = parseInt(process.env.PORT) || 8899;
  await app
    .use(
      '/admin/queues',
      passport.authenticate('basic', {
        session: false,
      }),
    )
    .listen(port);
}
bootstrap();

[Antoine-Genonceau]

@felixmosh finally got some time to flush this out, let me know what you think.

• Created a class to act as a pool for my queues

import { BullAdapter } from '@bull-board/api/bullAdapter';
import { BaseAdapter } from '@bull-board/api/dist/src/queueAdapters/base';
import { Injectable } from '@nestjs/common';
import { Queue } from 'bull';

@Injectable()
export class BullBoardQueue { }

export const queuePool: Set<Queue> = new Set<Queue>();

export const getBullBoardQueues = (): BaseAdapter[] => {
    const bullBoardQueues = [...queuePool].reduce((acc: BaseAdapter[], val) => {
        acc.push(new BullAdapter(val))
        return acc
    }, []);

    return bullBoardQueues
}

• In my main.ts I get my queue pool and loop over it and call addQueue

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { ExpressAdapter } from '@bull-board/express';
import { createBullBoard } from '@bull-board/api';
import { getBullBoardQueues } from './bull-board-queue';
import { BaseAdapter } from '@bull-board/api/dist/src/queueAdapters/base';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);

  const serverAdapter = new ExpressAdapter();
    const queues = getBullBoardQueues();

    serverAdapter.setBasePath('/admin/queues');
    app.use('/admin/queues', serverAdapter.getRouter());

    const { addQueue } = createBullBoard({
        queues: [],
        serverAdapter
    });

    queues.forEach((queue: BaseAdapter) => {
        addQueue(queue);
    });

  await app.listen(3000);
}
bootstrap();

I just import my class and add my queue pool => queuePool.add(updateQueue) — not perfect but gets the job done. Thanks for the help!

I spend the day trying to 'share' addQueue from main.ts to my queueService. A good solution was to use the HttpAdapterHost of my AppModule

@Injectable()
export class QueueService {

    constructor(
        @InjectQueue(QueueEnums.QueueName) private readonly _queue: Queue,
        private adapterHost: HttpAdapterHost
    ) {
        const serverAdapter = new ExpressAdapter() 
        serverAdapter.setBasePath('/admin/queues')
        createBullBoard({
            queues: [new BullAdapter(_queue)], 
            serverAdapter: serverAdapter
        })      
        this.adapterHost.httpAdapter.use('/admin/queues', serverAdapter.getRouter())
    }

[UncleVic]

And some more solutions. I decided that bootstrap isn't a good place for registering routes and middlewares.

At first, I created a class called BullBoardMiddleware, but it doesn't implement NestMiddleware:

@Injectable()
export class BullBoardMiddleware {
  static readonly ROUTE = '/admin/queues';

  readonly serverAdapter = new ExpressAdapter();

  constructor(@InjectQueue(OFFLINE_LOGS_QUEUE) private readonly queue: Queue) {
    this.serverAdapter.setBasePath(BullBoardMiddleware.ROUTE);
    createBullBoard({
      queues: [new BullAdapter(this.queue)],
      serverAdapter: this.serverAdapter,
    });
  }
}

Next, I registered the router in the AppModule:

export class AppModule implements NestModule {
  constructor(private readonly bullBoard: BullBoardMiddleware) {}

  public configure(consumer: MiddlewareConsumer): void {
    consumer
      .apply(this.bullBoard.serverAdapter.getRouter())
      .forRoutes(BullBoardMiddleware.ROUTE);
  }
}

Finally, (it's optional if you don't use setGlobalPrefix), I excluded the BullBoard route from setGlobalPrefix in the bootstrap:

app.setGlobalPrefix('api', { exclude: [BullBoardMiddleware.ROUTE] })

[zenstok]

If you have multiple queues and the queue registration is spread throughout your app, you can do something like this: Global BullQueueModule with 1 service:

@Global()
@Module({
  providers: [BullQueueService],
  exports: [BullQueueService],
})
export class BullQueueModule {}

BullQueueService:

import { Injectable, OnModuleInit } from '@nestjs/common';
import { HttpAdapterHost } from '@nestjs/core';
import { Queue } from 'bull';

import * as basicAuth from 'express-basic-auth';
import { ExpressAdapter } from '@bull-board/express';
import { createBullBoard } from '@bull-board/api';
import { BullAdapter } from '@bull-board/api/bullAdapter';

@Injectable()
export class BullQueueService implements OnModuleInit {
  private readonly queues = new Set<Queue>();

  constructor(private readonly adapterHost: HttpAdapterHost) {}

  addQueue(queue: any) {
    this.queues.add(queue);
  }

  onModuleInit() {
    const serverAdapter = new ExpressAdapter();
    serverAdapter.setBasePath('/admin/queues');

    createBullBoard({
      queues: Array.from(this.queues).map((queue) => new BullAdapter(queue)),
      serverAdapter,
    });

    this.adapterHost.httpAdapter.use(
      '/admin/queues',
      serverAdapter.getRouter(),
    );
    this.adapterHost.httpAdapter.use(
      '/admin/queues',
      basicAuth({
        challenge: true,
        users: { user: 'pass' },
      }),
    );
  }
}

Modules that add their queues to bull queue service:

@Module({
  imports: [
    BullModule.registerQueue({
      name: VIDEO_PROCESSOR_QUEUE,
    }),
  ],
  providers: [VideoProcessorService, VideoProcessorConsumer],
  exports: [VideoProcessorService],
})
export class VideoProcessorModule {
  constructor(
    @InjectQueue(VIDEO_PROCESSOR_QUEUE)
    public readonly videoProcessorQueue: Queue,
    public readonly bullQueueService: BullQueueService,
  ) {
    this.bullQueueService.addQueue(this.videoProcessorQueue);
  }
}

[felixmosh]

@zenstok createBullBoard function returns an object with add, replace, remove function to add queues dynamically, it may help to add queues "on the fly"

[lodi-g]

I used @asomethings's solution and added a BasicAuth middleware. My goal was to have everything related to queues in a single module and not spread out in main/bootstrap.ts.

import {
  DynamicModule,
  MiddlewareConsumer,
  Module,
  NestMiddleware,
  NestModule,
} from '@nestjs/common';
import { BullModule, InjectQueue } from '@nestjs/bullmq';
import { Queue } from 'bullmq';
import { createBullBoard } from '@bull-board/api';
import { BullAdapter } from '@bull-board/api/bullAdapter';
import { ExpressAdapter } from '@bull-board/express';

import { TestProcessor } from './test.processor';
import { NextFunction, Request, Response } from 'express';

class BasicAuthMiddleware implements NestMiddleware {
  private readonly username = 'user';
  private readonly password = 'password';
  private readonly encodedCreds = Buffer.from(
    this.username + ':' + this.password,
  ).toString('base64');

  use(req: Request, res: Response, next: NextFunction) {
    const reqCreds = req.get('authorization')?.split('Basic ')?.[1] ?? null;

    if (!reqCreds || reqCreds !== this.encodedCreds) {
      res.setHeader(
        'WWW-Authenticate',
        'Basic realm=Yours realm, charset="UTF-8"',
      );
      res.sendStatus(401);
    } else {
      next();
    }
  }
}

@Module({})
export class QueuesModule implements NestModule {
  static register(): DynamicModule {
    const testQueue = BullModule.registerQueue({
      name: 'test',
      defaultJobOptions: {
        attempts: 3,
        backoff: {
          type: 'exponential',
          delay: 1000,
        },
      },
    });

    if (!testQueue.providers || !testQueue.exports) {
      throw new Error('Unable to build queue');
    }

    return {
      module: QueuesModule,
      imports: [
        BullModule.forRoot({
          connection: {
            host: 'localhost',
            port: 15610,
          },
        }),
        testQueue,
      ],
      providers: [TestProcessor, ...testQueue.providers],
      exports: [...testQueue.exports],
    };
  }

  constructor(@InjectQueue('test') readonly queue: Queue) {}

  configure(consumer: MiddlewareConsumer) {
    const serverAdapter = new ExpressAdapter();
    const { addQueue, removeQueue, setQueues, replaceQueues } = createBullBoard(
      { queues: [new BullAdapter(this.queue)], serverAdapter },
    );
    serverAdapter.setBasePath('/queues');
    consumer
      .apply(BasicAuthMiddleware, serverAdapter.getRouter())
      .forRoutes('/queues');
  }
}

[felixmosh]

@lodi-g will you be able to create an "official" nestJS example?

[lodi-g]

@felixmosh Done in #569

[DennisSnijder]

@felixmosh @lodi-g coming back to this issue, I've just published a NestJS module which greatly simplifies using bull-board with NestJS. 😄

https://github.com/DennisSnijder/nestjs-bull-board

[felixmosh]

Looks cool @DennisSnijder, can it be implemented as ServerAdapter (as part of this lib)?

[DennisSnijder]

Looks cool @DennisSnijder, can it be implemented as ServerAdapter (as part of this lib)?

Looking at the IServerAdapter, I don't think so. NestJS is more of a framework layer on top of Express/Fastify. The module I created ultimately uses the ExpressAdapter (since that's the default for NestJS). From there the module makes it easy to register bull-board in the framework's dependency injection container and consume the registered queue's from there.

However, it could be an interesting addition to the bull-board library, since I've seen quite some people trying to use bull-board with NestJS. The module just makes it easier to get it integrated with NestJS. Let me know what you think 😄

[felixmosh]

I'm not familiar with Nest.js... It can be part if the bull-board scope, I need a small example of a nest project that uses your lib (to understand the usage) can you prepare one?

[DennisSnijder]

I'm not familiar with Nest.js... It can be part if the bull-board scope, I need a small example of a nest project that uses your lib (to understand the usage) can you prepare one?

Yes! I can do that! i'll link you the repository in a bit.

[DennisSnijder]

@felixmosh here you go! https://github.com/DennisSnijder/nestjs-bull-board-example

Make sure to run the redis container using the docker-compose.yml 😄. The example illustrates a minimal setup using @nestjs/bullmq library and the nestjs-bull-board library I created. If there's any confusion and or questions, let me know 😃.

[felixmosh]

@DennisSnijder it looks really clean ... Few small things that pop to mind

1. Where there is an access to the bull-board dynamic api (what is returns when you call to createBullBoard)
2. There is a way to change the underline express.js in nest.js, if so, where you can pass different server adapter?

[DennisSnijder]

@DennisSnijder it looks really clean ... Few small things that pop to mind

1. Where there is an access to the bull-board dynamic api (what is returns when you call to createBullBoard)
2. There is a way to change the underline express.js in nest.js, if so, where you can pass different server adapter?

Thanks for the feedback!

For the first one, i'll update the example on how to do that! For the second one, at the moment the ExpressAdapter is hardcoded, however.... NestJS only supports Express and Fastify, since you also have a Fastify adapter available, i'll take a look into supporting Fastify. Thanks!

edit: I updated the example repository with a "feature controller" whichs is getting the "BullBoardInstance" injected for usage.

[DennisSnijder]

@felixmosh I just updated the package to 1.2.0, update allows you to pass in either ExpressAdapter or FastifyAdapter. 😄

This is how that looks.

 BullBoardModule.forRoot({
    route: "/queues",
    adapter: FastifyAdapter, //or ExpressAdapter
 }),

Switching to the FastifyAdapter requires you to change NestJS to use Fastify: https://docs.nestjs.com/techniques/performance

[felixmosh]

@DennisSnijder, Looks GREAT!

Can you prepare a PR with a new package (@bull-board/nestjs-module) (wdyt about the name?) with the content of your lib, and let's make an example that uses this module (can be this example), let me know if you need any help.

Thank you for your contribution 🙏🏼.

[DennisSnijder]

@DennisSnijder, Looks GREAT!

Can you prepare a PR with a new package (@bull-board/nestjs-module) (wdyt about the name?) with the content of your lib, and let's make an example that uses this module (can be this example), let me know if you need any help.

Thank you for your contribution 🙏🏼.

Thanks! Sure! I will start working on the PR! Regarding the name, when it comes to other NestJS module packages, they rarely use the word "module" in there. Perhaps @bull-board/nestjs is just fine, what do you think? 😄

[felixmosh]

This is the reason that I've asked, since I'm not familiar with Nest.js ecosystem :]

[xtrinch]

Thanks @DennisSnijder I wouldn't have succesfully set this up without your original example, @felixmosh I would consider adding this into examples

[peternixey]

Hi chaps, do you all have any idea whether this still works? I can get @DennisSnijder's minimalist example to work but while it runs on the package.json file included, there are some conflicts when bringing up to current versions.

And I can't get bullboard working with nestjs at all. Chunks of UI are missing and it doesn't seem to register any of my queues.

My solution is probably going to be to run it on a raw express instance but I wondered whether I was missing anything obvious before trying that and wehther it should be expected to run on the latest versions of everything?

Thank yo