Multiple hosts support?

[broql]

Hello,

I couldn't find in docs anything related to multiple hosts. My ideal would be to have ability to provide options as array. Like this:

const opts = { host: ['http://example.com', 'http://example2.com'], record: talkback.Options.RecordMode.NEW, port: 5544, path: "./offline-tapes" };

Is this already possible or is it something I need to introduce to the project via PR?

[ijpiantanida]

how would talkback know to which host it should proxy to?

The way that it's currently recommended is to start 2 different independent talkback servers, but I'm open to suggestions on how to improve this!

[reith]

Can't it just get the host from Host header? How about running Talkback as an actual HTTP Proxy Server which clients are configured to use as proxy, rather than sending original HTTP requests directly to?

[ijpiantanida]

Why not as a proxy? The main intent of talkback has been to be used as a "mock" server for integration tests. With that in mind, using talkback as a proxy would mean that the app/client now behaves differently when you test vs when you use the real server (now the http client/app env needs all the proxy setup), which can introduce subtle bugs.

Having said that, it could be added as an option, to run as a "proxy" vs "http-server". I will re-open the issue to consider this.

[denislutz]

I use talkback all over the place proxy or mock server in integration tests mostly in development situations. What I miss for now is a clear way to support a multiple microservices situation, where the requests are going to many different back ends but are all saved into one set of tapes, separated by folders. What I ended up doing is a small express.js server handling this case. I think it would be useful to extend talkback to support that with a clean example.

const express = require("express");
const talkback = require("talkback");

const TAPES_PORT = 3009;

// place here any back ends you need....
const allEnvs = {
  local: {
    "your.host.localhost": "https://your.host.localhost",
    "your.other.host.localhost": "https://your.other.host.localhost",
  },
  dev: {
    "your.host.localhost": "https://your-dev-host.com",
    "your.other.host.localhost": "https://your-dev-host.com",
  }
}

const urls = allEnvs[process.env.BACKEND_SOURCE || 'local']

console.log(urls)

let record = talkback.Options.RecordMode.DISABLED;
if (process.env.RECORD) {
  record = talkback.Options.RecordMode.NEW;
  console.log(
    `Recording is active, but only really new tapes will be recorded (RecordMode:${record})`
  );
}
if (process.env.OVERWRITE) {
  record = talkback.Options.RecordMode.OVERWRITE;
  console.log(
    `Recording is OVERWRITE, ALL tapes will be called first (RecordMode:${record})`
  );
}

const tapeNameGenerator = (tapeNumber, tape) => {
  const { req } = tape;

  const TAPE_MAX_FILE_LENGTH = 150;
  const stripUnfriendlyChars = req.url ? req.url.replace("?", "Q") : "NOURL";
  const stripUnfriendlyCharsShortened = stripUnfriendlyChars.slice(
    0,
    Math.max(0, TAPE_MAX_FILE_LENGTH)
  );
  const fileNameEncodedUrl = stripUnfriendlyCharsShortened;

  return `generated/${req.method}/${fileNameEncodedUrl}_${tapeNumber}`;
};

const services = {};

async function createHandlers() {
  if (Object.keys(services).length > 0) {
    return;
  }
  for (const serviceName of Object.keys(urls)) {
    const service = await talkback.requestHandler({
      host: urls[serviceName],
      path: __dirname + `/tapes/${serviceName}`,
      record: record,
      debug: false,
      name: serviceName,
      allowHeaders: [],
      tapeNameGenerator,
      summary: true,
    });

    services[urls[serviceName]] = service;
  }
}

express()
  .use(async function (request, res, next) {
    const serviceName = request.get("host").split(":")[0];

    await createHandlers();

    let fullHost = urls[serviceName];
    let requestHandler = services[fullHost];

    console.log("Incoming", serviceName, fullHost);

    let requestBody = Buffer.alloc(0);
    if (request.body) {
      requestBody = Buffer.from(request.body);
    }

    console.log(request.originalUrl)
    const pathWithQuery = request.originalUrl
    const talkbackRequest = {
      url: pathWithQuery,
      method: request.method,
      headers: request.headers,
      body: requestBody,
    };

    requestHandler
      .handle(talkbackRequest)
      .then((r) => {
        res.status(r.status).send(r.body);
      })
      .catch((error) => {
        console.log("Error handling talkback request", error);
      });
  })
  .listen(TAPES_PORT);

console.log("Listening", TAPES_PORT);

[ijpiantanida]

@denislutz that's a great way to setup talkback.

What I'm curious about from your example is what setup do you use to point all your different microservice hosts ("your.host.localhost", "your.other.host.localhost") to the same express app?

[reith]

I'm using talkback in a similar method but in standalone mode. I'm testing actual webpages, not just APIs. My software is based on Puppeteer and I'm lucky I can map domain names to local addresses to direct the traffic to talkback servers. It eliminated multiple talkback servers if talkback would actually consider host name during tape matching and recording. That said, caching and replaying modern web pages with lots of dynamic urls generated by JavaScript, is not always possible and I still couldn't find a perfect solutions.

On Mon., Feb. 20, 2023, 7:28 p.m. Ignacio Piantanida, < @.***> wrote:

@denislutz https://github.com/denislutz that's a great way to setup talkback.

What I'm curious about from your example is what setup do you use to point all your different microservice hosts ("your.host.localhost", "your.other.host.localhost") to the same express app?

— Reply to this email directly, view it on GitHub https://github.com/ijpiantanida/talkback/issues/36#issuecomment-1437810418, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMVLOOLUMIXKINW2GLSGV3WYQY6VANCNFSM4OB6TKTQ . You are receiving this because you commented.Message ID: @.***>

[denislutz]

@denislutz that's a great way to setup talkback.

What I'm curious about from your example is what setup do you use to point all your different microservice hosts ("your.host.localhost", "your.other.host.localhost") to the same express app?

Yes its the same app, its the file itself. The point is that its really one local talkback server that can support countless microservices... The only on extra here is that I have configured some stuff in /etc/hosts