bug: default value of user-agent flagged as bot by is-bot

[manuel-rw]

Hello, I am using Umami to send some data from a server side Node.js application. This applications uses @umami/node to send track data to the Umami instance. Some example code for this:

const umamiInstance = new Umami();

umamiInstance.init({
  hostURL: UMAMI_HOST_URL,
  websiteId: UMAMI_WEBSITE_ID
});

await umamiInstance.track('my-event', {
  "exampleKey": "exampleValue"
});

The track() function returns a Response from the fetch: https://github.com/umami-software/node/blob/933104710f268ec512566bd22fd68a157ae3fae5/index.ts#L41-L48

I noticed that no matter what I configured, my Umami track events were not showing up in Umami. So I decided to dig a bit in the code and found a potentially unwanted behaviour / default value.

In the Umami server, there are multiple paths for the runtime to go:

https://github.com/umami-software/umami/blob/7a75639dd3d7aeff46104b71ebfb3853fc0eee09/src/pages/api/send.ts#L85-L100

One of them is the bot check that will fail silently if the request is being detected as a bot by the is-bot package. The default value of your library here is Mozilla/5.0 Node/${process.version} -> the variable interpolation doesn't matter in this case:

https://github.com/umami-software/node/blob/933104710f268ec512566bd22fd68a157ae3fae5/index.ts#L45

is-bot detects this as a bot and will therefore return true on the check:

Therefore, Umami will silently drop this track event without log message or any indications why it failed. Additionally, the 200 status code indicates that the request was successful (which it wasn't).

A possible workaround is to set the undocumented variable DISABLE_BOT_CHECK to true - but this disables the check entirely. It would be favorable if this library has a different user agent as the default value or supports setting a custom agent.

If Umami thinks that this is intended behavior, please add a log warning to Umami server.

The commit 933104710f268ec512566bd22fd68a157ae3fae5 has not changed this problem, because the regex from is-bot will match either user agent.

[MichaelBelgium]

Bump, encountering this too. The package is unusable

And the usage in the readme is kinda not working either, it only does when we instantiate some variable with the main Umami class

eg

import { Umami } from '@umami/node';

const umami = new Umami()

//...

Not a biggie but just mentioning

[manuel-rw]

@mikecao tagging you for visibility

[franciscop]

Same thing here, after some debugging and basically reading+debugging this code locally I realized I had two bugs:

• "hostUrl" was supposed to be https://cloud.umami.is/ and not the "server host url" as I thought (meaning, not my website url). Recommending either explicitly saying it, and/or defaulting to "https://cloud.umami.is/".
• Receiving { "beep": "boop" } with the default package, meaning it believes this is a bot. It's nice that umami.track() returns the reply though! Would love to see that documented as well.

Note: I'd be happy to send PRs about the documentation as well, just mentioning it to see if it's wanted before doing the work.

[franciscop]

As a temporary fix, and since I only need to track page views right now, I wrote this, which is not detected as a bot:

// Function definition
async function track(url, { userAgent, ...data } = {}) {
  const payload = { website: process.env.UMAMI, url, data };
  const res = await fetch(`https://cloud.umami.is/api/send`, {
    method: "POST",
    headers: { "Content-Type": "application/json", "User-Agent": userAgent },
    body: JSON.stringify({ type: "event", payload }),
  });
  if (!res.ok) console.error(await res.text());
}

// Usage
await track(ctx.url.pathname, { userAgent: ctx.headers["user-agent"] });

[mikecao]

Sorry for the delay. Just published v0.4.0 that should resolve the isbot issue as well as let you pass in your own user agent.

[franciscop]

Just after I published mini-umami and tagged Umami on Twitter! Happy to see it fixed though! :)

Is there a way to also pass the client IP to Umami Cloud BTW? I could only find that Umami (the main project/repo) uses request-ip, but I attempted to pass X-Client-IP and I'm not sure if it worked or not. I found that if Umami Cloud set any custom headers IP set it'd use that instead, but couldn't find info about whether Umami Cloud has that variable set or not (and if so, to what value):

export function getIpAddress(req: NextApiRequestCollect) {
  const customHeader = String(process.env.CLIENT_IP_HEADER).toLowerCase();
  // Is this set in Umami Cloud??
  if (customHeader !== 'undefined' && req.headers[customHeader]) {
    return req.headers[customHeader];
  }
  // ...
  return getClientIp(req);
}

[mikecao]

We will be upgrading the API soon and it will allow you to specify a unique value for the session. Since IP is only used for calculating the session, you'll be able to pass the IP or any other value you want.

[franciscop]

Oh I thought it'd also be used for the location, since in Umami dashboard the location is def "wrong" when sending events through Node.js, but that's a topic for a different thread though, thanks!

[manuel-rw]

Thanks for the fix. Closing this now as the release has been published. Thank you