User agent blocked

[Emilleopold]

Hello, today I saw that the timber-query is giving this debug message :

{"responseMessage":"User agent bisand/tibber-api/5.1.9 is blocked. Please downgrade to the previous version or upgrade to a later version that works properly","httpCode":400,"statusCode":400,"statusMessage":"Bad Request"}

The flow was working for a long time without problems.

Is this a temporary issue ?

My Version of node-red-contrib-tibber-api is 6.0.1, should be the latest.

Thanks for any information.

[andreaske76]

I have the same issue

[tossa-fan]

Same issue here

[ErZett13]

Same here

[Zomhad]

+1

[soeren487]

Here the same. Have also version 6.0.1 in latest Node Red.

[stafbulp]

+1

[hombach]

new bisand/tibber-api/5.1.11 is working fine for me - try to upgrade

[bisand]

A new version of node-red-contrib-tibber-api (v6.0.4) that should resolve this issue has just been released. It will be available for download/install in short time.

I will start working on a new version that will enable setting a custom userAgent prefix in the config node.

[soeren487]

v6.0.4 is working fine for me! Thanks for providing an update that fast!

[gizmocuz]

@bisand , I was experiencing the same with my opensource project... To solve this, I generated a user agent string at startup (or sometimes before every call) like

std::string GenerateUserAgent()
{
    std::random_device rd;  // a seed source for the random number engine
    std::mt19937 gen(rd()); // mersenne_twister_engine seeded with rd()
    std::uniform_int_distribution<> distrib_FFFF(0, 0xFFFF);

    int cversion = distrib_FFFF(gen) % 0xFFFF;
    int mversion = distrib_FFFF(gen) % 3;
    int sversion = distrib_FFFF(gen) % 3;
    std::stringstream sstr;
    sstr << "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/" << (601 + sversion) << "." << (36+mversion) << " (KHTML, like Gecko) Chrome/" << (53 + mversion) << ".0." << cversion << ".0 Safari/" << (601 + sversion) << "." << (36+sversion);
    return sstr.str();
}

Of course it is needless to say you should not use your own strings... (bisand/tibber...) but use existing browser strings

[bisand]

@gizmocuz, to clearify, I was contacted by Tibber and they asked me to use an own UserAgent string for this library (bisand/tibber…), to be able to identify and potentially block unwanted user agents, which the clearly have done now!

I wonder if I should instead have a more generic string, like your suggestion. I do not have too much time working on this, so every contributions are welcome!

Thanks!

[gizmocuz]

I'm afraid I am no expert in Node-Red or API integrations, but looking in the code I noticed on 4 places you create the current string. And there is an option in the config for this it seems. But a random (correct) string is the best for various reasons

[jstaro]

Hi,

We did indeed block bisand/tibber-api/5.1.9 because of excessive malformed queries that DoS'd the API and degraded the experience for all the users of our API and platform. It is hard for us to tell if the bug is in your library, or if someone is using your library incorrectly. We should've contacted you earlier. Sorry about that.

Setting this is a good practice, and we encourage users of the library to use it so we can more easily figure out what app/integration might need some attention or help from us.

We would advise against spoofing browser user-agents or generating random user-agents. We have so far been fairly lax in our allowing nonconforming user-agents, but we're considering becoming more strict (and actually following what we're saying in our documentation) to mitigate the havok some of these kind of bad clients can wreak. Normally, almost 2% of all calls we get today are complete garbage, after all.

FWIW, we have not yet seen any bad behavior by any clients reporting the 5.1.11 user-agent, but will let you know if we do.

[gizmocuz]

@jstaro, yes, "if someone is using your library incorrectly", this is usually the case. I have the same issues and for this reason I enabled a rate-limiter on our NGNIX server. And blocking out countries where Tibber is not operating (yet) might be an idea. (Vietnam, Korea etc)

Requesting dynamic prices more than a few times a day is not needed, but I can assume users are polling each hour.

Maybe for this project it's an option to store internally the result in the flow and compare this when another request is made. If today prices are requested, check if it is already requested online before and return that result. If it's another day, refresh the data

The problem I have with blocking a complete user-agent api-version, is that systems depend on it. I don't want to check for new updates for Node Red modules or API versions each day/week. Sure, If I receive an email that API version x.y.z gets deprecated, then it's time for action.