discord / discord-api-docs

Official Discord API Documentation
https://discord.com/developers/docs/intro
Other
5.91k stars 1.25k forks source link

HTTP calls to any API function results in cloudflare returning a 403 #849

Closed UltraTechX closed 5 years ago

UltraTechX commented 5 years ago

When I try to send a message using HTTP POST I get the following response body along with a 403 forbidden error. A google search of the title "Attention Required! | Cloudflare" seems to suggest it being a captcha. It only stopped working about a week ago:

(response placed in gist because it didnt seem to format properly) https://gist.github.com/UltraTechX/29153aa04723f9bb116977c37c6f6238

I am using the following url, parameters, and headers: Url: https://discordapp.com/api/channels/480682139881570306/messages Headers: Authorization = Bot <InsertTokenHere> User-Agent = MyBot Content-Type = application/x-www-form-urlencoded

Parameters: content = test

It also might be worth mentioning that it does work on a modern web browser, but the error occurs when I try to use an older web browser that a game I am trying to write a bot for (that uses its own browser) attempts to send messages.

shikhir-arora commented 5 years ago

That's Cloudflare's firewall triggering a CAPCHA/verification that it expects for that request. This can't be modified manually by Cloudflare, and the settings for the global firewall and rules are set by Discord. Most likely the IP or browser type/pattern of connections is causing that.

It generally is not good practice to send tokens like that over a browser anyway, though, especially in production.

UltraTechX commented 5 years ago

Is the captcha something new for older browsers? It worked without issue until just a few days ago. Its been running for a few months now without problems

Zoddo commented 5 years ago

The main issue here, is Discord not exempting API endpoints from the CloudFlare's browser check. How they expect bots to pass this check (which is meant to identify and block bots)?

shikhir-arora commented 5 years ago

The main issue here, is Discord not exempting API endpoints from the CloudFlare's browser check. How they expect bots to pass this check?

I don't think the intended usage of the bot API is to send requests with your bot's token through a browser at any real scale - that's not the safest thing to do, even if it is encrypted. Applications that need to send messages can do so many ways, but doing it through a browser for anything but private development or something fairly local is not ideal. It's also likely to trigger systems like the Cloudflare firewall since that is more prone to malicious use. (in this case, it isn't malicious, but the settings are fairly global).

z64 commented 5 years ago

I would try passing the correct user agent; MyBot is not to spec -

https://discordapp.com/developers/docs/reference#user-agent

Zoddo commented 5 years ago

I don't think the intended usage of the bot API is to send requests with your bot's token through a browser

And so, they are triggering a check that can only be passed by web browsers to block web browsers? 🤔

UltraTechX commented 5 years ago

I would try passing the correct user agent; MyBot is not to spec -

https://discordapp.com/developers/docs/reference#user-agent

I changed the user agent to: "DiscordBot (http://discordapp.com/api/guilds/480682139881570304/audit-logs, v1)" and it still returns the same cloudflare captcha with a 403.

shikhir-arora commented 5 years ago

I don't think the intended usage of the bot API is to send requests with your bot's token through a browser

And so, they are triggering a check that can be only passed by web browsers to block web browsers? 🤔

No, it is likely fairly global and deployed for the entire https://discordapp.com/* site/scope (since pretty much all of Discord's HTTP stuff goes through Cloudflare), so it's triggering just like it would for a user request that was caught by the rules in the filter.

UltraTechX commented 5 years ago

Well something had to have changed on discord/cloudflares end because it was working fine just days ago. I didnt edit anything before it decided to start returning 403.

It first started returning 403 yesterday right after 3PM EST and hasnt stopped since. Keep in mind it has been working properly for months before this occurred.

jhgg commented 5 years ago

It's very well possible your IP address was banned or flagged for API abuse. I'd reach out to support: https://dis.gd/support for more assistance - as we're not able to provide assistance for that here.

UltraTechX commented 5 years ago

I do not believe that is the case as it sends the message completely fine using Postman on the latest version of chrome (Chrome 71). The version of the browser on the same network trying to send and failing is Chrome 18.

sirrobindoger commented 5 years ago

Also having issues, same circumstance.

jhgg commented 5 years ago

Chrome 18?

UltraTechX commented 5 years ago

Chrome 18?

yes, that is the version of chrome identified in the agent of the browser that gets the 403 error: Mozilla/5.0 (Windows; Valve Source Client) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1003.1 Safari/535.19 Awesomium/1.7.5.1 GMod/13

The browser that works with sending messages using HTTP calls and is on the same network a much newer version of chrome (Chrome 71 where the latest version is 72)

Cynosphere commented 5 years ago

Clearing some things up:

So you've pretty much screwed over the entirety of Garry's Mod's playerbase (not the first time mind you) just like you've done in the past with ROBLOX users and webhooks.

You're shutting out the use of grabbing from the CDN (mainly emotes and possibly avatars) and even having a relay without having to use some sort of middleware or proxy to do so. It's annoying and inconveniencing.

I'm tired of seeing this platform going in the worst directions possible.

UltraTechX commented 5 years ago

Clearing some things up:

  • It is not an IP ban at all and just an outright block through Cloudflare (speculating that an option was changed in Cloudflare to block certain useragents)
  • Chrome 18 is Awesomium, which is used by the default branch for Garry's Mod
  • Chrome 65 is also blocked, which is the chromium branch for Garry's Mod

So you've pretty much screwed over the entirety of Garry's Mod's playerbase (not the first time mind you) just like you've done in the past with ROBLOX users and webhooks.

You're shutting out the use of grabbing from the CDN (mainly emotes and possibly avatars) and even having a relay without having to use some sort of middleware or proxy to do so. It's annoying and inconveniencing.

I'm tired of seeing this platform going in the worst directions possible.

Yea now we have to use a proxy to make it work properly, and I agree its annoying. I sort of doubt they will read this though as most people dont care about tickets once they are closed.

Also I run my own website through cloudflare, and it did block Garry's Mod from displaying it as a loading screen until I messed with the cloudflare settings, so I do believe that a setting they changed in cloudflare is the cause of this. Hopefully they can reverse the change and not screw over an entire playerbase over an easy solution to remote moderation.

VictorienXP commented 4 years ago

I get a 403 when I just want to execute a webhook from Garry's Mod. This is really nice...

Why this issue is closed @jhgg? This issue is really affecting all Garry's Mod users. And it is not an IP ban.

It seems Cloudflare doesn't want us because it expects the Garry's Mod's HTTP() function to support cookies?! (CloudFlare asks for enabled cookies in the response body but it may be unrelated)

Do we have to makeshift cookie support now?! wtf?! This was working fine like two years ago.

So who should fix that?

I want answers and don't consider this issue closed until you're really aware of it and giving it a proper response.

Response body:

<!DOCTYPE html>
<!--[if lt IE 7]> <html class="no-js ie6 oldie" lang="en-US"> <![endif]-->
<!--[if IE 7]>    <html class="no-js ie7 oldie" lang="en-US"> <![endif]-->
<!--[if IE 8]>    <html class="no-js ie8 oldie" lang="en-US"> <![endif]-->
<!--[if gt IE 8]><!--> <html class="no-js" lang="en-US"> <!--<![endif]-->
<head>
<title>Attention Required! | Cloudflare</title>
<meta charset="UTF-8" />
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=Edge,chrome=1" />
<meta name="robots" content="noindex, nofollow" />
<meta name="viewport" content="width=device-width,initial-scale=1,maximum-scale=1" />
<link rel="stylesheet" id="cf_styles-css" href="/cdn-cgi/styles/cf.errors.css" type="text/css" media="screen,projection" />
<!--[if lt IE 9]><link rel="stylesheet" id='cf_styles-ie-css' href="/cdn-cgi/styles/cf.errors.ie.css" type="text/css" media="screen,projection" /><![endif]-->
<style type="text/css">body{margin:0;padding:0}</style>

<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/zepto.min.js"></script><!--<![endif]-->
<!--[if gte IE 10]><!--><script type="text/javascript" src="/cdn-cgi/scripts/cf.common.js"></script><!--<![endif]-->

</head>
<body>
  <div id="cf-wrapper">
    <div class="cf-alert cf-alert-error cf-cookie-error" id="cookie-alert" data-translate="enable_cookies">Please enable cookies.</div>
    <div id="cf-error-details" class="cf-error-details-wrapper">
      <div class="cf-wrapper cf-header cf-error-overview">
        <h1 data-translate="block_headline">Sorry, you have been blocked</h1>
        <h2 class="cf-subheadline"><span data-translate="unable_to_access">You are unable to access</span> discordapp.com</h2>
      </div><!-- /.header -->

      <div class="cf-section cf-highlight">
        <div class="cf-wrapper">
          <div class="cf-screenshot-container cf-screenshot-full">

              <span class="cf-no-screenshot error"></span>

          </div>
        </div>

Example test code from Garry's Mod:

local function webhook_failure(reason)
    ErrorNoHalt(reason)
end

local function webhook_success(code, body, headers)
    print(code)
    print(body)
    PrintTable(headers)
end

local webhook = "webhook url here"

local function execute_webhook(webhook, params)
    return HTTP({
        failed = webhook_failure,
        success = webhook_success,
        method = "POST",
        url = webhook,
        --[[headers = {
            ["Content-Type"] = "application/json",
            ["User-Agent"] = "Garry's Mod Lua"
        },]]
        body = util.TableToJSON(params),
        type = "application/json"
    })
end

hook.Add("PlayerSay", "Discord_Webhook_PlayerSay", function(ply, text, team)
    if !team then
        print(execute_webhook(webhook, {content = text}))
    end
end)

References: https://wiki.facepunch.com/gmod/Global.HTTP https://wiki.facepunch.com/gmod/Structures/HTTPRequest https://wiki.facepunch.com/gmod/GM:PlayerSay https://discordapp.com/developers/docs/resources/webhook#execute-webhook

EstevanTH commented 4 years ago

@VictorienXP the function HTTP() from Garry's Mod does support cookies. It just does not support multiple cookies set by the server during the same call because the response headers are contained in a key-value table (aka. hashtable / dictionary), so only 1 Set-Cookie header remains.

My current suggestions are, if more that 1 cookie is set in the response:

UltraTechX commented 4 years ago

From the time I initially made this issue, I have seen the following workarounds:

1 - Route requests through a proxy server (like @EstevanTH said)

2a - Use a websocket binary module, like gwsockets

2b - A friend of mine recently made a library based off this method, which may interest you, called discord.glua

VictorienXP commented 4 years ago

Like I said I'm mostly complaining about the fact that we need to add extra steps for something that used to be simple and straightforward...

I was just looking to do some simple webhook executions and nothing more. Nothing fancy. But no I need to basically rely on a third party anything to make the call for the script... I'm just saying it's dumb.

Anyway yeah now that I think about it, if the header names are directly in keys it's bad for all those non unique headers. I see now how I can't rely on HTTP() for some stuff.

And no using 64 bits/(updated)Chromium gmod won't fix this. And I don't believe HTTP() is using the embedded browser. I'm saying that based on the reported user agent which is Valve/Steam HTTP Client 1.0 (4000). Which is not the same user agent as the browser frame which is reported as: Mozilla/5.0 (Windows; Valve Source Client) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1003.1 Safari/535.19 Awesomium/1.7.5.1 GMod/13 (normal branch) or Mozilla/5.0 (Windows NT; Valve Source Client) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.122 Safari/537.36 GMod/13 (x86-64 branch)

But maybe using a browser frame could be some kind of an another workaround that won't involve needing other stuff. But it's only clientside...

In the end I don't really care and I wasn't asking for workarounds. Sorry for digging up that issue. I don't think Discord would care about it anyway.

jhgg commented 4 years ago

Valve/Steam HTTP Client 1.0 (4000) is deliberately blocked due to abusive traffic patterns spread across a variety of clients. It's the same reason Roblox/WinInet is blocked as well.

People distribute bad (not necessarily malicious, but... poorly coded) scripts that do stuff like poll for messages, or spam webhook requests.

Since these scripts were distributed to end-users and not run on a single server - we were not really able to ban say "one bad server sending us requests" because there were "tens of thousands of clients" sending us bad requests. So, we block these UA's, forcing you to route traffic through a single server, that if the traffic is abusive, we can (automatically) block the single IP.

There is no plans to unblock these user agents. At it's peak, we were seeing thousands of bad requests per second from them (ratio of 9:1 bad to good requests from these user agents).

VictorienXP commented 4 years ago

Thanks for your response

Cynosphere commented 4 years ago

If you want a simple "proxy" server, matterbridge is a good bridge/middleman, especially if you have multiple services you want to link together. While there's not a public adapter for GMod, matterbridge's HTTP API is very straight forward. I might eventually release mine but its a bit proprietary which is why i haven't done so yet.

CapsAdmin commented 4 years ago

How about requiring authorization tokens for the flagged user agents?

greendoescode commented 2 years ago

I'm reopening this. I'm getting a captcha from cloudflare, when it's just playing music from youtubde_dl!

(403 forbidden Access denied | discord.com used Cloudflare to restrict access)

There is NO way for me to bypass this, seeing as I can't get the same reaction from the the web browser.

https://mystb.in/FixturesCartoonLanguages.xm - This is the full error, the reason it shows as a python error is because its to big for a message.

Jupith commented 2 years ago

Discord blocked the typing endpoint temporarily

greendoescode commented 2 years ago

Thank you for saying, but how come all the endpoints? Seeing as I cannot even run a command without and error. I get on the status page it shows the "typing" is down. It should say what end points are down

Cynosphere commented 2 years ago

I personally feel that your addition to this issue is unrelated, as the original issue was related to at the time unexplained 403's from specific game(server) user agents. Bumping old closed issues just because they have a vague title related to your issue without reading the contents of the previous discussion should be discouraged.

but how come all the endpoints? Seeing as I cannot even run a command without and error. I get on the status page it shows the "typing" is down. It should say what end points are down

This is down to poor error handling on your (library's) part. Only the typing endpoint is down and 403ing for the time being.

greendoescode commented 2 years ago

I am sorry, As I said. The only reason I replied here is because there was no other webpages saying about the issue.

This was the closest issue and didn't know if I made a new one if it would be considered spam.

Again I am sorry

ibanner56 commented 2 years ago

As someone who just started seeing the error now, the recent comments are the only reason I found this and got any information about the problem. If anything GreenDiscord raising this back up was more helpful than doing nothing 🤷.

NovaFox161 commented 2 years ago

If you're subscribed to the DDevs announcements channel you can have stuff like this broadcasted to your servers. All of the major bot list servers and discord API dedicated servers subscribe to this announcement channel.

What else do you want? a client pop up? It was announced quickly and responsibly and honestly for the first time ever I'm completely happy with how discord handled this.

sirrobindoger commented 2 years ago

As someone who just started seeing the error now, the recent comments are the only reason I found this and got any information about the problem. If anything GreenDiscord raising this back up was more helpful than doing nothing 🤷.

This issue (3 years old) was opened by my friend Ultra only because it returned a cloudflare error on a garrys mod server running chrome 18. Unless your issue pretains to that, it is unrelated to this issue.

ibanner56 commented 2 years ago

If you're subscribed to the DDevs announcements channel you can have stuff like this broadcasted to your servers. All of the major bot list servers and discord API dedicated servers subscribe to this announcement channel.

We're leveraging discord.py, so we're one step removed from the issue, hence the lack of visibility here. Anyways, apologies for the spam I guess.

MahdeenSky commented 2 years ago

I started running into this issue ever since, the 10th of October, and I resolved this issue by changing the user agent of the post request to the api to just "myBot" as I was using a "bot apiKey" as my authorization.

fzn0x commented 2 years ago

Using proxy rotator solves this issue without any further workaround.

Juct commented 2 years ago

Discord WebHook is locked in garrys mod, so I made a bridge.

FIX! https://pastebin.com/ZCQ1bhuV GARRYS MOD