Can't send messages to a chat?

[vesche]

Sending messages to a group chat works fine like so:

from groupy.client import Client
client = Client.from_token(TOKEN)
for c in client.groups.list_all():
    chat = c
    break
message = chat.post(text="Hello, world!")

But when I try to send a message to a chat, it does not:

for c in client.chats.list_all():
    chat = c
    break
message = chat.post(text="Hello, world!")

received a bad response
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/groupy/session.py", line 28, in request
    response.raise_for_status()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 840, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://api.groupme.com/v3/direct_messages
Traceback (most recent call last):
  File "/usr/lib/python3.6/site-packages/groupy/session.py", line 28, in request
    response.raise_for_status()
  File "/usr/lib/python3.6/site-packages/requests/models.py", line 840, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 429 Client Error: Too Many Requests for url: https://api.groupme.com/v3/direct_messages

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.6/site-packages/groupy/api/chats.py", line 63, in post
    return self.messages.create(text=text, attachments=attachments)
  File "/usr/lib/python3.6/site-packages/groupy/api/messages.py", line 242, in create
    response = self.session.post(self.url, json=payload)
  File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 511, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/usr/lib/python3.6/site-packages/groupy/session.py", line 32, in request
    raise exceptions.BadResponse(response) from e
groupy.exceptions.BadResponse: Got a bad response

So the primary error seems to be: Too Many Requests for url: https://api.groupme.com/v3/direct_messages coming from the Requests library. I thought maybe this was because the Requests user-agent string was being rejected by the groupme website from too much abuse. I went into the Requests library and had it return a valid user-agent string by changing the default_user_agent function in /usr/lib/python3.6/site-packages/requests/utils.py. This didn't work unfortunately.

Anyone else have any idea why this might be happening?

[rhgrant10]

I've only ever seen the API return 429 when I make requests too quickly and rate limiting kicks in.

Unfortunately I'm on holiday without a laptop so I won't be able to investigate this for several days, but one thing you could try is to make the request using the underlying requests session and see if the response contains a Retry-After header indicating how long before you can try again. Also the API just might be returning some valuable information in JSON too.

[vesche]

I'm only sending single requests, and I got a 429 the first time I tried to use a chat. I'm not spamming messages or anything. I tried to recreate what's going on in api/messages.py here. But I must not be doing something right, because I keep getting 404's.

[vesche]

Alright, I got it working now. Here's what my code looks like:

import requests
import time

headers = {
    'content-type': 'application/json',
    'x-access-token': token
}

payload = {'direct_message': {'source_guid': time.time(), 'recipient_id': recipient_id, 'text': 'Hello, world!'}}

response = requests.post('https://api.groupme.com/v3/direct_messages', json=payload, headers=headers)

print(response.text)
print(response.headers)

Which results in:

429 Too Many Requests (Rate Limit Exceeded)

{'Content-Type': 'text/plain; charset=utf-8', 'Date': 'Tue, 03 Jul 2018 02:27:31 GMT', 'Server': 'nginx/1.12.2', 'Status': '429 Too Many Requests', 'X-Frame-Options': 'sameorigin', 'X-Xss-Protection': '1; mode=block', 'Content-Length': '44', 'Connection': 'keep-alive'}

I don't see a Retry-After header. The only interesting header is that they have X-Xss-Protection enabled. I doubt using their API would trigger XSS protection however.

I've tried multiple different recipients, I always get the same 429. Again, I'm not spamming. I've never had a direct message work, but sending messages to group chats works fine.

[rhgrant10]

Yeah them providing that header was kind of a long shot. I also don't see that xss would come into play here. I'm fairly certain that this worked when I did functional testing on it... not sure why you're getting rate limited. The docs don't mention that at all. Could be that they changed things on their end.

I'm be happy to investigate what's going on when I return. One tactic I found helpful was to reverse engineer what their web client does so you might try that for further insights. I've not had much success with reaching out to the folks at GroupMe since they did away with their help chat but perhaps they can pinpoint what's wrong.

Once we know what to change I can update the code (or review a PR :smile:)

[vesche]

I'd be happy to send a PR when I can figure out what's going on :(

Just as a sanity check I tried changing my IP (using proxy & VPN) and changing my user agent string to something more organic- both, no dice.

I took your advice and reverse engineered the request through the web client. What your API does and what my browser is doing are nearly identical. The parameters and URL are exactly the same. I went ahead and mirrored the request headers and... still getting 429.

headers = {
    'Accept': 'application/json, text/plain, */*',
    'Accept-Encoding': 'gzip, deflate, br',
    'Accept-Language': 'en-US,en;q=0.5',
    'Connection': 'keep-alive',
    'Content-Type': 'application/json;charset=utf-8',
    'Host': 'api.groupme.com',
    'Origin': 'https://web.groupme.com',
    'Referer': 'https://web.groupme.com/',
    'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64?) Gecko/20100101 Firefox/60.0',
    'X-Access-Token': token
}

One thing I did notice is that my token is different in the web client than my dev token, which could explain all this... Perhaps GroupMe is blocking direct messages for dev tokens specifically? Maybe they were getting a lot of global abuse / malicious activity and had to stop it? I hope that this isn't happening to just me, because that would be really confusing.

[rhgrant10]

Dang. I really hoped there would be a significant difference compared to the web client.

Your theory about blocking certain tokens is plausible but it's not clear to me how they would differentiate a seemingly identical request. A big part of me wants to cut vacation short so I can see whether I'm also seeing this behavior :joy:

Another thing we could try is to create a new account and see if it's being blocked. That gets tricky though unless you happen to have a spare phone number laying around...

[rhgrant10]

Okay, finally got a chance to try and send a chat message myself. Right off the bat I got a 429 for my first attempt. Something must have changed about the API. Time to see what the devs at GroupMe have to say about this.

[rhgrant10]

@vesche okay, so here's what finally worked. I created an Application at https://dev.groupme.com/applications. At the end of that process it provides you an API access token so you can go ahead and start making requests as yourself without using oauth. Note that despite authenticating you as you, this token is different. Anyway, using that token worked without a problem. I'm fairly certain that I didn't have to go through this process when I did functional testing before so I'm trying to confirm that with the devs...

Either way, I'll update my documentation to mention this. Thanks for bringing it to my attention and sorry for the time you lost looking into it.

[vesche]

Thanks for getting back to me, and I'm glad it wasn't an isolated issue. I will look into creating an application and getting a token that actually works. Appreciate the help.