HTTP GET request : status code 401

[arjunior26]

Hi Joe, Despite working link to download m3u file from my navigator, I've got status code 401 when running your script. Of course, my username and password are correctly set in the URL. Any idea ? Thanks.

[bebo-dot-dev]

No idea tbh, 401 is some server telling you that you're not authorized. I'm guessing that it has worked in the past because you've opened a couple of issues here in the past. What changed?

[arjunior26]

I've changed IPTV provider

[arjunior26]

Is file protocol really support ? I've tried -m="file://myfile.m3u" but doesn't work

[bebo-dot-dev]

yep file:// does work, if you're using CLI args try this: -m="file:///myfile.m3u"

Forward slashes need escaping, it looks like you might have two forward slashes and I think you need three for it to escape correctly.

[arjunior26]

OK you are right, slashes need escaping. I've also need to define correct path file : -m="file:///"e:\m3u\myfile.m3u"" I think I would use file protocol as connection from your script is refused by remote server.

Thank you for your help.

[bebo-dot-dev]

Cool, sounds like you've found a workaround.

It seems odd but it's not something that I can debug for you here.

The python script does nothing special to perform a HTTP GET request tbh, it's pretty much bog standard python requests module stuff as outlined in the docs here: https://docs.python-requests.org/en/latest/user/quickstart/

If you want to debug it yourself, you can test it directly at the command line with something similar to this to see what happens:

joe@joes-HP-EliteBook-8470p ~ $ python
Python 3.9.2 (default, Apr  3 2021, 09:20:11) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> r = requests.get('http://your.m3u.url?username=X&password=Y')
>>> print(r)
<Response [200]>
>>>

[arjunior26]

Yes it's strange I can't perform a HTTP GET request with this reote server. I've tested what you suggested and get 401 response too using python directly :

C:\Users\arjun>python
Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> r = requests.get('http://xxxxxxxxxxxxxx.xxx/get.php?username=xxxxxxxxx&password=yyyyyyyyyyy&type=m3u_plus&output=ts')
>>> print(r)
<Response [401]>

So for sure, no solution for this problem ! Thank you anyway ;)

[bebo-dot-dev]

If the request works in-browser and not with python I expect that they're user agent sniffing to decide if a request is "authorised". If so, that is a stupid thing to do.

[arjunior26]

You're right !

[bebo-dot-dev]

Maybe try something like this to confirm or otherwise:

joe@joes-HP-EliteBook-8470p ~ $ python
Python 3.9.2 (default, Apr  3 2021, 09:20:11) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requestheaders = {'User-Agent' : 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'}
>>> r = requests.get('http://your.m3u.url?username=X&password=Y', headers=requestheaders)
>>> print(r)
<Response [200]>

..specifying whatever user agent string your browser sends (that works as expected) in the requestheaders.

[arjunior26]

Well done ! It's work if we define User-Agent ! I've modifed get_m3u() function according to this and all is perfect now. Thank you very much 👍

[bebo-dot-dev]

ok that confirms that, personally I call this a dubious success. There's no good reason why an iptv service supplier should implement this sort of thing but I doubt that you would get a sensible response (or any response at all) if you were to ask them why they're doing this.

[arjunior26]

I doubt too :) Maybe you should define 'User-Agent' in your script too to bypass such stupid restriction.

[bebo-dot-dev]

That's a decent new feature suggestion -> a new generic "headers" argument to enable any number of custom header values (including the User-Agent and any others) to be included in the request. I might introduce that feature at some point.

[arjunior26]

Great ! Thank you for your impressive work and support !

[bebo-dot-dev]

Hi @arjunior26 just an fyi on the new feature being introduced.

Something like the following in json config / cli args should work to include a user-agent header value now:

json config:

"request_headers": [
        {"User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36"}
    ]

cli arg:

-rh="{\"request_headers\":[{\"User-Agent\":\"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36\"}]}"

[arjunior26]

Hi Joe, Thank you very much for update. I can't test for now but I will keep you inform.

[arjunior26]

It works as expected 👍 Great !