urequests: TypeError: object with buffer protocol required

[GM-Script-Writer-62850]

r=urequest.post("http://www.example.com",headers={"Content-Length":19},data='{"hello":"world"}')
print(r.status_code,r.content)

Seems that that 19 is the issue and it needs to be a string, should the user of the lib be expected to know this or should this:

        for k in headers:
            s.write(k)
            s.write(b": ")
            s.write(headers[k])
            s.write(b"\r\n")

be changed to this:

        for k in headers:
            s.write(k)
            s.write(b": ")
            s.write(str(headers[k]))
            s.write(b"\r\n")

on a side note, why does this not happen with header={} https://forums.raspberrypi.com/viewtopic.php?t=341355

[jimmo]

I agree that's confusing.

However, the "Content-Length" header is set automatically (a few lines below the bit you linked to), there's no need to include it in the headers dict.

on a side note, why does this not happen with header={} https://forums.raspberrypi.com/viewtopic.php?t=341355

I'm not sure what you're asking here sorry...

[GM-Script-Writer-62850]

i am sure there are other headers that will have a numeric value, that was the 1st header i though about

on the side note, notice how headers is defined as {} here Why does that work as expected where as when i did that my variable eg json was still populated with old data on the next function call, my solution was to use a boolean and if it is false change that value to {} inside the function

[jimmo]

@GM-Script-Writer-62850 I think this is what you're referring to https://stackoverflow.com/questions/1132941/least-astonishment-and-the-mutable-default-argument

[GM-Script-Writer-62850]

yes, i was wondering why this is not the behavior for the header parameter in the request function, i saw that and was going to report it as a bug, but the issue does happen somehow

[jimmo]

@GM-Script-Writer-62850 Oh I see. In the specific case where it's called with default headers and auth is set, then it will modify the default headers dict. This was introduced in https://github.com/micropython/micropython-lib/commit/e7e8eff86b219bec2d6e19baeb8f0d85f7b77e47

This should be done like Content-Length etc and handled later with direct writes to the socket rather than modifying the headers dict.

[GM-Script-Writer-62850]

i was wondering how/why does header always starts as a empty dict in this lib, but not when i do it if i put print(header) it is either empty or has what i just defined in my call to get/post

[jimmo]

i was wondering how/why does header always starts as a empty dict in this lib, but not when i do it if i put print(header) it is either empty or has what i just defined in my call to get/post

@GM-Script-Writer-62850 Could you share a full example of what you mean? I'm not quite sure what you're asking or what you're saying isn't working? Or are you asking why something does work when you think it shouldn't?

I assume by header you mean headers ?

[GM-Script-Writer-62850]

yes, i have a cat on my lap and should have zoomed in to see the s

[GM-Script-Writer-62850]

if i call post("http://www.example.com",data,headers={"DNT":1}) then if i call post("http://www.example.com",data) when post calls requests what do you think headers is? {"DNT":1} or {} the answer is {}, but when I make a function with a variable set as {} (eg def test(var1,var2={})) it's content persist between calls (var2), for what reason is the request function able to get around that annoying behavior with its headers variable

[jimmo]

@GM-Script-Writer-62850 you might need to post more of your test function.

def test(var1, var2={}):
    print(var1, var2)

test('a')
test('b', {'b': 2})
test('c')

prints (as expected)

a {}
b {'b': 2}
c {}

(Other than the specific case of when auth is enabled that I highlighted above, this is how urequests works -- it never modifies the headers dict).

Maybe you are doing this (i.e. modifying the argument):

def test(var1, var2={}):
    var2[var1] = True
    print(var2)

test('a')
test('b', {'b': 2})
test('c')

{'a': True}
{'b': True}
{'a': True, 'c': True}

This is explained in detail in the stackoverflow link above.

[sosi-deadeye]

It's a well-known problem, and there is PEP671.

In short terms:

def foo(data={}):
    ...

should be ...

def foo(data=None):
    data = data or {}

and maybe in near future

def foo(data=>{}):
    ...

With the cost of 2 pointers.

[GM-Script-Writer-62850]

Thanks jimmo, that behavior makes it even more confusing (only keeping data if it is set in the function but dropping it when it is supplied)

sorry for skimming the stackoverflow stuff and being too blind to see it, how i know why headers is not affected

this side question when on way longer than i expected, maybe this report should be purged of the side question