uurequests: Compatibility with CPython

[amotl]

Hi there,

when invoking the uurequests module on CPython, it will croak with

TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'str'

when starting to write to the socket stream:

s.write(b"%s /%s HTTP/1.0\r\n" % (method, path))

This has also been reported on https://github.com/adafruit/Adafruit_CircuitPython_Requests/issues/10 and was fixed by https://github.com/adafruit/Adafruit_CircuitPython_Requests/issues/11.

With kind regards, Andreas.

[pfalcon]

There's no problem here. Pycopy is a simpler implementation than Python, and due to this, also offers useful efficiency shortcuts. In particular, in Pycopy, strings explicitly use utf-8 encoding. This allows to intermix them with bytes in some operations, with well-defined behavior. Obviously, this feature should not be used when writing code portable across different Python implementation. In this regard, it's no different than any other feature pertinent only to a few particular Python implementations. For example, if you use f-strings, your code won't work with CPython 3.5, 3.4, 3.3, ..., 2.7, 2.6, 2.5, ..., Jython any version, IronPython any version, and many other Python implementations.

All this is not a problem for modules like uurequests, which are specifically designed to work with Pycopy, and thus use extensions provided by it for efficiency reasons. If you want to use similar functionality with CPython, that would be the original requests module.

[amotl]

Dear Paul,

we haven't meant to outline a problem here, but rather an improvement. We are conceiving a CPython-based test harness for our Terkin Datalogger, which might also spark your interest [1].

Thus, we have a strong need to invoke MicroPython programs on CPython. As others are also looking at MicroPython/CPython compatibility, we thought it would be a good idea to contribute such improvements to the community.

The full changes to make this work can be investigated at [2]. The outcome where this is used can be inspected at [3].

With kind regards, Andreas.

[1] https://github.com/hiveeyes/terkin-datalogger/tree/master/test [2] https://github.com/pfalcon/pycopy-lib/compare/master...daq-tools:improve-urequests [3] https://github.com/hiveeyes/terkin-datalogger/blob/93351bdf/test/test_wifi_http.py

[pfalcon]

Thanks for sharing info about your development, I appreciate that (both the info and development). I also fully agree that for some modules of Pycopy ecosystem, compatibility with CPython is useful and/or important. An obvious example of that is https://github.com/pfalcon/picotui/ , which is supported for both Pycopy and CPython (and by extension, with any Python implementation compatible with CPython).

But again, uurequest is a module where minimal size and efficiency prevails.

We are conceiving a CPython-based test harness for our Terkin Datalogger, which might also spark your interest [1].

That sounds interesting, but even more interesting would be implementing a Pycopy-based self-hosted solution. Because otherwise, there're very many solutions available for CPython, which you can use right away.

[amotl]

Even more interesting would be implementing a Pycopy-based self-hosted solution.

Sure. Do you know whether pytest will run on MicroPython/Pycopy for Unix? The choice of pytest for conceiving the test harness is important to us as it will offer so many bells and whistles and a rich ecosystem of addons.

[amotl]

Another note for others coming here: If you need a CPython-compatible umqtt module, see #50.

[pfalcon]

Do you know whether pytest will run on MicroPython/Pycopy for Unix? The choice of pytest for conceiving the test harness is important to us as it will offer so many bells and whistles and a rich ecosystem of addons.

Currently, pycopy-lib offers 2 test modules:

1. unittest of (C)Python stdlib. Of course, it's not complete. Initial implementation was largely contributed, and I slowly elaborate it for better CPython compatibility (mostly by running tests from CPython stdlib).
2. Recently, I added (very) basic implementation of nose (aka nosetests), which is my preferred testing module.

nose impl was done as part of an effort to port to Pycopy an application originally written for CPython. In this case, it was https://github.com/pfalcon/ScratchABlock , and it now passes its testsuite on Pycopy: https://github.com/pfalcon/ScratchABlock/runs/492397628 .

I doubt that any big CPython project can be just run under Pycopy. For ScratchABlock, I did a reasonable amount of adding missing functionality to Pycopy, and I'm happy to report that I was able to do that by putting my actions where my ideas/words are, and add most of the needed functionality in pure Python, on the pycopy-lib side, instead of patching C code. But that's because I'm the author of ScratchABlock, and I know that it's written utilizing Python as a "generic object-oriented language with functional programming elements", not as "bag of dirty tricks", which is sadly how a lot of Python software is written.

[amotl]

Dear Paul,

thanks for sharing these insights about the state of testing within Pycopy. I will try to stay on my path of using pytest as a testing framework and might report back how that will turn out in the future.

With kind regards, Andreas.