[FEATURE REQUEST] implement bitwise operators

[Derfies]

I am attempting to read 8bit colour data from a file on the pipico and convert it to separate r, g, b values. There is an example here which goes into greater detail on how the math should work.

Specifically from this page the methods bitwise_and (& operator) and right_shift (>> operator).

I have implemented this as a loop however I'm hoping to save ms if these are implemented as vectorised functions.

Many thanks in advance!

[v923z]

@Derfies https://github.com/v923z/micropython-ulab/pull/616 implements the and, or, xor, left_shift, and right_shift operators.

Do you think you could add some meaningful tests? The test files are already in https://github.com/v923z/micropython-ulab/tree/bitwise/tests/2d/numpy, you just have to extend them, and then compare the results to numpy's.

[Derfies]

Seems only fair :) Just looking at the testing protocol - I'm in windows so will running the .sh to build micropython work for me?

[v923z]

I don't have experience with windows, but have you looked at https://github.com/micropython/micropython/tree/master/ports/windows?

[Derfies]

Sorry, been held up. Will try to get to this on the weekend!

[Derfies]

Hi @v923z , I've successfully built micropython for windows using Visual Studio Community 2022. Is it just a matter of building ulab for windows and putting the resulting object into the same directory in order to run tests?

[v923z]

Yes, pretty much. You would do something like this:

./micropython ./tests/numpy/2d/some_test.py > ./test/numpy/2d/some_test.py.exp

and then you would inspect the contents of some_test.py.exp. If you are satisfied, then you add your test file and the expected file to the repository with git, and push to your remote.

[Derfies]

Hi @v923z - sorry I know I'm being a pain but I'm quite green with some of this stuff. A few more questions and hopefully I can contribute something of value:

• Is the build process meant to bundle ulab alongside micropython, or are these built separately?
• Building micropython in windows produces an .exe as you might expect - is the intent that the tests are run on windows or that I need to build everything to target a specific microcontroller (I'm running a Pi Pico) and perform the tests there?

Thank you!!

[v923z]

Hi @v923z - sorry I know I'm being a pain but I'm quite green with some of this stuff.

Oh, don't worry! We were all green at one point. It will be easier next time. I'm happy to help, if I can.

* Is the build process meant to bundle ulab alongside micropython, or are these built separately?

You'll get one piece of firmware, where ulab is part of the binary code. So, during the build, you should see something like this:

...
CC /home/v923z/sandbox/ulab/ulab/code/numpy/stats.c
CC /home/v923z/sandbox/ulab/ulab/code/numpy/transform.c
CC /home/v923z/sandbox/ulab/ulab/code/numpy/vector.c
CC /home/v923z/sandbox/ulab/ulab/code/numpy/numpy.c
CC /home/v923z/sandbox/ulab/ulab/code/scipy/scipy.c
CC /home/v923z/sandbox/ulab/ulab/code/user/user.c
CC /home/v923z/sandbox/ulab/ulab/code/utils/utils.c
CC /home/v923z/sandbox/ulab/ulab/code/ulab.c
CC ../../py/modsys.c
CC ../../extmod/moduplatform.c

Those are the ulab fragments being compiled.

* Building micropython in windows produces an .exe as you might expect - is the intent that the tests are run on windows or that I need to build everything to target a specific microcontroller (I'm running a Pi Pico) and perform the tests there?

In principle, you could test it on the microcontroller, but then copying the results back would not be particularly convenient. When you run the windows executable, you get a micropython prompt, exactly as on a microcontroller, thus, you can just do

from ulab import numpy as np
a = np.array([1, 2, 3, 4])
...

But that is probably not what you would do. Instead, you can just pass a script to the executable, exactly as with python.

micropython.exe some_test_script.py

or

micropython.exe some_test_script.py > some_test_script.py.exp

That produces the expected file that should eventually end up in the same folder as some_test_script.py.

[Derfies]

Oh, don't worry! We were all green at one point. It will be easier next time. I'm happy to help, if I can.

Thank you!!

You'll get one piece of firmware, where ulab is part of the binary code.

Ah ok so it's a single artifact that is produced by the build process, and that includes ulab and micropython? If that's the case then presumably I need to add some sort of link to my micropython visual studio project file / build script so visual studio builds the two together..?

[v923z]

Ah ok so it's a single artifact that is produced by the build process, and that includes ulab and micropython?

Yes. You can actually see that here:

https://github.com/v923z/micropython-builder/releases

If that's the case then presumably I need to add some sort of link to my micropython visual studio project file / build script so visual studio builds the two together..?

I don't know exactly, how visual studio handles that. But ulab is just a simple user project, so if you can compile the examples here, then you should be able to compile ulab, too. https://docs.micropython.org/en/v1.18/micropython-docs.pdf

[Derfies]

Ok! Managed to build on Windows using Ubuntu on wsl. Should be able to get these tests running now.

[v923z]

Implemented via https://github.com/v923z/micropython-ulab/pull/616, https://github.com/v923z/micropython-ulab/pull/628.

[s-ol]

Any particular reason these aren't exposed through operator overloading? Without that, there also is no support for in-place operation.

It would also be great to have the modulo operator (or function), at least for integer types.

I didn't realize the functions were implemented since I only looked at the circuitpython-ulab function listing and the micropython-ulab operator code, so I implemented %, <<, >> and their inplace versions as operators here:

https://github.com/v923z/micropython-ulab/compare/master...s-ol:micropython-ulab:mpy-add-bitops?expand=1

The non-inplace code should probably be replaced with calls to the implementation in code/numpy/bitwise.*? With some guidance on what to salvage and what to adapt I could prepare a PR for these features.

[v923z]

If there is need for these operators, I would be happy to add them.

This is a large number of features, I would suggest to split it into smaller chunks, namely, to implement each operator in its own PR. Otherwise, it'll be hard to keep track of the modifications.