Micropython/patch1

[TheFern2]

Short description of change

Types of changes

• [X] New feature (non-breaking change which adds functionality)
• [X] I have updated the documentation accordingly.
• [X] I have read tests/README.md.
• [X] All new and existing tests passed.

What is the change?

Adding minor changes to accomodate micropython limited set of libraries.

What does it fix/add?

Support for micropython

Test Configuration

• PLC Model Softlogix 5800
• PLC Firmware v20.11
• pylogix version 0.8.7
• python version 2 and 3
• micropython 1.19
• OS type and version ubuntu 20.04

[TheFern2]

Adding this PR just to track comments:

As of now all single reads but bool work, and all arrays read work. Also need to be really wise about adding new modules since micropython is really limited. I had to remove strings import from tests.

TODO

• [ ] add unittest for regex, test good vs bad tag(s)
• [x] fix tests for micropython, some types like slices do not exist and are causing errors
• [x] debug why single bool is failing

For those early adopters that want to try this branch, install with mip [WARNING some things might not work yet]:

>> import mip
>> mip.install("github:dmroeder/pylogix", version="micropython/patch1")

It can also be installed with mpremote although I haven't tried it yet.

[TheFern2]

Tested over wireless with a esp32:

Three issues on hardware:

• couldn't figure out how to get mip to work, I think I need to add another json to make it a mpy library and add it to the micropython-lib repo, well this isn't really a big deal if you just transfer the library manually as mentioned below

  got this working, mip isn't available on esp32 stable release, so if you want to use mip use latest unstable bin

• py files are a bit too big for these devices with small constraints, I used the mpy-cross to reduce size
• vendors dictionary on lgx_device.py is causing a memory issue and stops library from loading into memory. As a test I left 100 vendors and library then loaded properly. Looks like mpy has memory constrains on large variables. So I think I can just break into multiple dicts and refactor where they are being used.

  Just added a if for this, micropython socket module is limited, so I am not loading this part of the code at all for discover.

For now I am just manually compiling py files into mpy with mpy-cross tool and flashing the library over to the device. This step is needed, copying py files over without compiling does not load the library on device.

This is not the repo just a test project in pycharm:

Device path /dev/ttyUSB0
Quit: Ctrl+] | Stop program: Ctrl+C | Reset: Ctrl+D
Type 'help()' (without the quotes) then press ENTER.

>>> from pylogix import PLC, __version__
>>> __version__
'0.8.7'
>>> comm = PLC('192.168.0.89',2)
>>> comm.Read('BaseBOOL')
Response(TagName=BaseBOOL, Value=False, Status=Success)

[TheFern2]

Okay will pick this up after xmas lol. For now don't use mip since I have to write a script to compile the py files. I thought mip did it on its own. I need to write a script to compile to mpy and modify package.json.

Also there's some work on Exceptions since that's also limited on mpy some exception classes do not exist on mpy.

[TheFern2]

mip install has been fixed, vendors dict also sorted to a on the fly lookup. Though I am not 100% sure if Discover works for micropython. If anyone tries it please provide some feedback on that.

>> import mip
>> mip.install("github:dmroeder/pylogix", version="micropython/patch1")

At first glance everything else seems to be working i.e. reads and writes single and arrays, but I have not ran tests on esp32 itself yet. I assume some tests will fail specially anything to do with large reads or writes due to the limited resources. Although this might work with other devices that have more resources like pi pico.

I've added a boot and main.py example in the examples directory. Keep in mind boot file executes on boot, and then main.py afterwards. If you connect with a repl at least through USB/TTL the main.py program stops executing. In the repl you can also do anything as you would with the regular python repl as long as the modules exists in the micropython port. If you want main.py to be executed again you need to do a hard reboot and don't use the repl.

As of now all tox unittests pass which for me include latest python2 and python3.8 and python3.10. All unix/port tests have passed which doesn't include discover for now. I am a little bit concerned about the portability of exceptions so need more testing on that.

This is a good tutorial on getting started with pycharm. https://medium.com/@andymule/micropython-in-pycharms-basic-setup-9169b497ec8a

[TheFern2]

Did a little more digging on the discover feature and it is unfortunately a no go for micropython https://github.com/micropython/micropython/issues/2691 due to licensing issues no one has been able to recreate setsockopt for broadcast from scratch into micropython mainline. I have left the vendor lookup in the code just in case someone likes to give it a go later on, but that part of the code will never reach since there is a guard in the Discover method.

I need to do a few more tests in catching some exceptions and it is close to being ready for merge, although I prefer to wait until we have at least a few users test it out. But the tests are passing 100% so I am happy on my end.

[TheFern2]

@drbitboy okay got some time today. I made a maintenance oss youtube video but still haven't edited it. Is just me reviewing the pr and talking about micropython at the end.

I had to do some minor fixes to get all tests passing, tox all python versions failed tests due to the bin not being in the setup.py.

Python 2 __file__ yields filename.pyc unlike Python 3. So I also accounted for that on the filename when loading the binary file.

Other minor fixes package.json and removing softlinks to old txt file.

Thanks for the contribution.

[TheFern2]

TODO

• [ ] merge new vendors
• [ ] test exceptions on python, and mpy

[TheFern2]

@dmroeder Aight picking right where I left, the exceptions handling that works for both python and mpy. I'll focus on the first one, and I comment as I fix one at a time.

What I am doing in the REPL, I wait a few minutes between the last Read calls. So obviously not what one would normally do in a normal application. Do you think this could be an issue or non issue, or maybe future improvement? Should this exception be caught by the end user and re instantiate the PLC object?

Python3:

Python 3.10.4 (main, Apr  8 2022, 17:35:13) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pylogix import PLC
>>> comm = PLC('192.168.0.89', 2)
>>> comm.Read('BaseBOOL')
Response(TagName=BaseBOOL, Value=True, Status=Success)
>>> comm.Read('BaseBOOL')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/fernandob/git/pylogix/pylogix/eip.py", line 116, in Read
    return self._read_tag(tag, count, datatype)
  File "/home/fernandob/git/pylogix/pylogix/eip.py", line 325, in _read_tag
    status, ret_data = self.conn.send(request)
  File "/home/fernandob/git/pylogix/pylogix/lgx_comm.py", line 69, in send
    return self._getBytes(eip_header, connected)
  File "/home/fernandob/git/pylogix/pylogix/lgx_comm.py", line 162, in _getBytes
    ret_data = self.recv_data()
  File "/home/fernandob/git/pylogix/pylogix/lgx_comm.py", line 188, in recv_data
    payload_len = unpack_from('<H', part, 2)[0]
struct.error: unpack_from requires a buffer of at least 4 bytes for unpacking 2 bytes at offset 2 (actual buffer size is 0)

In micropython something similar happens:

>>> comm.Read('BaseBOOL')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pylogix/eip.py", line 116, in Read
  File "pylogix/eip.py", line 325, in _read_tag
  File "pylogix/lgx_comm.py", line 69, in send
  File "pylogix/lgx_comm.py", line 164, in _getBytes
ValueError: buffer too small
>>> ret = comm.Read('BaseBOOL')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pylogix/eip.py", line 116, in Read
  File "pylogix/eip.py", line 325, in _read_tag
  File "pylogix/lgx_comm.py", line 69, in send
  File "pylogix/lgx_comm.py", line 161, in _getBytes
  File "pylogix/lgx_comm.py", line 187, in recv_data
ValueError: buffer too small

[TheFern2]

@dmroeder pinging in regards to previous comment. Got a few days to spare to work on this PR.

[dmroeder]

Shoot, I dropped the ball responding. The issue here is that it is not recovering when a certain amount of time passes between reads? I should see if I can fix that. I think it should recover automagically.

[dmroeder]

@TheFern2, I tried 0.8.7 and the latest, I don't get the same results. It doesn't crash, it just gives a connection lost, the very next read makes a new connection. Though, I have a CompactLogix and v30. I'll have to do some homework to figure out if this has something to do with the version or softlogix.

[TheFern2]

@TheFern2, I tried 0.8.7 and the latest, I don't get the same results. It doesn't crash, it just gives a connection lost, the very next read makes a new connection. Though, I have a CompactLogix and v30. I'll have to do some homework to figure out if this has something to do with the version or softlogix.

Ok, then I'll move on to the next exception.

[TheFern2]

@dmroeder @drbitboy she's ready for merging! Please review PR, I know is been a while and is a big PR but the core functionality hasn't been modified that much. Only a few exceptions where changed to accommodate mpy, others might need more as we go along. And a few things changed where sugar functions weren't available in mpy, where we had to get creative.

[TheFern2]

I need to create some scripts, one to run testing for both tox and mpy. And another script to build everything mpy needs, as of now it is a bit of manual process to edit the package.json and up the version. I should pull version from pylogix.__version__ and also auto-generate the package.json. The less stuff is manually edited the better for future maintenance. This is complete. I leave the testing commands up to the user, unless you have no issues me adding a sh script.