Open two9ers opened 4 years ago
jorticus
commented
3 years ago Yeah I think this is probably related to issue #22.
Can you enable full debug logging? Just add the following to the top of readout.py:
import logging
log = logging.getLogger('mate')
log.setLevel(logging.DEBUG)
log.addHandler(logging.StreamHandler())I expect you'll see something similar where there are extra 00's at the start of the packet.
jorticus
commented
3 years ago Could you try the latest changes in branch issue-22? I've added a more thorough workaround for the issue I believe you are encountering. Please also enable the debug logs in addition to this.
two9ers
commented
3 years ago The latest from branch issue-22:
~/pymate-issue-22 $ python readout.py
MATE emulator (MX)
Send [Port0, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
RETRY
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
RETRY
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
Traceback (most recent call last):
File "readout.py", line 17, in <module>
port = bus.find_device(MateNET.DEVICE_MX)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 209, in find_device
dtype = self.query(0x00, port=i)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 134, in query
resp = self.send(MateNET.TYPE_QUERY, addr=reg, param=param, port=port, response_len=MateNET.QueryResponse.size)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 87, in send
rxbuf = self.port.recv(response_len)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet_ser.py", line 171, in recv
return MateNETSerial._parse_packet(rawdata, expected_len)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet_ser.py", line 119, in _parse_packet
if self.TRIM_LARGE_PACKETS:
NameError: global name 'self' is not defined
~/pymate-issue-22 $Ooops, just pushed a fix, please pull & try again. I'm not able to test the code right now (don't have the hardware set up)
two9ers
commented
3 years ago New error with the latest:
pi@datapi:~/pymate-issue-22 $ python readout.py
MATE emulator (MX)
Send [Port0, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
RETRY
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
RETRY
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
Traceback (most recent call last):
File "readout.py", line 17, in <module>
port = bus.find_device(MateNET.DEVICE_MX)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 209, in find_device
dtype = self.query(0x00, port=i)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 134, in query
resp = self.send(MateNET.TYPE_QUERY, addr=reg, param=param, port=port, response_len=MateNET.QueryResponse.size)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 87, in send
rxbuf = self.port.recv(response_len)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet_ser.py", line 170, in recv
return MateNETSerial._parse_packet(rawdata, expected_len)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet_ser.py", line 112, in _parse_packet
raise RuntimeError("Error receiving mate packet - Received packet too small (%d bytes)" % (len(data)))
RuntimeError: Error receiving mate packet - Received packet too small (2 bytes)Thank you, getting closer... I forgot to account for additional bytes in the response packet. Please try the latest changes (0a6be1c).
two9ers
commented
3 years ago Latest results. I turned off debugging as scan.py completes successfuly.
pi@datapi:~/pymate-issue-22 $ python scan.py
MATE Bus Scan
Port0: Hub (Rev: 001.000.001)
Port1: FX (Rev: 000.000.000)
Port2: FX (Rev: 000.000.000)
Port3: MX (Rev: 11683.045.000)
Port4: MX (Rev: 11691.045.000)Error from readout.py, Not sure why it goes past port 4 as I have a 4 port hub and in scan.py I have changed: "if dtype == MateNET.DEVICE_HUB: for i in range(1,5):"
pi@datapi:~/pymate-issue-22 $ python readout.py
MATE emulator (MX)
Send [Port0, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 00
TX: [0] 02 00 00 00 00 00 02
RX: 00 00 01 00 01
Send [Port1, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 01
TX: [0] 02 00 00 00 00 00 03
RX: 02 00 02 00 04
Send [Port2, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 02
TX: [0] 02 00 00 00 00 00 04
RX: 02 00 02 00 04
Send [Port3, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 03
TX: [0] 02 00 00 00 00 00 05
RX: 00 00 02 11 03 00 16
Send [Port4, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 04
TX: [0] 02 00 00 00 00 00 06
RX: 00 00 02 11 03 00 16
Send [Port5, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 05
TX: [0] 02 00 00 00 00 00 07
RETRY
TX: [1] 05
TX: [0] 02 00 00 00 00 00 07
RETRY
TX: [1] 05
TX: [0] 02 00 00 00 00 00 07
RETRY
Send [Port6, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 06
TX: [0] 02 00 00 00 00 00 08
RETRY
TX: [1] 06
TX: [0] 02 00 00 00 00 00 08
RETRY
TX: [1] 06
TX: [0] 02 00 00 00 00 00 08
RETRY
Send [Port7, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 07
TX: [0] 02 00 00 00 00 00 09
RETRY
TX: [1] 07
TX: [0] 02 00 00 00 00 00 09
RETRY
TX: [1] 07
TX: [0] 02 00 00 00 00 00 09
RETRY
Send [Port8, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 08
TX: [0] 02 00 00 00 00 00 0a
RETRY
TX: [1] 08
TX: [0] 02 00 00 00 00 00 0a
RETRY
TX: [1] 08
TX: [0] 02 00 00 00 00 00 0a
RETRY
Send [Port9, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 09
TX: [0] 02 00 00 00 00 00 0b
RETRY
TX: [1] 09
TX: [0] 02 00 00 00 00 00 0b
RETRY
TX: [1] 09
TX: [0] 02 00 00 00 00 00 0b
RETRY
Traceback (most recent call last):
File "readout.py", line 17, in <module>
port = bus.find_device(MateNET.DEVICE_MX)
File "/home/pi/pymate-issue-22/pymate/matenet/matenet.py", line 219, in find_device
raise KeyError('%s device not found' % MateNET.DEVICE_TYPES[device_type])
KeyError: 'MX device not found'If I specify port 4: "# Find an MX device on the bus
port = 4"
pi@datapi:~/pymate-issue-22 $ python readout.py
MATE emulator (MX)
Send [Port4, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 04
TX: [0] 02 00 00 00 00 00 06
RX: 00 00 02 11 03 00 16
Send [Port4, Type=0x02, Addr=0x0002, Param=0x0000]
TX: [1] 04
TX: [0] 02 00 02 00 00 00 08
RX: 00 00 02 ad ab 01 5a
Send [Port4, Type=0x02, Addr=0x0003, Param=0x0000]
TX: [1] 04
TX: [0] 02 00 03 00 00 00 09
RX: 00 00 02 00 ad 00 af
Send [Port4, Type=0x02, Addr=0x0004, Param=0x0000]
TX: [1] 04
TX: [0] 02 00 04 00 00 00 0a
RX: 00 00 02 00 00 00 02
Revision: 44459.173.000
Getting log page... (day:-1)
Traceback (most recent call last):
File "readout.py", line 30, in <module>
logpage = mate.get_logpage(-1)
File "/home/pi/pymate-issue-22/pymate/matenet/mx.py", line 177, in get_logpage
resp = self.send(MateNET.TYPE_LOG, addr=0, param=-day, response_len=MXLogPagePacket.size)
TypeError: send() got an unexpected keyword argument 'response_len'For the first error - readout.py calls find_device() (see matenet.py L197) which scans each port 0-9, then returns when it finds the first connected MX. It seems I missed a fix for a bug that was found a while ago, so I've pushed that change to the branch. (Some MX devices have something in the upper byte for the scan result, which throws off the ID detection)
If you know your network won't change, you can hard-code the ports for each device and skip the call to find_device(). You'll probably need to do that anyway since you have two MX units.
The second error was a simple python mistake, and I've pushed a new change to the branch.
Thanks!
two9ers
commented
3 years ago I really appreciate your help on this, and wish I could set up a port forward so you could ssh into the datapi, but I'm off-grid and working with phone data. Can't wait for Starlink!
The latest. I have skipped find_device(), specified port = 3, and requested logpage = mate.get_logpage(-2). The returned logpage is not accurate. And then the get_status error...
pi@datapi:~/pymate-issue-22 $ python readout.py
MATE emulator (MX)
Send [Port3, Type=0x02, Addr=0x0000, Param=0x0000]
TX: [1] 03
TX: [0] 02 00 00 00 00 00 05
RX: 00 00 02 11 03 00 16
Send [Port3, Type=0x02, Addr=0x0002, Param=0x0000]
TX: [1] 03
TX: [0] 02 00 02 00 00 00 07
RX: 00 00 02 0c 29 00 37
Send [Port3, Type=0x02, Addr=0x0003, Param=0x0000]
TX: [1] 03
TX: [0] 02 00 03 00 00 00 08
RX: 00 00 02 00 0c 00 0e
Send [Port3, Type=0x02, Addr=0x0004, Param=0x0000]
TX: [1] 03
TX: [0] 02 00 04 00 00 00 09
RX: 00 00 02 00 00 00 02
Revision: 3113.012.000
Getting log page... (day:-2)
Send [Port3, Type=0x16, Addr=0x0000, Param=0x0002]
TX: [1] 03
TX: [0] 16 00 00 00 02 00 1b
RX: 00 00 2a 00 99 00 58 00 00 00 00 00 00 00 00 01 1b
MX Log Page:
Day: -0
0Ah 244.8kWh
0Vpk 51.2Apk 0.000kWpk
Min: 0.0V Max: 1.0V
Absorb: 88min Float: 0min
Status:
Send [Port3, Type=0x04, Addr=0x0001, Param=0x0000]
TX: [1] 03
TX: [0] 04 00 01 00 00 00 08
RX: 00 03 00 80 80 00 8f 00 00 00 00 00 f8 02 da 03 66
Traceback (most recent call last):
File "readout.py", line 35, in <module>
status = mate.get_status()
File "/home/pi/pymate-issue-22/pymate/matenet/mx.py", line 169, in get_status
return MXStatusPacket.from_buffer(resp)
File "/home/pi/pymate-issue-22/pymate/matenet/mx.py", line 51, in from_buffer
values = cls.fmt.unpack(data)
struct.error: unpack requires a string argument of length 13Hmm if your logpage is not correct, then maybe we're on the wrong path. It might just be that your serial link is not 100% reliable or can't handle all the devices connected to the hub (I only have 1 MX & FX device to test with).
It will be tricky to debug further without an oscilloscope or logic analyzer to prove that pymate is receiving the data correctly. It could be as simple as the opto-isolator circuit having wrong resistor values (corrupting incoming bytes).
Do you get a correct logpage if you connect directly to one of your MX units?
I'm getting the following error when running readout.py Apologies for formatting issues, having a bit of trouble with that also...
MATE emulator (MX) Traceback (most recent call last) File "readout.py", line 13, in
port = bus.find_device(MateNET.DEVICE_MX)
File "/home/pi/pymate-master/pymate/matenet/matenet.py", line 213, in find_device
raise KeyError('%s device not found' % MateNET.DEVICE_TYPES[device_type])
KeyError: 'MX device not found'
I added some print commands and set port 4:
bus = MateNET(SERIAL_PORT) print "bus", bus
port = bus.find_device(MateNET.DEVICE_MX)
port = 4 print "port", port
mate = MateMXDevice(bus, port) print "mate", mate
mate.scan()
print "Revision:", mate.revision
New error shows up:
MATE emulator (MX) bus <pymate.matenet.matenet.MateNET object at 0xb669e470> port 4 mate <pymate.matenet.mx.MateMXDevice object at 0xb669e830> Revision: 44451.173.000 Getting log page... (day:-1) MX Log Page: Day: -0 0Ah 361.6kWh 1Vpk 0.1Apk 0.000kWpk Min: 0.0V Max: 1.5V Absorb: 0min Float: 0min
Status: Traceback (most recent call last): File "readout.py", line 34, in
status = mate.get_status()
File "/home/pi/pymate-master/pymate/matenet/mx.py", line 167, in get_status
return MXStatusPacket.from_buffer(resp)
File "/home/pi/pymate-master/pymate/matenet/mx.py", line 50, in from_buffer
values = cls.fmt.unpack(data)
struct.error: unpack requires a string argument of length 13