Connecting in W10 --> freezes

[Eheran1]

My Riden RD6006P is COM4 on my windows machine and works with the original riden software. If I try to run your (and Baldanos) library I get nothing, it just freezes. Simple example: The first print happens, the second doesnt:

from riden import Riden
print("connecting...", flush=True)
r = Riden(port="COM4", baudrate=115200, address=1)
print("connected", flush=True)

Any idea how I can connect to the Riden in Windows?

[ShayBox]

The freezing is normally caused by wrong port name or mismatched baudrate, however it should run fine on Windows, pyserial supports COM0, COM1, etc port names.
The python -m serial.tools.list_ports command can be used to list serial ports.
Try running the Riden libraries info menu via cli riden --port <port> --baudrate <baud> and see if that works.
You could also try connecting with pyserial itself

from serial import Serial
Serial(port, baudrate, timeout=5)

That may produce a more verbose error.
It may be that you need to run as Administrator, I usually keep UAC disabled on Windows, so I'm not sure if that's required.

[Eheran1]

python -m serial.tools.list_ports

COM4
1 ports found

r=Serial("COM4", 115200, timeout=5) print(r)

Serial<id=0x21fe8cbbe50, open=True>(port='COM4', baudrate=115200, bytesize=8, parity='N', stopbits=1, timeout=5, xonxoff=False, rtscts=False, dsrdtr=False)

riden --port COM4 --baudrate 115200 Freezes. riden --port "COM4" --baudrate 115200 Freezes too. After a bunch of seconds this was printed a bunch of times:

Traceback (most recent call last):
  File "c:\programdata\anaconda3\lib\site-packages\riden\riden.py", line 59, in read
    response = self.master.execute(
  File "c:\programdata\anaconda3\lib\site-packages\modbus_tk\utils.py", line 39, in new
    raise excpt
  File "c:\programdata\anaconda3\lib\site-packages\modbus_tk\utils.py", line 37, in new
    ret = fcn(*args, **kwargs)
  File "c:\programdata\anaconda3\lib\site-packages\modbus_tk\modbus.py", line 306, in execute
    response_pdu = query.parse_response(response)
  File "c:\programdata\anaconda3\lib\site-packages\modbus_tk\modbus_rtu.py", line 46, in parse_response
    raise ModbusInvalidResponseError("Response length is invalid {0}".format(len(response)))
modbus_tk.exceptions.ModbusInvalidResponseError: Response length is invalid 0

During handling of the above exception, another exception occurred:

And at some point python aborted with "maximum recursion depth exceeded while calling a Python object". Admin rights are provided. What else should I try to narrow the issue down?

[ShayBox]

Hmm, this is very strange, what version of windows are you using, what terminal program/shell are you using, and do you have the serial drivers installed https://www.wch-ic.com/downloads/CH341SER_EXE.html, maybe I can replicate this problem in a matching vm, My Windows 11, Windows Terminal, and Powershell vm runs it just fine (This is RidenGUI, using the Riden library)

PS C:\> riden --port COM3 --baudrate 250000
ID      : 60181
SN      : 00011608
FW      : 136
TYPE    : RD6018
INT_C   : 42
INT_F   : 107
V_SET   : 13.0
I_SET   : 4.0
V_OUT   : 13.01
I_OUT   : 3.24
P_OUT   : 42.29
V_IN    : 68.01
KEYPAD  : False
OVP_OCP : None
CV_CC   : CV
OUTPUT  : True
PRESET  : 0
BAT_MODE: True
V_BAT   : 13.01
EXT_C   : -89
EXT_F   : -128
AH      : 7.278
WH      : 91.699
DATETIME: 2020-01-02 05:49:08
TAKE_OK : True
TAKE_OUT: False
BOOT_POW: False
BUZZ    : True
LOGO    : False
LANG    : 0
LIGHT   : 5

[Eheran1]

Since that link didnt work I went here (http, not https) and installation failed at first. I then connected the powersupply and did it again, this time it was successful. So far I can not see a difference.

OS: W10 21H2 build 19044.1503 terminal program/shell: Spyder, elevated CMD and now also PS like you did, which also freezes and then stops with the error above. PS C:\> riden --port COM4 --baudrate 115200

Im actually using Anaconda, so I also started a CMD from there, but its the same. Can I manually do some communication to narrow down what the issue is?

[Eheran1]

Here is a Serial dump.zip I made with RidenPowerSupply1.0.0.12.exe. And here is a Python serial dump.zip I made with your library. "IRP_MJ_READ" always causes a timeout, this seems to be where it goes wrong.

[ShayBox]

As far as I can tell that's just the read on windows, check your ports again, you would have gotten a new com port after installing the driver if you didn't already have it.
I'm going to try to replicate all of that stuff and see if I can get the same behavior.

[Eheran1]

The COM-port changes if I use a different USB-port, the installation of the driver didnt. What do you mean "read on windows"? I installed a program to sniff the serial communication to (maybe) help debug the problem. I tested with a different system, also W10 and also anaconda, otherwise different. Below you can see all I did to recreate the problem. I didnt install the driver due to lack of admin rights. Since RidenPowerSupply1.0.0.12.exe works out of the box with no installation im not sure if the (other) driver is needed. I know some people in the EEVblog needed the driver for things to work first, but since the issue happens regardless of this driver installation and looks the same on my 2 systems I dont really think its the issue.

(base) C:\Users\me>pip install git+https://github.com/ShayBox/Riden.git
Collecting git+https://github.com/ShayBox/Riden.git
  Cloning https://github.com/ShayBox/Riden.git to c:\users\me\appdata\local\temp\pip-req-build-sgn0k672
  Running command git clone -q https://github.com/ShayBox/Riden.git 'C:\Users\me\AppData\Local\Temp\pip-req-build-sgn0k672'
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: click<9.0.0,>=8.0.3 in c:\users\me\anaconda3\lib\site-packages (from riden==1.1.0) (8.0.3)
Collecting pyserial<4.0,>=3.5
  Downloading pyserial-3.5-py2.py3-none-any.whl (90 kB)
     |████████████████████████████████| 90 kB 1.2 MB/s
Collecting modbus_tk<2.0.0,>=1.1.2
  Downloading modbus_tk-1.1.2.tar.gz (33 kB)
Requirement already satisfied: colorama in c:\users\me\anaconda3\lib\site-packages (from click<9.0.0,>=8.0.3->riden==1.1.0) (0.4.4)
Building wheels for collected packages: riden, modbus-tk
  Building wheel for riden (PEP 517) ... done
  Created wheel for riden: filename=Riden-1.1.0-py3-none-any.whl size=7752 sha256=a85afb35d8170ea9b58f82c5734f9eba93865fc88cb379dbff59f85a20ae2f53
  Stored in directory: C:\Users\me\AppData\Local\Temp\pip-ephem-wheel-cache-ge8y1ekc\wheels\8e\20\af\061d99204fa852e09ef5cc9bac66007d544616fb81a944cec6
  Building wheel for modbus-tk (setup.py) ... done
  Created wheel for modbus-tk: filename=modbus_tk-1.1.2-py3-none-any.whl size=38391 sha256=6ed210d13931f55dbc3098fa5f731055d830516a62b8f833eaddb0d1d777b593
  Stored in directory: c:\users\me\appdata\local\pip\cache\wheels\a5\43\4e\fee94581ef58371870cacd8835bbcbd53bb95807b736e78fe5
Successfully built riden modbus-tk
Installing collected packages: pyserial, modbus-tk, riden
Successfully installed modbus-tk-1.1.2 pyserial-3.5 riden-1.1.0

(base) C:\Users\me>riden --port COM6 --baudrate 115200
[freezes like on the other system, throws the same error as above]

[ShayBox]

I mean that looks like the windows api call for reading serial (and possibly filesystem?)
Sometimes I've found I need the driver, sometimes not, I created a Windows 10 19044.1503 vm and installed python, using powershell, and it works, no driver needed, I really don't know why it doesn't work for you... I've even tested using anaconda3 powershell prompt
It could be something related to the RD6006P, I only have an RD6018, maybe some of the registers changed or something

EDIT: You could try using Baldanos's rd6006 library, it may or may not work, it uses a different modbus library, that would at-least narrow down if it's modbus_tk somehow

[zalexua]

It could be something related to the RD6006P,

I don't think so. When I developed my changes 4 months ago, I used RD6006P model. So it worked at the time. p.s. I'm on Linux, did not try to run it on Windows.

[Eheran1]

I used HKJs TestController and after changing the RidenRD60xx.txt to include the RD6006P it works. So no issue with driver/unit/... it must be somewhere in my python setup or something blocking the communication.

[Eheran1]

This simple example is where the issue has its roots and results in: NoResponseError: No communication with the instrument (no answer)

import minimalmodbus
instr = minimalmodbus.Instrument('COM6', 1)
instr.debug = True
#minimalmodbus.TIMEOUT = 0.5
instr.serial.baudrate = 115200
a = instr.read_register(0)

However, as I said, communication with RidenPowerSupply1.0.0.12.exe as well as HKJs TestController works flawless. So to dig deeper, I used the same commands as TestController within python:

import serial

ser = serial.Serial(port='COM6', baudrate=115200, timeout=0.5)
command = b'\x01\x03\x00\x00\x00\x01\x84\x0A'
ser.write(command)
s = ser.read(10).hex()
print(s) --output--> 01 03 02 ea a1 36 9c

command = b'\x01\x03\x00\x01\x00\x02\x95\xCB'
ser.write(command)
s = ser.read(10).hex()
print(s) --output--> 01 03 04 00 00 07 70 f9 e7

command = b'\x01\x03\x00\x03\x00\x01\x74\x0A'
ser.write(command)
s = ser.read(10).hex()
print(s) --output--> 01 03 02 00 8c b9 e1

command = b'\x01\x03\x00\x80\x00\x01\x85\xE2'
ser.write(command)
s = ser.read(10).hex()
print(s) --output--> 01 03 02 00 00 b8 44

command = b'\x01\x10\x00\x12\x00\x02\x04\x00\x00\x00\x01\xB2\xBA'
ser.write(command)
s = ser.read(10).hex()
print(s) --output--> 01 10 00 12 00 02 e1 cd

The RD6006P responds as it should, tho I cant decode that data as of now.

[ShayBox]

Is RidenPowerSupply1.0.0.12.exe the software from riden?
That is very, very strange, do either of you @Eheran1 @zalexua use UniSoft's custom firmware?
If reading register 0 is causing this it may be an official firmware change, does TestController read register 0?

[Eheran1]

Yes, its their software. I dont use any custom firmware, I just got the unit a few days ago (firmware is 1.40). AFAIK TestController reads the registers like this: command = b'\x01\x03\x00***x00***\x00\x01\x84\x0A' Note the 01, 03 and 80 at the same position from the other commands indicating the registers. But it doesnt just send the register, there is much more going on. I cant check the raw data that minimalmodbus is sending, so I dont know how much more it sends other than just the register. The documentation says this: read_register [...] Read an integer from one 16-bit register in the slave, possibly scaling it.

[ShayBox]

Okay, I flashed my RD6018 with stock firmware, and yeah, it doesn't work anymore, I'm guessing riden changed something in newer firmware that broke modbus communication, maybe they changed communication protocol completely, or some registers...
I would personally suggest using UniSoft's custom firmware, it adds many new features and modbus still works
https://drive.google.com/file/d/1FKAXFBIbRVujsal-6V2Ta0ogtcvQAIPd/view

EDIT: Also the official riden software apparently installs the serial driver itself

[Eheran1]

Holy cow what a ride... but still, TestController is working, why doesnt modbus work? Im at a point where I think this is modbus RTU but am stuck as to how I can get anywhere. Nothing works regardless of what I try.

[ShayBox]

I'm not sure, TestController may not be using a modbus library, but raw serial like you did, or maybe it doesn't interact with the register(s) that have been changed/removed
These units do use Modbus RTU, but I tried minimalmodbus and modbus_tk, both don't work with stock, this may be a bug or an intentional change by Ruiden
This library is completely based off the reverse engineering work Baldanos did for their library, so I wouldn't know how to do what they did to figure out all the registers

EDIT: In fact it appears using my library with stock firmware freezes my unit and needs to be powered off to fix it, this issue was also fixed in custom firmware with the latest "SysFailureRst" feature
My unit was stuck in an invalid interface mode from the custom firmware, changing it to usb stopped the freezing, still doesn't work though

[Eheran1]

Flashed firmware to RD60065_V1.40.1g and it works, see below. 2 things I noticed: There is no way to disconnect in code, so every time I run a program for testing I need to disconnect and reconnect the device so the com-port is not blocked anymore. Is there a way to avoid this eg. close the port at start or end of my code? Second thing: I can push out commands too fast. Seems like somewhere between 100ms and 200ms is the maximum speed for a loop like set_v_set -> get_v_set -> get_v_out. Is it possible that a appropriate delay can be added in your library directly? If its too fast it sesults in: raise ModbusInvalidResponseError("Response length is invalid {0}".format(len(response))) Maybe you can recover from that in the library so the program is not terminated but instead resumes if/and/or the automatic delay cant be added?

C:\Users\me>riden --port COM6 --baudrate 115200

ID      : 60065
SN      : 0000xxxx
FW      : 140
TYPE    : RD6006P
INT_C   : 25
INT_F   : 77
V_SET   : 5.0
I_SET   : 6.1
V_OUT   : 0.0
I_OUT   : 0.0
P_OUT   : 0.0
V_IN    : 20.11
KEYPAD  : False
OVP_OCP : None
CV_CC   : CV
OUTPUT  : False
PRESET  : 0
BAT_MODE: False
V_BAT   : 0.001
EXT_C   : -89
EXT_F   : -128
AH      : 0.0
WH      : 0.0
DATETIME: 2020-01-01 12:07:55
TAKE_OK : True
TAKE_OUT: False
BOOT_POW: False
BUZZ    : False
LOGO    : True
LANG    : 0
LIGHT   : 4

Hm, after some more playing around, this even happens if there is a 2 second delay after doing 20...40 iterations:

for i in range(0, 101):
    V_set = i/5
    r.set_v_set(V_set)
    V_set = r.get_v_set()
    V_out = r.get_v_out()
    T_ext = r.get_ext_c()
    print("i={}, V_set={:.3f}, V_is={:.3f}, T_is={:.1f}".format(i, V_set, V_out, T_ext))
    time.sleep(2)

raise ModbusInvalidResponseError("Response length is invalid {0}".format(len(response)))
ModbusInvalidResponseError: Response length is invalid 0

This worked for 650 iterations befor the error, while it is overall faster there is a delay after every command:

V_stepsize = 0.2
V_set = 0
for i in range(0, 10001):
    n = i // 10**1 % 10 #to get 2. digit only eg. i = 1234 --> n = 3
    if (n % 2) == 0: #check if even -> increase Voltage
        vorzeichen = 1
    if (n % 2) == 1: #check if odd -> decrease Voltage
        vorzeichen = -1

    V_set += V_stepsize *vorzeichen
    time.sleep(0.2)
    r.set_v_set(V_set)
    time.sleep(0.2)
    V_set = r.get_v_set()
    time.sleep(0.2)
    V_out = r.get_v_out()
    time.sleep(0.2)
    T_ext = r.get_ext_c()
    print("i={}, V_set={:.3f}, V_is={:.3f}, T_is={:.1f}".format(i, V_set, V_out, T_ext))

[ShayBox]

Serial has no concept of connect or disconnect, it's only whether you send read/write commands or not, multiple programs can be connected at the same time, however only one can send commands at the same time, so really only one can connect at a time.
There's no limit to how fast you can send requests, I've run loops as fast as possible and had no issue, the only time I've experienced said error was when there were multiple programs connected to the serial device, I had multiple instances of the library in my code, or I tried to use the library in a multi threaded way (thread/asyncio).
In any case, serial programs either have to use locks or be single threaded blocking.
You can take a look at RidenGUI to see how I handle polling with a separate thread, QThread handles locking while native python threading doesn't, and needed a lock before I switched to QThread.

EDIT: And to help with your performance, I would suggest not using getters, and switching to the polling method described in the readme, it improves performance, every getter sends a single request for a single value, update sends 2 requests total for every value EDIT3: After testing in your example, it is faster not to use polling, but to use getters

EDIT2: The Response length is invalid error is already attempted to be recovered from once, it only fails if it happens twice, that error means some other program read/wrote at the same time, probably in the middle of this library trying to read/write and got garbage data EDIT4: I did forget to add a try-block to write requests, my bad, update to v1.1.1

EDIT5: I was able to run your example without sleeps and it took 4m 21s EDIT6: Also, with the custom firmware you can go faster than 115200 baud, the stock serial chip can't reach 1,000,000 baud, but I've been able to reach 250,000, above that you'd have to replace the chip

[Eheran1]

This does not match my experience with com-ports. If one program uses a com-port, even if there is no communication at the time (eg. I programed an arduino, so I know there is 0 communication), no other program can access it. In this case: If I run a few simple lines with you library and then try to run it again it throws a access denied error. Same as if I connect in TestProgram or Ridens software. And same the other way around - these programs cant connect if your library is running (even if the program finished) and report such a error. This doesnt happen with other programs like the Riden software (they dont stop working after some time), so it seems unrealistic that only python gets "interrupted" by something like a virus scan or whatnot. So doing some more reading I find this in the minimalmodbus documentation:

Closing serial port after each call
In some cases (mostly on Windows) the serial port must be closed after each call.
Enable that behavior with:
instrument.close_port_after_each_call = True
This will slow down the port considerably.

Here are the errors, just for reference and maybe help people find this on google. This happens after letting a simple bit of code finish that communicated with the RD6006P: SerialException: could not open port 'COM6': PermissionError(13, 'Access is denied.', None, 5) In TestProgramm:

;; Start thread for: COM6 - Riden RD6006P
;; Stopping thread for: COM6 - Riden RD6006P

In the Riden software: "open serial port error"

[ShayBox]

That's very interesting, I guess that's a similar windows design flaw to not being able to delete an open file, you can't open an open port.
On linux I can connect many programs to a single serial port, the only issue comes from simultaneously reading/writing.
It appears this is the commit that added support for that in minimalmodbus
I don't see this feature in modbus_tk, however they have hooks so I may be able to add this feature into Riden, though as minimalmodbus says, this is going to very drasticly slow down all the reads/writes, and other programs sharing the bus. And if any other program is similar and doesn't wait for the port to "close" before opening it won't work.

[ShayBox]

I pushed v1.2.0, see if close_after_call option solves the problem

[Eheran1]

I didnt update or change anything and this is what I found. This breaks really fast, like after a few iterations:

import time
from riden import Riden
from simple_pid import PID
pid = PID(1, 0.1, 0.05, setpoint=1)
pid.output_limits = (0, 14)    # Output value will be between 0 and 14
T_u = 1.84
K_u = 3.8
pid.tunings = (0.12*K_u, 0.36*K_u/T_u, 0.021*K_u*T_u) #best so far

r = Riden(port="COM6", baudrate=115200, address=1)
print(r.is_buzz())
r.set_buzz(0)
print(r.is_buzz())
print("V_set=", r.get_v_set())
r.set_v_set(0.00)
r.set_i_set(2.00)

r.update()
print("V_set=", r.v_set)
print("I_set=", r.i_set)
r.set_output(1)
time.sleep(0.2)
pid.setpoint = 1 #Watt

for i in range(0, 10001):
    time.sleep(0.02)
    r.update()
    V_set = r.v_set
    V_out = r.v_out
    I_out = r.i_out
    T_ext = r.ext_c
    Power = V_out * I_out
    PID_output = pid(Power)
    r.set_v_set(PID_output)
    print("V_set={:.3f}, {:.3f}V, {:.4f}A, {:.4f}W, PID_out={:.3f}V, {:.0f}°C".format(V_set, V_out, I_out, Power, PID_output, T_ext))

This ran for like 30 minutes now with no issue. It might still be the same timing issue tho, since "interval=20" represents a 20 ms delay of some sort:

import matplotlib.pyplot as plt
import matplotlib.animation as animation
import datetime
import time
from riden import Riden
from simple_pid import PID
pid = PID(1, 0.1, 0.05, setpoint=1)
pid.output_limits = (0, 14)    # Output value will be between 0 and 14
T_u = 1.84
K_u = 3.8
pid.tunings = (0.12*K_u, 0.36*K_u/T_u, 0.021*K_u*T_u) #best so far

r = Riden(port="COM6", baudrate=115200, address=1)
print(r.is_buzz())
r.set_buzz(0)
print(r.is_buzz())
print("V_set=", r.get_v_set())
r.set_v_set(0.00)
r.set_i_set(2.00)

r.update()
print("V_set=", r.v_set)
print("I_set=", r.i_set)
r.set_output(1)
time.sleep(0.2)

pid.setpoint = 3 #Watt

# Create figure for plotting
fig = plt.figure()
ax = fig.add_subplot(1, 1, 1)
xs = [] #store trials here (n)
ys = [] #store relative frequency here
n=0
t_start=datetime.datetime.now()
t_previous = 0.0
# This function is called periodically from FuncAnimation
def animate(i, xs, ys):
    r.update()
    V_set = r.v_set
    V_out = r.v_out
    I_out = r.i_out
    T_ext = r.ext_c

    Power = V_out * I_out
    PID_output = pid(Power)
    r.set_v_set(PID_output)
    t_now=datetime.datetime.now()
    dt = t_now - t_start
    t = dt.total_seconds() * 1
    global t_previous
    dt = t - t_previous
    t_previous = t
    print("t={:.1f}, dt={:.3f}, V_set={:.3f}, {:.3f}V, {:.4f}W, PID_out={:.3f}V, {:.0f}°C".format(t, dt, V_set, V_out, Power, PID_output, T_ext))
    global n
    n += 1
    xs.append(t)
    ys.append(Power)
    ax.clear()
    ax.plot(xs, ys, label="P over t")
    plt.legend()
    plt.axis([0, None, 2.97, 3.03]) #Use for arbitrary x and y max

ani = animation.FuncAnimation(fig, animate, fargs=(xs, ys), interval=20)
plt.show()

Now updated from 1.1.0 to 1.2.0 and so far no more issue with this. Also, speed is still just as high. Will do longer test runs tomorrow. Thanks!

Only issue I can see right now is that the port stays blocked if I interrupt the program. Is there a way to "unblock" it or check if its already open befor trying to open it again?

[ShayBox]

You can also try messing with Riden.master.set_timeout, I set it to 0.05 when initializing riden because default is some tiny calculated time based on bytes which always results in an error, 0.05 was the lowest I could use that also fixed the errors, but maybe more is needed in some cases.
I assume at some point windows decides to release things opened by non existing processes, but if the program is being killed/interrupted in the middle of a read/write and it never finishes, it's also not running any object exit functions.
You would have to either find a way to handle interrupts so they don't instantly close the program and instead exit the program normally, letting reads/writes finish and objects to be destroyed, or change how you exit your program to not use interrupts but an exit method that stops loops and lets code finish, and gracefully exit in a known state.

You can also manually control the serial and rtumaster objects through riden.serial and riden.master, both have open/close functions or _do_open/_do_close functions that you can use in any exit code/interrupt handler, but as all read/writes finish and the program exits in a way that python destroys objects then everything should be closed normally

[Eheran1]

So far I didnt have this error once. Solved! Thank you!

both have open/close functions or _do_open/_do_close functions that you can use in any exit code/interrupt handler

Im not good enough for that and still have no idea how to do that after 2 h of reading.