Buffer underflow, processdata thread not reliable ?

[Phiraos]

Hi,

This issue follows the #73.

I use a thread to periodicaly use processdata.

But, even using a thread to send cyclicly processdata, I get systematicaly the error : PVT buffer underflow.

That means, somehow, the processdata loop is not reliable in terms of period (my loop period is approximately 4ms).

Here is the source :

"""Toggles the state of a digital output on an EL1259.

Usage: python basic_example.py <adapter>

This example expects a physical slave layout according to
_expected_slave_layout, see below.
"""

import sys
import ctypes
import time
import threading
import math

from collections import namedtuple

import pysoem

class InputPdo(ctypes.Structure):
    _pack_ = 1
    _fields_ = [
        ('statusword', ctypes.c_uint16),
        ('position_actual_value', ctypes.c_int32),
        ('demand_position', ctypes.c_int32),
        ('demand_current', ctypes.c_int32),
        ('modes_of_operation_display', ctypes.c_int8),
        ('byte_padding', ctypes.c_int8),
    ]

class OutputPdo(ctypes.Structure):
    _pack_ = 1
    _fields_ = [
        ('controlword', ctypes.c_uint16),
        ('target_position', ctypes.c_int32),
        ('target_velocity', ctypes.c_int32),
        ('modes_of_operation', ctypes.c_int8),
        ('byte_padding', ctypes.c_int8),
    ]

modes_of_operation = {
    'No mode': 0,
    'Profile position mode': 1,
    'Profile velocity mode': 3,
    'Homing mode': 6,
    'Cyclic synchronous position mode': 8,
    'Cyclic synchronous velocity mode': 9,
    'Cyclic synchronous torque mode': 10,
}

class LinmotActuator:

    BECKHOFF_VENDOR_ID = 0x960970
    EK1100_PRODUCT_CODE = 0x4c4e5449
    output_data = OutputPdo()
    output_data.modes_of_operation = modes_of_operation['Homing mode']
    check_thread_period = 0.002
    process_data_thread_period = 0.003

    def __init__(self, ifname):
        self.output_data.target_position = 0 #0.1µm
        self.output_data.target_velocity = 0 #µm/s
        self.last_output_sent = None
        self._ifname = ifname
        self._pd_thread_stop_event = threading.Event()
        self._ch_thread_stop_event = threading.Event()
        self._actual_wkc = 0
        self._master = pysoem.Master()
        self._master.in_op = False
        self._master.do_check_state = False
        SlaveSet = namedtuple('SlaveSet', 'name product_code config_func')
        self._expected_slave_layout = {0: SlaveSet(
            'EK1100', self.EK1100_PRODUCT_CODE, None)}

    def _processdata_thread(self):
        while not self._pd_thread_stop_event.is_set() or self.last_output_sent != self._master.slaves[0].output:
            self._master.send_processdata()
            self.last_output_sent = self._master.slaves[0].output
            self._actual_wkc = self._master.receive_processdata(10000)
            if not self._actual_wkc == self._master.expected_wkc:
                print('incorrect wkc')
            time.sleep(self.process_data_thread_period)

    def _home(self):
        self.output_data.modes_of_operation = modes_of_operation['Homing mode']
        for control_cmd in [6,7,15,0x83F]:
            self.output_data.controlword = control_cmd
            self._master.slaves[0].output = bytes(self.output_data)
            time.sleep(0.004)
        time.sleep(10)

    def _go_to_position(self,speed,position):
        self.output_data.modes_of_operation = modes_of_operation['Cyclic synchronous position mode']
        self.output_data.controlword = 0x0F
        self.output_data.target_position = position #0.1µm
        self.output_data.target_velocity = speed #µm/s
        self._master.slaves[0].output = bytes(self.output_data)

    def _linear_move(self):
        target_velocity = 0
        target_counter = 0

        try:
            while 1:
                vitesse = 1000
                borne = 40 #mm
                target_counter += vitesse
                target_velocity = 10*vitesse
                if target_counter < borne*10000:
                    target_position = target_counter
                elif target_counter < borne*20000:
                    target_position = borne*10000 - (target_counter - borne*10000)
                else:
                    target_counter = 0
                self._go_to_position(target_velocity,target_position)
                time.sleep(0.004)
        except KeyboardInterrupt:
            # ctrl-C abort handling
            self._clean_exit()
            print('stopped')

    def _sin_move(self):
        i=0
        try:
            while 1:
                target_position = int(math.fabs(200000*math.sin(i)))
                self._go_to_position(1000,target_position)
                i += 0.001
                time.sleep(0.0002)
        except KeyboardInterrupt:
            # ctrl-C abort handling
            self._clean_exit()
            print('stopped')

    def _clean_exit(self):
            self._master.slaves[0].output = bytes(len(self._master.slaves[0].output))

    def _pdo_update_loop(self):

        self._master.in_op = True

        try:
            self._home()
            #self._linear_move()
            self._sin_move()
            """ while 1:

                time.sleep(0.0005) """
        except KeyboardInterrupt:
            # ctrl-C abort handling
            self._clean_exit()
            print('stopped')

    def run(self):

        print('opening connection')
        self._master.open(self._ifname)

        if not self._master.config_init() > 0:
            self._master.close()
            raise LinmotActuatorError('no slave found')
        print('init done')

        if not ((self._master.slaves[0].id == self.BECKHOFF_VENDOR_ID) and
                (self._master.slaves[0].man == self._expected_slave_layout[0].product_code)):
            self._master.close()
            raise LinmotActuatorError('unexpected slave layout')
        self._master.slaves[0].config_func = self._expected_slave_layout[0].config_func
        self._master.slaves[0].is_lost = False

        print('Transition system to SAFEOP_STATE')
        io_map_size = self._master.config_map()
        print('IOMap Size: {}'.format(io_map_size))
        print('config map done for {}'.format(self._master.slaves[0].name))

        if self._master.state_check(pysoem.SAFEOP_STATE, 50000) != pysoem.SAFEOP_STATE:
            self._master.read_state()
            if not self._master.slaves[0].state == pysoem.SAFEOP_STATE:
                print('{} did not reach SAFEOP state'.format(self._master.slaves[0].name))
                print('al status code {} ({})'.format(hex(self._master.slaves[0].al_status), pysoem.al_status_code_to_string(device.al_status))
                )

            raise Exception('not all slaves reached SAFEOP state')
        else:
            print('Device in SAFEOP state: {}'.format(hex(self._master.slaves[0].state)))
        self._master.state = pysoem.OP_STATE

        check_thread = threading.Thread(target=self._check_thread)
        check_thread.start()
        proc_thread = threading.Thread(target=self._processdata_thread)
        proc_thread.start()

        # send one valid process data to make outputs in slaves happy
        self.output_data.controlword = 0x8F #acknowledge error
        self._master.slaves[0].output = bytes(self.output_data)
        self._master.send_processdata()
        self._master.receive_processdata(2000)
        # request OP state for all slaves

        self._master.write_state()

        if self._master.state_check(pysoem.OP_STATE, 50000) != pysoem.OP_STATE:
            self._master.read_state()
            if not self._master.slave[0].state == pysoem.OP_STATE:
                    print('{} did not reach OP state'.format(self._master.slave[0].name))
                    print('al status code {} ({})'.format(hex(self._master.slave[0].al_status), pysoem.al_status_code_to_string(self._master.slave[0].al_status)))
            raise Exception('Not all slaves reached OP state')

        self._pdo_update_loop()

        print('done')

        self._pd_thread_stop_event.set()
        self._ch_thread_stop_event.set()
        proc_thread.join()
        check_thread.join()
        self._master.state = pysoem.INIT_STATE
        # request INIT state for all slaves
        self._master.write_state()
        self._master.close()

    @staticmethod
    def _check_slave(slave, pos):
        if slave.state == (pysoem.SAFEOP_STATE + pysoem.STATE_ERROR):
            print(
                'ERROR : slave {} is in SAFE_OP + ERROR, attempting ack.'.format(pos))
            slave.state = pysoem.SAFEOP_STATE + pysoem.STATE_ACK
            slave.write_state()
        elif slave.state == pysoem.SAFEOP_STATE:
            print(
                'WARNING : slave {} is in SAFE_OP, try change to OPERATIONAL.'.format(pos))
            slave.state = pysoem.OP_STATE
            slave.write_state()
        elif slave.state > pysoem.NONE_STATE:
            if slave.reconfig():
                slave.is_lost = False
                print('MESSAGE : slave {} reconfigured'.format(pos))
        elif not slave.is_lost:
            slave.state_check(pysoem.OP_STATE)
            if slave.state == pysoem.NONE_STATE:
                slave.is_lost = True
                print('ERROR : slave {} lost'.format(pos))
        if slave.is_lost:
            if slave.state == pysoem.NONE_STATE:
                if slave.recover():
                    slave.is_lost = False
                    print(
                        'MESSAGE : slave {} recovered'.format(pos))
            else:
                slave.is_lost = False
                print('MESSAGE : slave {} found'.format(pos))

    def _check_thread(self):

        while not self._ch_thread_stop_event.is_set():
            if self._master.in_op and ((self._actual_wkc < self._master.expected_wkc) or self._master.do_check_state):
                self._master.do_check_state = False
                self._master.read_state()
                if self._master.slaves[0].state != pysoem.OP_STATE:
                    self._master.do_check_state = True
                    LinmotActuator._check_slave(self._master.slaves[0], 0)
                if not self._master.do_check_state:
                    print('OK : all slaves resumed OPERATIONAL.')
            time.sleep(self.check_thread_period)

class LinmotActuatorError(Exception):
    def __init__(self, message):
        super(LinmotActuatorError, self).__init__(message)
        self.message = message

if __name__ == '__main__':

    print('ACMT v0 started')

    if len(sys.argv) > 1:
        try:
            LinmotActuator(sys.argv[1]).run()
        except LinmotActuatorError as expt:
            print('LinmotActuator failed: ' + expt.message)
            sys.exit(1)
    else:
        print('usage: LinmotActuator ifname')
        sys.exit(1)

I don't know how to make more stable the processdata loop.

[bnjmnp]

For now I don't get where the error is coming from. Dose it pop up on the display of the acutator? Dose it happen in the CSP mode only or also in homing?

[bnjmnp]

Dose the actuator go into SafeOP state when the error occurs, and stops the motor?

With Python and non-realtime operating systems you will always have big jitter, also network interface cards connected via USB are not good to achieve realtime. Your actuator seems not to like that jitter. I thought "Profile Position" mode would be more resistant to jitter, but this seems also not be the case for your actuator?

[Phiraos]

For now I don't get where the error is coming from. Dose it pop up on the display of the acutator? Dose it happen in the CSP mode only or also in homing?

I get the error on the utilitary software of the vendor. I use it for monitoring. I get the error only in CSP mode.

[Phiraos]

Dose the actuator go into SafeOP state when the error occurs, and stops the motor?

With Python and non-realtime operating systems you will always have big jitter, also network interface cards connected via USB are not good to achieve realtime. Your actuator seems not to like that jitter. I thought "Profile Position" mode would be more resistant to jitter, but this seems also not be the case for your actuator?

When the error occurs, the motor stops working but I can't tell is state. I use a ethernet port on a NUC. I don't know how the send_processdata works but, I was thinking maybe the update loop and the processdata loop try to access in the same time the output and that's why it gets out of period.