Micro Queue is already full

skykep commented 1 year ago

Current Behavior

At random times, the script will stop reading data from a 1756-L81E ControlLogix. The script also reads from a 1769-L32E and that data continues to read normally. This error occurred before the introduction of the controller manager, too.

From my PM2 log: 0|plc | 2023-08-29 07:26:16: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:16: Reading Error - Analog_Inputs[6] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:17: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:18: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:18: Reading Error - Discrete_Outputs[35].Out - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:19: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:20: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:21: Reading Error - F8[41] - TIMEOUT occurred while writing Reading Tag Group. 0|plc | 2023-08-29 07:26:22: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:23: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:23: Reading Error - Discrete_Outputs[35].Out - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:24: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:25: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:26: Reading Error - F8[41] - Micro Queue is already full at size of 100!!! 0|plc | 2023-08-29 07:26:26: Reading Error - Analog_Inputs[6] - Micro Queue is already full at size of 100!!!

Expected Behavior

Should continue reading PLC.

Possible Solution (Optional)

Context

Steps to Reproduce (for bugs only)

Unable to reproduce. Will work fine for days or weeks before throwing these errors.

Your Environment

Package version (Use npm list - e.g. 1.0.6): 2.6.1
Node Version (Use node --version - e.g. 9.8.0): 12.21
Operating System and version: 4.4.180+
Controller Type (eg 1756-L83E/B): 1756-L81E
Controller Firmware (eg 30.11): 30.11

SerafinTech commented 1 year ago

You must be doing lots of reads and writes and not waiting for them to complete before doing another one.

You could try to increase the max queue size. This will double it:

const {Controller} = require('st-ethernet-ip');

const PLC = new Controller()

PLC.workers.read.max = 200;
PLC.workers.write.max = 200;
PLC.workers.group.max = 200;

I would have to see some of you implementation to help further

skykep commented 1 year ago

No writes, but about 80 total reads. Most of those occur once per minute, with about 10 of them at once per 5sec, and 3 of them occurring once per second. Didn't think that would be enough to make it barf.

I'll start with changing the queue size and if still an issue, share the code with you.

skykep commented 11 months ago

Increased to 200, worked fine since until a power outage yesterday. One PLC (a CompactLogix) returned communication normally, but the other, a L81 ControlLogix, did not and had this error:

2023-09-22 08:23:41: Reading Error - SolidsTemp - Micro Queue is already full at size of 200!!! 2023-09-22 08:23:41: Reading Error - F8[41] - Micro Queue is already full at size of 200!!!

It seems the connection manager is not working properly in my scenario.

var { Controller, Tag, TagGroup, EthernetIP } = require("st-ethernet-ip"); ... PLC[i] = [new Controller(false), result[i].area, result[i].ipaddress, result[i].slot]; ... PLC[i][0].connect(PLC[i][2], PLC[i][3]).then(async (err) => { PLC[i][0].scan_rate = 1000; PLC[i][0].workers.read.max = 200; PLC[i][0].workers.write.max = 200; PLC[i][0].workers.group.max = 200; `

SerafinTech commented 11 months ago

I would try implementing the ControllerManager class when using the library. It automatically reconnects to controllers after disconnect and you don't have to worry about too many write or read requests.

Here is an example of how to implement which I think will work well for your application

const {ControllerManager} = require('st-ethernet-ip');

let cm = new ControllerManager();

//If you want to do something with the value every 1000ms , I would set the rpi to something smaller (500) so your sure to have a new value.
cm.addController('192.168.121.10', 0, 500);
//Always connect controller after adding it
cm.controllers[cm.controllers.length - 1].connect();
//Add Your tags to the controller (Example is DINT Controller scope tag called 'RATE_TEST99' )
cm.controllers[cm.controllers.length - 1].addTag('RATE_TEST99');
//Add Your tags to the controller (Example is REAL Program:MainProgram scope tag called 'TEST_POINT1' )
cm.controllers[cm.controllers.length - 1].addTag('TEST_POINT1', 'MainProgram');

// READING TAG VALUE EXAMPLE
//Do something with tag values every 1000ms
setInterval(() => {
   //ALWAYS do a fresh cm.getAllValues() before using those values
  let tagValues = cm.getAllValues();

  // Display all tag names and values at controller '192.168.121.10'
  console.log(tagValues['192.168.121.10']);
  // Display tag named 'RATE_TEST99' value
  console.log(tagValues['192.168.121.10']['RATE_TEST99']);
  // Display tag named 'TEST_POINT1' value
  console.log(tagValues['192.168.121.10']['Program:MainProgram.TEST_POINT1']);

  //Another way to get a current tag value
  console.log(cm.controllers.find(c => c.ipAddress === '192.168.121.10').tags.find(n => n.tagname === 'RATE_TEST99').tag.value);

}, 1000);

//WRITING TAG VALUE EXAMPLE
//Write a incrementing value to a controller tag after 1000ms
setInterval(() => {
  //Increment value and write to DINT controller scope tag 'RATE_TEST99' at controller '192.168.121.10'
  let tag1 = cm.controllers.find(c => c.ipAddress === '192.168.121.10').tags.find(n => n.tagname === 'RATE_TEST99').tag;
  tag1.value = tag1.value + 1;

  //Increment value and write to REAL Program:MainProgram scope tag 'TEST_POINT1' at controller '192.168.121.10' 
  let tag2 = cm.controllers.find(c => c.ipAddress === '192.168.121.10').tags.find(n => (n.tagname === 'TEST_POINT1' && n.program === 'MainProgram')).tag;
  tag2.value = tag2.value + 0.1;

  //ALWAYS find the tag as shown before doing anything with the tag to insure you are not using a stale tag instance

}, 1000);

skykep commented 7 months ago

Can you still do tag groups using the ControllerManager?

d-r-lenin commented 7 months ago

Can you still do tag groups using the ControllerManager?

@skykep did the Micro Queue problem resolve from previous response from @SerafinTech ??

skykep commented 7 months ago

Can you still do tag groups using the ControllerManager?

@skykep did the Micro Queue problem resolve from previous response from @SerafinTech ??

I don't think so because the issue was still occurring after power outages where the PLC went offline. I am implementing the ControllerManager now, but noticing it is not actually reconnecting on it's own. I opened a new Issue for that one.

radrevere commented 4 months ago

I saw this when I first started using the controller.readTagGroup function. When I looked into it I noticed that in controller/index.js the code is as follows:

readTagGroup(group) {
        return this.workers.group.schedule(this._readTagGroup.bind(this), [group], {
            priority: 1,
            timestamp: new Date()
        });
    }

you will note that it returns whatever the schedule function does which is happens to be asynchronous call so it returns immediately. If you look further in the same file you will see the following for the scan function:

async scan() {
        this.state.scanning = true;
        while (this.state.scanning) {
            await this.workers.group
                .schedule(this._readTagGroup.bind(this), [this.state.subs], {
                priority: 10,
                timestamp: new Date()
            })

Notice the await used when calling the same schedule function. Because schedule is an asynchronous call it returns before the scheduled tasks are completed unless you use await. Therefore you must use await with the readTagGroup to ensure you are not stacking up reads as SerafinTech pointed out earlier.
That being said, if it is returning a promise you should be able to use the .then functionality to resolve it as well:

controller.readTagGroup(mygroup).then(()=>{
// code to react to group values
});

Can someone confirm that the .then functionality works please?

zenovix commented 1 month ago

Hii, facing the same issue, Any Solution?

SerafinTech / ST-node-ethernet-ip