search

svenhb / GRBL-Plotter

A GCode sender (not only for lasers or plotters) for up to two GRBL controller. SVG, DXF, HPGL import. 6 axis DRO.
https://grbl-plotter.de/
GNU General Public License v3.0
674 stars 176 forks source link

Invisible Alarm #241

Closed amoineau closed 2 years ago

amoineau commented 2 years ago

Describe the bug

Hello,

We had a bug where the 3rd serial COM would get stuck and even resetting or restarting wouldn't fix the problem. At first we thought it was a problem with our board or its firmware. After some time trying to find the problem we realised it was just GRBL-plotter's 3rd terminal refusing to send commands (it was still able to receive data from the board). It took me some time before I realised it was the same behaviour than when we are blocked by an alarm (although the main terminal wasn't stuck apparently). So, on a hunch, I pressed "Kill Alarm" and it unstucked the terminal.

The problem is that there was none of the usual signals telling us we had an alarm. Even now that we have a solution it's really not intuitive and the operator just stays stuck on the problem every time it happens.

To Reproduce

Unfortunately, it happens now and then, in various circumstances. We haven't been able to find what was triggering this "invisible alarm" yet. We are not even sure that it requires an "alarm triggering situation" for it to happen. I know it's really not ideal...

I hoped with your help we would be able to diagnose the issue more precisely.

Expected behavior

First, I think we need to figure out if this "fake-alarm" is actually supposed to happen or if it's a glitch. And depending on that, we would like to make it completely disappear or completely visible ^^.

Thank you in advance for your help !

svenhb commented 2 years ago

Perhaps logging helps to find reason... https://github.com/svenhb/GRBL-Plotter/releases

amoineau commented 2 years ago

Hello,

Yes the problem is so vague for now that being able to figure out how where it's coming from is the only move. I hope we will be able to catch it with these new logs.

Since last time we already noticed that the "alarms" didn't seem to be legitimate, they can happen when nothing is wrong, and seemed related to the 3rd serial terminal.

amoineau commented 2 years ago

While I was testing the new version I actually encountered the bug ! And I was in the middle of the logs so, this time, I know exactly how and when it happened.

I was processing a very common file I did a million time before, in a hardware simulation setup (at my desk but with the actual boards). So the first thing I can say is that it's not the files or the machine (interference, etc..) causing the problem.

Everything was going fine, the blockage happened at the very end of the file. Normally grbl-p is supposed to send a (^3 $SH0) command to request that the shutter of the laser is closed. I noticed the problem because the file was seemingly complete but grbl-p was not considering the process as complete, I realised hat we were missing the shutter command and the end of process one. I pressed kill alarm and the missing commands were sent.

It matches the previous experiences I had with this problem, it happens right before sending a command to the 3rd COM, the command isn't even sent yet. That's why at first it wasn't obvious that it might concern the 3rd serial.

Here are the logs : logfiles.zip

The file just in case : formation montage_EXT.zip

And the setup : GRBL-Plotter_20211220_115950.zip

svenhb commented 2 years ago

Problem: before sending any command to the 3rd serial, the 1st serial (grbl) must be "IDLE" and buffer must be empty.

2021-12-20 11:29:39.0726 | Trace | GrblPlotter.ControlSerialForm  | process 3rd free:98  size:127
2021-12-20 11:29:41.5257 | Trace | GrblPlotter.ControlSerialForm  | process 3rd free:109  size:127
2021-12-20 11:29:41.7294 | Trace | GrblPlotter.ControlSerialForm  | process 3rd free:109  size:127

The log shows, that the buffer is theoretically not empty and "free" doesn't change anymore (probably because of missing "ok" messages). I built in a workarround: if there is no change in "free" it will send to the 3rd serial anyway, because grbl is already "IDLE".

amoineau commented 2 years ago

Hello Sven, happy new year ! Sorry to bother you, I just wanted to bump this subject. Is there something I can do to help figure out a solution ? Do more logs with this problem could help you figure out the issue ? Thank you for your help.

svenhb commented 2 years ago

Happy new year!

I wrote: I built in a workarround: if there is no change in "free" it will send to the 3rd serial anyway, because grbl is already "IDLE".

Did you try the latest release 1.6.4.0?

amoineau commented 2 years ago

Oh sorry ! I didn't think you changed this in 1.6.4.0 I was still using 1.6.3.5. I thought maybe you were not done making the changes or it wasn't working yet.

I'll try it and get back to you.

amoineau commented 2 years ago

I'm currently running tests on the machine, I haven't met the problem in an hour so it's looking good on that front ! However, from time to time, it seems that the 3rd serial commands are sent too early. In my case it means that we still have a few lines to draw and the conveyor start running, with the laser still on...

amoineau commented 2 years ago

Actually, I think the few times it happened was on "Point" shapes, which are more or less ignored, aren't they ?

amoineau commented 2 years ago

I've tried several files, several times and I haven't clearly noticed the problem again. Maybe it's just with the point (we were not supposed to have points in our files anyway).

Anyway, for now let's say that the problem is solved, I'll get back to you if we can clearly see a problem again.

amoineau commented 2 years ago

Ok the problem is manifesting again and repeatedly on the file mentioned in #255. Maybe the two problems are linked so I will try the new version and see how it goes but it seems far fetched. To sum up the problem : sometimes the spindle stays on between pages, when the conveyor moves. Thus cutting the pages. Apparently it happens consistently between pages 4 and 5 (1 meter per pages). Sometimes it seems to actually be finishing the last lines of the previous page because it cut and move the head.

amoineau commented 2 years ago

I received a video of the last time it happened, and we can clearly see that the commands that are supposed to be sent between pages are sent just before the last figure is finished. Which leads to the last line/figure being drawn while moving :/

svenhb commented 2 years ago

Any logfiles which could help to identify the problem?

amoineau commented 2 years ago

I will try to get them

amoineau commented 2 years ago

Ok, I got one. However it's kinda hard to know when the problem occurred from the logs, at least for me. Fortunately the log is not too long : log_20220214.0.txt

svenhb commented 2 years ago

I found the problem: I wrote: I built in a workarround: if there is no change in "free" it will send to the 3rd serial anyway, because grbl is already "IDLE". Unfortunatly I didn't clear the "noChangeCounter" after a change appeared...

amoineau commented 2 years ago

It would be great if it was that !

svenhb commented 2 years ago

Seems to be solved