GRBL: newlines consume RX buffer space

[tbfleming]

@cprezzi

[cprezzi]

I will check that again. I thought I checked that with the buffer space reported by Grbl.

[cprezzi]

Indeed, this is an error, but there is another error (in grblBufferSpace) which caused that there was never a buffer overflow ;) Thing is, I get differences between calculated and reported space left, also with "correct" calculation. Grbl allways report more speace than I calculate, which doesn't create a failure, but we don't reach the optimal speed. I will investigate some more...

[tbfleming]

I've been debugging grbl itself. Here's some of what I found so far.

• If I pipe server.js's output to a file, it keeps up better. Windows console is slow.
• Without this fix grbl's rx buffer sometimes overflows.

[tbfleming]

Differences between calculated and reported space left:

• The description of '?' in the wiki is misleading. It doesn't immediately report status back. Instead the serial ISR sets a flag and skips adding it to the RX buffer. The main loop eventually handles the flag. If the main loop is currently parsing gcode, then it will also parse gcode that's after the '?' before reporting the status.
• There's a bug in the status report; it's sometimes off by 1.

[ghost]

@chamnit ^

On Jan 12, 2017 6:09 PM, "Todd Fleming" notifications@github.com wrote:

Differences between calculated and reported space left:

• The description of '?' in the wiki is misleading. It doesn't immediately report status back. Instead the serial ISR sets a flag and skips adding it to the RX buffer. The main loop eventually handles the flag. If the main loop is currently parsing gcode, then it will also parse gcode that's after the '?' before reporting the status.
• There's a bug in the status report; it's sometimes off by 1.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/LaserWeb/LaserWeb3/pull/218#issuecomment-272204728, or mute the thread https://github.com/notifications/unsubscribe-auth/AHVr2ze-9sCwqBijJqFIYSo_HZT3c0v0ks5rRlAwgaJpZM4LhR0o .

[tbfleming]

Sorry @chamnit for not mentioning these to you; until now I didn't think anyone else would care about them since that field's normally disabled.

I thought there was an issue in serial_get_rx_buffer_available(), but looking closer I see that I was wrong. I was thrown off by it using RX_BUFFER_SIZE instead of RX_RING_BUFFER. That adjusts for the 1.

[cprezzi]

@chamnit : My tests have shown, that immediate commands (like !) consumes space in the reported buffer space, event if they are never going into the buffer (as I think). Can you confirm that?

[tbfleming]

@cprezzi I'm looking at that part of GRBL's code now. ! never enters the RX buffer. When you test it are you including a \n? That would enter the buffer.

[cprezzi]

Oh yes, because I tested with manual commands instead of klicking buttons there is allways a \n sent with each command. Forgot that, sorry. But we still have some drift somewhere.

[cprezzi]

I was able to verify, that my buffer space calculation is correct. Every time I get an ok I free up the space of the corresponding sent line. The only reason to see a drift apart would be, if not all sent commands would generate an ok (or I miss one for some reason).

[tbfleming]

@cprezzi I hit a case that I tracked down to a bad serial link. It causes occasional character drops. What board are you testing with?

[tbfleming]

Also, if you send me a test case, I'll run it though my board and see if it drifts. So far I'm not seeing any; the final free value your code reports is 128.

[chamnit]

@tbfleming : Yes, I altered the serial ring buffers to be the assigned value, rather than off by one. So RX will be 128 bytes, rather than 127 in the past. Over the years, I got one too many people writing their own streaming programs that did not account for the lost byte in a ring buffer. For existing streaming programs, this should not effect how it performs.

[cprezzi]

@tbfleming I am testing with different Arduino Unos and Nanos (all with CH340 USB chip). I know that these can cause transfer problems, but I never had missing lines or a shift in the axes. For raster testing, I use this file:

Stuttering allways starts at locations with big contrast (in the hair). It seems that big S PWM changes take more time than small changes. Could you replicate that?

[tbfleming]

Are you generating with LW3 or 4? What settings are you using?

[tbfleming]

If you're using LW4 with Join Pixels on, then the hair sections have high block rates. You need to reduce Cut Rate or increase Laser Diameter compared to images with lower detail.

[tbfleming]

grbl-lpc with LW3's grbl-buffer branch handle this image fine at 500 mm/s, .3mm spacing, with Join Pixels = off.

[cprezzi]

I was creating raster gcode with LW3 only, join pixels on, laser diameter 0.1mm, 300dpi for the picture and set both speeds to 50mm/s for starting. During running I search the max. speed before stutter starts with feed overrides. On smoothieware, I got around 42mm/s before stuttering starts.

Now I test with grbl-lpc and LW3 grbl-buffer. I still create the raster gcode with LW3 (to get comparable results). It runs much faster now, but when I increase the feed to >= 100mm/s the frontend gets unresponsive. The reason seems to be the massive amount of ok messages sent to the browser. When I don't forward the ok's to the browser I am able to run the same job with 96mm/s.

So you did at least duble speed compared to smoothieware! :)

And I think that's not the limit yet. It looks like the server.js is the limiting factor at the moment. I will try to optimize server.js to get better performance (for example: array shift, unshift, push is much slower than manual index pointers).

How did you reach 500mm/s? What was the settings? Was the frontend still responsible?

[ghost]

We can have the backend handle feedback and only update frontend less frequently. I really dont mind (:

On Jan 16, 2017 3:50 PM, "Claudio Prezzi" notifications@github.com wrote:

I was creating raster gcode with LW3 only, join pixels on, laser diameter 0.1mm, 300dpi for the picture and set both speeds to 50mm/s for starting. During running I search the max. speed before stutter starts with feed overrides. On smoothieware, I got around 42mm/s before stuttering starts.

Now I test with grbl-lpc and LW3 grbl-buffer. I still create the raster gcode with LW3 (to get comparable results). It runs much faster now, but when I increase the feed to >= 100mm/s the frontend gets unresponsive. The reason seems to be the massive amount of ok messages sent to the browser. When I don't forward the ok's to the browser I am able to run the same job with 96mm/s.

So you did at least duble speed compared to smoothieware! :)

And I think that's not the limit yet. It looks like the server.js is the limiting factor at the moment. I will try to optimize server.js to get better performance (for example: array shift, unshift, push is much slower than manual index pointers).

How did you reach 500mm/s? What was the settings? Was the frontend still responsible?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/LaserWeb/LaserWeb3/pull/218#issuecomment-272866555, or mute the thread https://github.com/notifications/unsubscribe-auth/AHVr23fbNdBjQMuCvdtmp2R7R5i9JLcfks5rS3WrgaJpZM4LhR0o .

[cprezzi]

Ah ok, you tested with 3 times bigger laser diameter, that's a big difference. With 0.3mm I also get more than 500mm/s 👍

[cprezzi]

@openhardwarecoza : Most of the ok messages are anyways not needed in the frontend, so no problem to hold them back. There is also some optimization potential in server.js, as discussed on issue #219 allready.

[tbfleming]

Join Pixels varies the block rate depending on image detail; if you maximize the feed for one image it may not handle another. I do all my testing with it off; it means I can measure the block rate and know that the settings I come up with will work with all images. The block rate decreases during acceleration at the endpoints, so it's more reliable to measure with wide images. For LW4-produced gcode, block rate = feed / laser diameter.