[FR] Binary format for gcode-files for faster transfer

[jtkDvlp]

Description

Hi there,

a few weeks ago I decided to write my own wifi printer client (not public yet, will comming). Its working well, I also upload gcode files via wifi to the printers sd card and of course as others already mentioned its slow caused by the serial interface. Of course you could try to get the communication better, but I thinks thats difficult if you dont want to make big changes to the communication itself. But the gcode transfered actual is human readable. Of course thats useful if you want to control the printer via console. But for printing you dont do this just for configuring. So there is no need for human readable gcode files per se. So I decied to give it a try and two afternoon later I had written a snippet converting gcode to binary gcode and vise versa saving up to 50% file size. For my simplicity its written in clojure. But I also could provide an JavaScript Version for clients and C / C++ Version for the firmware.

If there is interest in such a feature and the marlin contributors are willing to accept such a pull request, it would be nice to get this done. As already mentioned I can do / help with this.

For writting the C / C++ part for the firmware it would be nice to get a little api description I have to provide or some direct source links to read getting an overview for the integration.

Best regards jtkdvlp

Additional Information

Clojure snippet to convert gcode to binary gcode and vise versa saving up to 50% file size. https://gist.github.com/jtkDvlp/05322a3c333410f1b89ce98808964d9d

[ellensp]

There have been several attempts at this see BINARY_FILE_TRANSFER (currently in Marlin) and https://reprap.org/wiki/Packed_Binary_GCode

[jtkDvlp]

Mmh XD perfect!

So this option accepts gcode files with schema of the wiki articel? Do you know some client code for this? Otherwise I tweak my code.

[ellensp]

Different systems.

From reading the code BINARY_FILE_TRANSFER uses compression, this https://github.com/atomicobject/heatshrink

[rhapsodyv]

Here is the original PR. It have a script for upload binary data to marlin too. https://github.com/MarlinFirmware/Marlin/pull/14817

[jtkDvlp]

@ellensp does BINARY_FILE_TRANSFER work. I send M28 B1 foo, then sending some binary stuff and ending with M29 foo but the printer does not respond. Any ideas?

[p3p]

BINARY_FILE_TRANSFER #14817 as it is currently in Marlin, can only be used to transfer files to the sdcard. To use BINARY_FILE_TRANSFER you need to implement the protocol as described, or just use the script provided, I think there's an octoprint plugin that takes advantage of it for GCode file uploads.

It is still a project I'm working on and the version in Marlin is unfinished (my dev version is nothing like the it), other things have taken priority, at its core this feature is just a multi channel serial binary transport layer, and you could implement any binary protocol you wanted on top of it, I think streaming compressed gcode is enough of a bandwidth improvement, but you could also implement a binary gcode protocol which would probably be more computationally efficient on the mcu.

[ellensp]

Never looked at it.

But I just gave it a try.

NB transfer.py does not handle long files names (neither does marlin). only 8.3

On USBCDC 32bit Machines this seems to work as expected On serial 8 bit machines I get timeout errors

[jtkDvlp]

@p3p do you have slack or discord for this feature?

[p3p]

Nothing specific to the feature no, I'm on the Marlin discord if you would like to discus something.