search

OctoPrint / OctoPrint-FirmwareUpdater

OctoPrint plugin for flashing pre-compiled firmware images to a 3D printer.
https://plugins.octoprint.org/plugins/firmwareupdater/
GNU Affero General Public License v3.0
353 stars 76 forks source link

[Feature Request] Use Marlin's Binary File Transfer protocol for SD card flashing #161

Closed yet-another-average-joe closed 3 years ago

yet-another-average-joe commented 3 years ago

Hardware Setup

BTT SKR 1.4 Turbo + RaspBerry 3B+ + HDMI touchscreen + TouchUI ; last versions of OctoPrint (1.5.0) and FirmwareUpdater (1.7.3), OctoPi and OctoPrint updated/upgraded today.

Describe the problem

Connect the board and the Raspi using the GPIO, NOT the USB cable. Then attempt to do a firmware update over the RasPi GPIO serial port = /dev/ttyAMA0 or /tty/serial0 (its alias). It fails telling the board reset failed. The board resets as usual, but nothing happens.

If "reboot before updating" is unchecked, flashing fails. It seems FirmwareUpdater is unable to flash through GPIO and board Tx/Rx.

***** THIS COULD A BOARD LIMITATION I'M NOT AWARE OF ! (???????????) **

But it always works fine over /dev/tty/ACM0 (serial USB)

Log Files

Log files were generated this way :

/dev/serial0 and /dev/ttyAMA0 being the same, no surprise.

logs.zip

Additional context

I'd like to update the firmware over TTL serial connection rather than serial USB, as I'd like to get rid of the USB cable and all the interface overhead. It seems FirmwreUpdater does not detect the reset ; the board DOES reset (the display flashes, and the terminal shows it did. No problem either with M20-M21, or the Marlin "file explorer" in the menu.

Workaround : having the two connections (GPIO and USB), and choosing USB for firmware upgrades...

benlye commented 3 years ago

I can't see how you're going to be able to copy the new firmware to the SD card over a serial connection. Does the host see the printer's SD card?

In any case, without a board to experiment with there's really nothing I can do with this.

yet-another-average-joe commented 3 years ago

Yes, it does. M20 and M21 return expected responses :

SD

If I insert a SD card with firmware.bin, and then send M997, it reboots (and therefore updates the firmware), but I get a timout error. Firmware.bin is "eaten". AFAIK, when updating over USB with FirmwareUpdater, firmware.bin remains on the SD card (when updating over USB from a PC, or when inserting a SD card and rebooting, it is copied to firmware.cur, en then deleted )

SD2

If I undestand correctly, FirmwareUpdater does this :

and it cannot copy firmware.bin over a pure serial (non USB) connection. Or it sends data as usual, but Marlin can't handle the data ? I have a question : when copying files to a SD card, does FirmwareUpdater use USB communication, or emulated serial over USB ? If it sends data through emulated serial, the problem (if any) should be on the board side... But I see no reason why data coming from emulated serial couldn't be sent to the SD card SPI bus (if someone coded that in Marlin). After all, M997 can be used over WiFi...

benlye commented 3 years ago

The only way to copy the firmware over a pure serial connection is to use Marlin's 'experimental' binary file transfer protocol, which I've never looked at implementing (I'm not really inclined to build an implementation until it goes final). https://github.com/MarlinFirmware/Marlin/pull/14817

Unless your OctoPrint server can see and mount the printer's SD card, there is no other way I know of.

yet-another-average-joe commented 3 years ago

I want to avoid using the RasPi USB ports because they will be ar the back of the 3D printer control enclosure.

Last idea : a USB hub hat. I imagine that FU doesn't care, and as long as there's an OS recognized USB connection, it will use it successfully. In your opinion, will it work ? If the answer if no, I will not purchase. If it is yes or maybe, I order right now. Flex PCB cables are perfect for internal wiring (just received some from Ali)

benlye commented 3 years ago

Sure, I can understand why you would want to do it (it's actually quite cool), but the serial protocol alone does not support transferring files at this time. It will in the future, but the implementation isn't ready for use yet. Once the file transfer protocol is ready I can look at implementing it, but there is no timeframe for that on the Marlin side as far as I know.

If you used a hub to connect the board's USB port to your Raspberry Pi it will be no different to a direct connection as far as the Pi is concerned, so should work just fine from the Firmware Updater plugin's point of view.

yet-another-average-joe commented 3 years ago

Obviously, I understand why you don't want to develop (or just release ? ;) ) software using experimental interfaces !

Thanks for the answers ! They helped a lot.

yet-another-average-joe commented 3 years ago

Firmware update over bare GPIO serial connection works for boards using the LPC1768x. I've been testing it many times, for 1 1/2 month with no issues. The binary is simply copied to the onboard SD card using Marlin Binary File Transfer plugin, and then the firmware is updated at next reboot (as ususual with these boards, hard reset or M997).

I think it is a viable option for OctoPrint Firmware-Updater.

benlye commented 3 years ago

Firmware update over bare GPIO serial connection works for boards using the LPC1768x. I've been testing it many times, for 1 1/2 month with no issues. The binary is simply copied to the onboard SD card using Marlin Binary File Transfer plugin, and then the firmware is updated at next reboot (as ususual with these boards, hard reset or M997).

I think it is a viable option for OctoPrint Firmware-Updater.

Like I've said, I'll add support for binary file transfer once the protocol is no longer 'experimental'.

(Of course anyone else is welcome to code it and submit a PR, if they were inclined to do so.)

yet-another-average-joe commented 3 years ago

It was just a return of experience after intensive testing (maybe 100 times... for now), as I am also developping things and have been using this protocol for all firmware updates, tinkering with Serial and SPI on both sides, and bricked nothing. It was not a feature request : I wrote that I understand you don't want to rely on an experimetal protocol for critical operations. Just thought it could help ; OctoPrint Firmware Updater is not installed on my dev rig.

A follow up, nothing more, just because this thread wasn't closed after 2 months...

benlye commented 3 years ago

I'm not really concerned about reliability, I'm concerned that the protocol implementation is going to change, meaning I would potentially have to redo all my work. I would much rather wait until the protocol implementation is stable, then implement it once.

Plus, I do not have a board to test it on.

benlye commented 3 years ago

I now have a BTT SKR v1.4 Turbo board. As time permits I'll look into adding support for the BFT protocol.

benlye commented 3 years ago

I have a very basic implementation of this working. I have some more work to do but you can expect to see a beta release with this feature soon.

benlye commented 3 years ago

I've just published v1.8.1b0 with initial support for Binary File Transfer.

IMPORTANT: Read the release notes!! There are a couple of things you need to know.

Then, switch the Firmware Updater plugin to the 'Development' release channel in the Software Update plugin and install 1.8.1b0 and try it out. Please give feedback in this issue.

image

benlye commented 3 years ago

I've had to unbundle the dependencies from the FirmwareUpdater plugin installer because they were causing Python2/3 compatibility issues.

Updated release notes: https://github.com/OctoPrint/OctoPrint-FirmwareUpdater/releases/tag/1.8.1b2

If you already have the MarlinBFT plugin installed (or a previous installation) then it won't matter, but for new installations of the Firmware Updater plugin with BFT support the dependencies need to be installed manually. If the dependency isn't installed the plugin will give an error saying that it needs to be installed. Eventually this will include a link to documentation.

benlye commented 3 years ago

Version 1.9.0 with support for the current beta version of Marlin Binary File protocol has been released.

mriscoc commented 3 years ago

I Think that the last Marlin bugfix-2.0.x release has broken the binary file transfer protocol. Now when I upload a binary file, the printer does not respond to commands coming from serial interface, but it is not hung.

benlye commented 3 years ago

I Think that the last Marlin bugfix-2.0.x release has broken the binary file transfer protocol. Now when I upload a binary file, the printer does not respond to commands coming from serial interface, but it is not hung.

It's working here. I just compiled the bugfix-2.0.x branch with all the commits up to 5 minutes ago and can flash and re-flash my BTT SKR 1.4 Turbo without any problems.

mriscoc commented 3 years ago

I Think that the last Marlin bugfix-2.0.x release has broken the binary file transfer protocol. Now when I upload a binary file, the printer does not respond to commands coming from serial interface, but it is not hung.

It's working here. I just compiled the bugfix-2.0.x branch with all the commits up to 5 minutes ago and can flash and re-flash my BTT SKR 1.4 Turbo without any problems.

Thank you for the confirmation. I'm using bugfix-2.0.x with Creality Ender 3V2 Configuration files. Previous binary compiles just were working ok.

github-actions[bot] commented 3 years ago

This issue has been automatically marked as stale because it has not had any recent activity. Please add a reply if you want to keep this issue active, otherwise it will be automatically closed in 5 days.

github-actions[bot] commented 3 years ago

This issue has been automatically locked because there was no further activity after it was closed. Please open a new issue for any related problems.