search

NebraLtd / helium-syncrobit

Nebra OpenFleet for Syncrobit CM4 Miners
https://nebra.io/syncrobit
MIT License
11 stars 5 forks source link

all of my nebra flashed units are offline #17

Open nickarpik90 opened 1 year ago

nickarpik90 commented 1 year ago

I flashed my hotspots with 2023.02.07.0-25, reinstalled them and found them to be online and working but as of a few days ago they have all gone offline again. im assuming this is because updates are not being pushed to the units.

shawaj commented 1 year ago

@nickarpik90 try rebooting the unit

We have not seen this issue on any other devices.

How do you know they are offline?

nickarpik90 commented 1 year ago

odd that all 6 exhibited the same behavior

nickarpik90 commented 1 year ago

i'll try to find time to drive out to them again to power cycle.

ifeign commented 1 year ago

Just fyi, I've been messing around with a Syncrobit to repurpose it and have tried several images including this one, Home Assistant, Ubuntu and others. They all had issues with going offline when the device rebooted for updates etc. They would only start up again after a manual unplug power cycle.

The fix was to update the EEPROM to the latest firmware. It now power cycles properly. I highly recommend everyone having issues update their EEPROM

shawaj commented 1 year ago

@WayneNebra just FYI on the above ^^

@ifeign is this the same process you followed to update it? https://www.jeffgeerling.com/blog/2022/how-update-raspberry-pi-compute-module-4-bootloader-eeprom

ifeign commented 1 year ago

@WayneNebra just FYI on the above ^^

@ifeign is this the same process you followed to update it? https://www.jeffgeerling.com/blog/2022/how-update-raspberry-pi-compute-module-4-bootloader-eeprom

Yeah, Jeff's guide is great. You can ignore the stuff about update-pieeprom.sh and just run the update

First, confirm you have the death-on-reboot issue: use the UI to reboot your miner, if you get a solid green light on the board after about 30 seconds or so, your miner is stuck in limbo.

Here's the steps to fix this:

You also might find this release interesting @shawaj, Syncrobit tried to fix this with an OTA update.

WayneNebra commented 1 year ago

@shawaj Thanks! 🙂

shawaj commented 1 year ago

@ChristopherRush can we get this eeprom update info added to the Syncrobit guide as an FAQ perhaps? https://github.com/NebraLtd/helium-syncrobit/issues/17#issuecomment-1563346543

https://support.nebra.com/support/solutions/articles/24000083132-getting-started-with-syncrobit

And maybe can add it for Linxdot CM4 and Pycom as well as presumably they might be affected by similar

shawaj commented 1 year ago

Also @ccrisan do you think this can be done OTA based on your work here? https://github.com/syncrobit/chameleonos/releases/tag/version-2022.08.05.0

Or it was not successful?

ccrisan commented 1 year ago

Unfortunately EEPROMs on CM4s cannot be updated remotely, they need to be connected locally, via USB, to an I/O board.

On the other hand, from my experience with these CM4s, I think the problem could be at least reduced by adding hung_task_panic=1 and panic=10 to kernel command line.

The reason why this works is that somehow one of the internal kernel processes hangs when trying to talk to the SDMMC (or simply detect its presence or something), while the watchdog hasn't yet been initialized so it can't be relied upon. hung_task_panic=1 will generate a kernel panic when a kernel task hangs, while panic=10 will reboot the kernel 10 seconds after a panic.

shawaj commented 1 year ago

@ccrisan just to clarify, these lines go in cmdline.txt in boot partition?

ccrisan commented 1 year ago

just to clarify, these lines go in cmdline.txt in boot partition?

Yes, that is correct, those need to be added to cmdline.txt.

shawaj commented 1 year ago

Ah ok, so we actually can't do that from Balena I don't think, annoyingly (at least without making a custom build of the OS)