Avoid filesystem corruption

[vguttmann]

I would very much like to build the mower and help with development, but for me to build it, it needs to be foolproof long-term, as I will be moving out soon, and I don't think my parents have any interest in reinstalling the operating system on their lawn mower.

Raspberry Pis have a history of corrupting the file system on their SD cards, so I'd like to know if there are any mitigations for this, and if not, advocate for their implementation.

[Schaumi19]

As far as I know the OS is still Raspbian with ROS installed. The Problem with the SD-card is writing and in Raspbian there is a option to not write to SD. If that is selected it writes only to the RAM so if there is a power loss the system is set back to the point the option was selected. Hope it helps

[ClemensElflein]

This might be a problem indeed, since I'm using Ubuntu on the Pi. I haven't gotten ROS to work with Raspbian in a stable way. But @Schaumi19 is right, there is an option which just writes to RAM and does not modify the SD. I haven't used it yet, though since I'm working on the software and therefore need write access to the SD

[scottishjohn1982]

If memory serves me correctly it is done through some sort of /etc/fstab modification. I think it uses the overlay file system or something like that. https://en.m.wikipedia.org/wiki/OverlayFS

Overlays causes reads to go through memory first followed by a real file system if the file does not exist on the in-memory fs whereas rights go to the in-memory fs.

It should be simple to see how raspbian modifies fstab. The modification would be identical on ubuntu.

As it simply file system mounting you can modify fstab so that it mouns read only by default, and also set up scripts so that you can remount the root file system read write, and read only as needed.

The scripts would make upgrading software on a production system relatively easy. Or you could have your software update code execute the necessary commands individually.

It would also be possible to set up the system in a manner where directories that are written to frequently like /var/log are on in-memory file systems will the root file system is on disk.

It may be handy to implement it so that a sequence is done after mowing has been completed where any log files are uploaded to a file server that is not on the mower for fire analysis and debugging, followed by a reboot to clear out the in-memory file system so that you do not run out of memory.

On Thu, Apr 28, 2022, 11:27 Clemens Elflein @.***> wrote:

This might be a problem indeed, since I'm using Ubuntu on the Pi. I haven't gotten ROS to work with Raspbian in a stable way. But @Schaumi19 https://github.com/Schaumi19 is right, there is an option which just writes to RAM and does not modify the SD. I haven't used it yet, though since I'm working on the software and therefore need write access to the SD

— Reply to this email directly, view it on GitHub https://github.com/ClemensElflein/OpenMower/issues/10#issuecomment-1112346757, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV3PWKZPPMGM6CEFGBZYD4LVHKU5XANCNFSM5UF4NCKQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[noobydp]

I got bored and read up on this problem, so I'm no expert.

This article suggests the problem has been resolved and may have been due to bad rpi hardware or cheap sd cards.

I've used 3 or 4 pi's over the past few years starting with a Pi B, and I've never had an issue. I've always Used genuine samdisk cards, not sure if it helps.

That said the overlay fs/ read only sd card mode is handled at the os level in raspbian so might be an option.

[scottishjohn1982]

The Amazon warehouses in the us are full o of "legitimate" sd cards from major brands that are actually produced there manufacturer but are part of lots that failed qc and somehow (through illustrated supply chains) work their way in to the Amazon warehouses and get mixed back into the legitimate supply chain.

I have a stack of sd cards that I paid good money for that I only used for projects that can run in read only mode.

There were issues with the older pis that could easily corrupt quality sd cards, but those have been resolved.

On Fri, Apr 29, 2022, 08:19 noobydp @.***> wrote:

I got bored and read up on this problem, so I'm no expert.

This article https://hackaday.com/2022/03/09/raspberry-pi-and-the-story-of-sd-card-corruption/ suggests the problem has been resolved and may have been due to bad rpi hardware or cheap sd cards.

I've used 3 or 4 pi's over the past few years starting with a Pi B, and I've never had an issue. I've always Used genuine samdisk cards, not sure if it helps.

That said the overlay fs/ read only sd card mode is handled at the os level in raspbian so might be an option.

— Reply to this email directly, view it on GitHub https://github.com/ClemensElflein/OpenMower/issues/10#issuecomment-1113246909, or unsubscribe https://github.com/notifications/unsubscribe-auth/AV3PWK3GSZK4SXGN2KOP3MDVHPHTTANCNFSM5UF4NCKQ . You are receiving this because you commented.Message ID: @.***>

[Schaumi19]

It is true that you have less problems with genuine sdcards whatever brand, but it is also a fakt that the life time of an sd card is limited tue to the write cycles(up to 10k some sources say 100k). Thats why the problem is not the normal usage. But if there is something like a log file that gets written every second you are done afer a few days. Or if the file gets writen less the problem occurs afer a longer period of time what could be worse.

[FilipBE]

Some Raspberry Pi Compute Module 4 (CM4) boards come with a built-in eMMC flash chip for storing nonvolatile data. This chip replaces the need for a separate SD card, which can be useful if you want to avoid the use of SD cards.

[JoeKae]

Just boot from a small ssd via usb. They last much longer than sdcards or emmc.

[FadeFx]

i am not sure if adding an ssd inside the smal case of a mower.

[JoeKae]

You can use small m2 type ssds.

[vguttmann]

I did open this issue because I had a (as far as I can tell) genuine Samsung 64GB SD card go bad in a Pi that a personal project ran on.

I would agree that a CM4-based mower sounds like a good idea, given the onboard eMMC flash, and that it is intended with use in industrial settings in mind. Of course, that would mean sacrificing USB 3.0 connectivity, but as far as I can see, that's not really used here. The CM4 DOES have a USB 2.0 interface, so no USB controller on the board would be necessary.

[pedromorgan]

At end of day we want machine to be 100% reliable.. with ..a failover etc...(line replacable)

maybe an SD raid ? or ssd RAID ? as @JoeKae mentions, the m2 ssd's could be useful.. but expensive..

or a stpuid raid.. https://hackaday.com/2018/02/12/worlds-stupidest-solid-state-disk-drive-hack/

[Schaumi19]

ssds last longer but have the same problem. So just dont write do the ssd/sd or only write every few days. I don't think it is a real problem, that it gets reset a few days if the battery dies.

[wooter]

I've been using Log2Ram, and endurance type SD cards. None of them have failed yet, in other applications like RPi dash cams and power monitoring.

[ClemensElflein]

We're working on having the software as docker images, so that we can use Raspbian as base system. This would allow us to use the overlayfs setting in raspi-config which should fix the sd corruption problem without much work.

[plopaur]

I would recommend looking at Tinycore Linux for Rasperry Pi. This is a minimal ramdisk only distribution with no automatic writing to sdcard which i have used in the past. A new package for openmower has to be created. Configuration changes can be written to sdcard instantly.

http://www.tinycorelinux.net/ports.html

[docgalaxyblock]

@plopaur I could not find any information if ROS runs on TinyCore. If it does and it is stable that may be a way to do it

@vguttmann @pedromorgan I am planning to build my mower based on a CM4 w/ emmc and this adapter - It has a M.2 2230/42 slot too but I think a ssd is an overkill

It will take some time cause I accidentally ordered a CM4 w/o wifi so I am returning it and need to wait until my desired variant is available again. Also I plan to take a look around the OS side of the project Stay tuned 🤗