guysoft / OctoPi

Scripts to build OctoPi, a Raspberry PI distro for controlling 3D printers over the web
GNU General Public License v3.0
2.48k stars 367 forks source link

Mounting OctoPi as Read Only #132

Open jminardi opened 9 years ago

jminardi commented 9 years ago

While I usually try to safely shut down my pi with sudo shutdown -h now sometimes power is inadvertently lost and the system does not shut down cleanly. In some of these cases if the power is lost during a write the SD card can be left corrupted.

One potential solution for preventing this corruption is to mount certain directories as read only. This can be done using fstab, more info cab be found on this thread: https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=10775&sid=186806aab50f93c6594c209e7338ec8b&start=25

Has anyone tried this?

guysoft commented 9 years ago

That would require changing the partition table on the rasbian image file, I am not sure how we can do that using qemu-arm-static.

newzealandpaul commented 9 years ago

@jminardi were you able to figure out how to do this? I am tired of my Pi's sd card being corrupted when I have to pull power.

guysoft commented 9 years ago

Nope. Also, tend to pull the power out on my Pi quite often and it did not become corrupted. I am not sure why though. @newzealandpaul What version of OctoPi are you using? the recent one seems to be really stable regarding power cuts.

jminardi commented 9 years ago

@newzealandpaul No, I have not done this yet.

FWIW I did switch brands of SD card (from Samsung EVO to ATP) and I have not had a corruption since. My users still regularly pull the power without shutting down. I read somewhere that the Samsung EVO cards were having trouble with corruption. It might be worth experimenting with different cards (I highly recommend the ATP SD cards, they are "Industrial Grade")

foosel commented 9 years ago

For the record, I haven't had a single corruption to this day (only a broken SD port, the ports on the old 1 models are really horrible...). Using Transcend 8GB Class 10 cards almost exclusively (there's a Kingston and Platinum card here somewhere too, but the usual go to cards are all Transcends).

nophead commented 9 years ago

I have had corruption problems after not shutting down properly. The symptoms where the RPI kept crashing until I re-imaged the card.

On 29 September 2015 at 14:23, Gina Häußge notifications@github.com wrote:

For the record, I haven't had a single corruption to this day (only a broken SD port, the ports on the old 1 models are really horrible...). Using Transcend 8GB Class 10 cards almost exclusively (there's a Kingston and Platinum card here somewhere too, but the usual go to cards are all Transcends).

— Reply to this email directly or view it on GitHub https://github.com/guysoft/OctoPi/issues/132#issuecomment-144057724.

newzealandpaul commented 9 years ago

Thanks everyone for the tips.

I am exclusively using Samsung EVO 16gb Micro SD card. Maybe 50% of the time the filesystem gets corrupted if the power is pulled. Then I have to boot up a Linux VM to fsck the MicroSD card ... which is a pain in the ass.

@jminardi where do you buy your ATP cards from. ATP Industrial Grade cards have all sorts of enhanced protection ... but at a crazy high cost. The Industrial Grade cards go for about US$100 here (New Zealand) for 4gb!!!! The non-industrial grade ATP cards are also very expensive, but affordable, but I can only see them being sold by DigiKey which charges a crazy high amount to ship to NZ.

@guysoft I am using a recent build (1 or 2 months old) can't tell you right now because the filesystem is corrupt. As far as I can tell, it could be run from a readonly filesystem, as long as there was a rw partition/usb stick and/or a in-memory file system to hold the uploaded STLs and gcode files. I might investigate this.

jminardi commented 9 years ago

@newzealandpaul I do indeed get my cards off of DigiKey. It is about twice the cost here in the states but it seems worth it to me.

basilfx commented 8 years ago

For what it's worth, I have created this guide on how I hardened my OctoPi installation (tried this on an old version of beginning 2015 and todays latest Jessie nightly).

My printer shares power with Raspberry Pi, so when I have to interrupt my printer, I also interrupt the Raspberry Pi. No corrupt cards until now (Sandisk 8GB class 10).

Edit: It's now June 2016, and I still run the same image + tweaks without corruption. I use my approximately twice per week and I never shut down OctoPi (I just pull the plug).

guysoft commented 8 years ago

Well done @basilfx ! Though I am not sure if we can implement it this way on OctoPi, because we can't assume users have more space on the SD card. However this is worth keeping for people reporiting SD card problems.

basilfx commented 8 years ago

@guysoft Never ment to provide something that should be integrated :-) Just for the ones who want to turn their installation into a read-only one. I probably forgot a few things, but it's a start that works good enough for me.

newzealandpaul commented 8 years ago

@guysoft https://github.com/guysoft Can a script not be integrated into OctoPi to switch it to a read only configuration?

On 25 November 2015 at 05:49, Bas Stottelaar notifications@github.com wrote:

@guysoft https://github.com/guysoft Never ment to provide something that should be integrated :-) Just for the ones who want to turn their installation into a read-only one. I probably forgot a few things, but it's a start.

— Reply to this email directly or view it on GitHub https://github.com/guysoft/OctoPi/issues/132#issuecomment-159334261.

guysoft commented 8 years ago

@newzealandpaul yes that could be done. And we could add it to the OctoPi repo. The only step I am not sure how to do is "Use sudo gparted and configure the remaining space as EXT4." Does anyone know how to automatically partition the renaming space in a non-interacting mode?

newzealandpaul commented 8 years ago

@guysoft let me see if I can figure it out tonight. The parted tool (the cli version of gparted) is probably the tool to use. fdisk can be scripted but it's horrendous.

newzealandpaul commented 8 years ago

@guysoft This works:

$ sudo parted -s /dev/mmcblk0 -- mkpart primary ext4 3400MB -1 $ sudo mkfs.ext4 /dev/mmcblk0p3

newzealandpaul commented 8 years ago

@guysoft a couple other notes

I intentionally left a few mb of space between the system and new home partition.... just in case.

In @basilfx instructions he says to run

$ sudo systemctl enable tmp.mount

In the 2015-05-05-octopi-wheezy-0.12.0.img version of the octopi distribution, this command is not available. I don't know what it does or what the alternative is.

Other than that, everything can be done on the pi itself.

basilfx commented 8 years ago

@newzealandpaul the sudo systemctl enable tmp.mount is only available on the newer Raspbian builds because these build are systemd based. I think there is a similar one for the older images :-)

guysoft commented 8 years ago

@newzealandpaul The latest nightly is running on the new Jessie, so thats not a problem.

GaryBoone commented 8 years ago

I followed the linked directions, but my RPi failed to boot. A couple of notes:

  1. Octopi is currently set to automatically automatically resize to full SD size on first boot. That explains why I had to shrink the partition to make space for a new rw partition.
  2. It looks like the directions about resolv.conf may not be right anymore? The current octopi, based on Jessie, uses a new package called resolvconf that apparently works with ro filesystems.

I think the project of creating a read-only distribution is extremely valuable. There's just no reason to suffer starting over due to failed sd cards when the OS can be configured to avoid them.

basilfx commented 8 years ago

@GaryBoone

Octopi is currently set to automatically automatically resize to full SD size on first boot. That explains why I had to shrink the partition to make space for a new rw partition.

I edited the guide to tell you to boot it at least once and remount it for read-write afterwards :-)

It looks like the directions about resolv.conf may not be right anymore? The current octopi, based on Jessie, uses a new package called resolvconf that apparently works with ro filesystems.

Works for me. resolvconf is shipped with Jessie. It generates/manages /etc/resolv.conf which is on the read-only file system by default. By symlinking it to /var/run/resolv.conf and telling resolvconf to update that file, it workaround the read-only restriction.

PProvost commented 8 years ago

You guys might consider looking at MotionEyeOS which mounts part of the filesystem readonly, but not all of it, leaving the Pi safe to simply power off at will.

https://github.com/ccrisan/motioneyeos

fabianhu commented 7 years ago

Hi, have found some very easy-to-apply solution: http://wiki.psuter.ch/doku.php?id=solve_raspbian_sd_card_corruption_issues_with_read-only_mounted_root_partition It is working with a transparent "RAM overlay" where all changed data is stored in RAM. It is also very easy to apply (shell script provided) and also easily to be reverted again.

You can add an USB-Stick for permanent storage of gcode and video: udo apt-get install usbmount and change change this line of etc/usbmount/usbmount.conf to FS_MOUNTOPTIONS="-fstype=vfat,uid=pi,gid=pi,dmask=0007,fmask=0111

Then change the storage locations in the octoprint settings to /media/usb/

The latter settings of course before making the SD card read-only.

(edited mount masks to avoid internal server error 500)

subatomicglue commented 3 years ago

Did this ever become a script, or an option I could enable on the octopi distribution? Would like to harden my octopi against corruption. Seems like an obviously desirable design goal.

Currently I use a battery backup to avoid it ever losing power, side benefit is that brownouts won’t affect my prints either.

guysoft commented 3 years ago

Hey, @subatomicglue it wasn't AFAIK, however someone has added a readonly module to CustomPiOS: https://github.com/guysoft/CustomPiOS/tree/devel/src/modules/readonly

So it should be possible to build an OctoPi readonly variant.

GaryBoone commented 3 years ago

Actually, it's now built in to Raspian and much easier to use. It's just an option in raspi-config, under Advanced Options | OverlayFS. Learn more about it in this thread https://www.raspberrypi.org/forums/viewtopic.php?f=63&t=253104&p=1549229#p1549117. There's some demonstration in this video https://www.youtube.com/watch?v=jjwt9GZGELo which shows that it can be configured via GUI or raspi-config. Finally, this page https://yagrebu.net/unix/rpi-overlay.md shows the script that was added into Raspian [So note that you do not install that script!]. But do notice the little overctl script at the bottom. It's a convenient way to flip between read-write and readonly from the command line. I'm surprised I couldn't find clearer documentation just now given that a year has passed since I last set these up. Maybe a little more searching will be fruitful.

Thanks, Gary

On Thu, Dec 24, 2020 at 9:36 AM Guy Sheffer notifications@github.com wrote:

Hey, @subatomicglue https://github.com/subatomicglue it wasn't AFAIK, however someone has added a readonly module to CustomPiOS: https://github.com/guysoft/CustomPiOS/tree/devel/src/modules/readonly

So it should be possible to build an OctoPi readonly variant.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/guysoft/OctoPi/issues/132#issuecomment-750933147, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGRX56XHTLLJ7AEXHTT6T3SWN32VANCNFSM4BMMSFQA .

subatomicglue commented 3 years ago

Good to know, thanks. So I guess the remaining question (if any) then, would be, is octopi/print compatible with this readonly fs...?
I'll try to find some time to test.
But that's exciting to hear!

I also wonder if an octoprint plugin could flip the flag in order to give the option to switch in the octoprint GUI... I think raspi-config can be manipulated via command line.

GaryBoone commented 3 years ago

Yes to both.

I've used the Octopi in ro mode since setting it up. IIRC, ro mode only makes the boot partition read-only; the rest of the file system is writable, so files can be written.

Yes, raspi-config settings can be manipulated from the command line. But I'm sure if that command is set up to work from the a command line invocation. But no matter; I just use overctl in my scripts, like : "/usr/local/sbin/overctl -w" to make writable.

Thanks, Gary

On Thu, Dec 24, 2020 at 12:10 PM km notifications@github.com wrote:

Good to know, thanks. So I guess the question then, would be, is octopi/print compatible with this readonly fs...? I'll try to find some time to test. But that's exciting to hear!

I also wonder if an octoprint plugin could flip the flag in order to give the option to switch in the octoprint GUI... I think raspi-config can be manipulated via command line.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/guysoft/OctoPi/issues/132#issuecomment-750969440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGRX5ZI5VNLXD6SVAC3RQ3SWON3PANCNFSM4BMMSFQA .