search

guysoft / CustomPiOS

A Raspberry Pi and other ARM devices distribution builder
GNU General Public License v3.0
509 stars 146 forks source link

many open loops/mount after build_dist #75

Closed steviehs closed 3 years ago

steviehs commented 4 years ago

I am working the first time with CustomPiOS and I am happy it's there. I just need to understand the concept a little bit more: I guess it is not mentioned, but it seems, that for most of the stuff you need to start either with sudo or as root because of all the loop mounts. After mydist/src/build_dist, there are many mounts ongoing. Is this intended? Also: build_dist does not produce a zip file, but qemu_boot.sh is looking for one. Is there a reason for that or is it just historical? I also do only partly understand the workflow idea: in my first sight, it seemed the modules should be used for a single shot setup: this means, on subsequent calls of mydist/src/build_dist the image should be always generated from the vanilla raspian, but this seems not to be the case? Does this mean, that you intend another workflow? If I get some enlightenment, I could at least contribute to the documentation and/or some script enhancements :-)

guysoft commented 4 years ago

Hey, Yest its intended. Because you want to mount the images /boot and the / and the image. Then /dev /proc. I suggest you use the docker build method, because if something goes wrong with the mouths killing the container tends to fix it all up. https://github.com/guysoft/CustomPiOS/wiki/Building-with-Docker

It would be great if you can contribute.

More info about these virtual file systems: https://opensource.com/article/19/3/virtual-filesystems-linux

steviehs commented 4 years ago

Yest its intended. Because you want to mount the images /boot and the / and the image. Then /dev /proc.

So the workflow idea is, after the build_dist I do what with those mounts? Right now I am not sure, if I'd prefer to go with the scripted build directly or with docker. For the scripted building I see the need for a cleanup script (to ummount and stop loops) and to zip for getting the similar base from what I get from the docker build.

Still I am not sure, what your intended workflow is for subsequent builds?

I suggest you use the docker build method, because if something goes wrong with the mouths killing the container tends to fix it all up. https://github.com/guysoft/CustomPiOS/wiki/Building-with-Docker

I was not successful with that page, but with https://github.com/guysoft/CustomPiOS/wiki/Building and corrected there what was necessary to have docker building a distro.

guysoft commented 4 years ago

So the workflow idea is, after the build_dist I do what with those mounts? Right now I am not sure, if I'd prefer to go with the scripted build directly or with docker. For the scripted building I see the need for a cleanup script (to ummount and stop loops) and to zip for getting the similar base from what I get from the docker build.

When the build finishes or fails they should unmount. It might not work if you break it somehow (many ways you could) or if you run to builds that clash somehow.

This is why I suggest you work with the docker container. Mounts can get stuck and if they do then its easier to manage them with the container. If you are doing subsequent make sure all mounts are clear from the image. Look at mount command with no arguments to see what is mounted. Make sure no loopback mounts (the mounts in the workspace folder) are present.

I was not successful with that page, but with https://github.com/guysoft/CustomPiOS/wiki/Building and corrected there what was necessary to have docker building a distro.

Can you say where you got stuck? That could help me improve that doc.

In general the docker, or vagrant build methods are there to resolve these issues you are mentioning (vagrant would also work, but it might also affect build speed). I know its another abstraction layer, but its a really beneficial one, because getting stuck with floating loopback mounts after you for example delete the image could cause issues to your main filesystem. Its worth it for that at the least. You could use the scripts without any container or virtualization. But then we don't have any grantee that you enviromen is the same. I don't know what distribution you are using, what version of sfdisk, etc. Those things might lead to build issues, nor can I support all the distros out there.

steviehs commented 4 years ago

When the build finishes or fails they should unmount. It might not work if you break it somehow (many ways you could) or if you run to builds that clash somehow.

Ok I found the reason why: in my module I install a package with apt install, which restarts syslogd and cron. Both are not stopped after build (why?) and they still occupy dev and therefore nothing could be unmounted. From what I understand restarting those services should not lead to that, is there a way to shut the built system down from a module more secure? BTW: this leads also to the behavior in a docker build, that there is no zip compression at the end.

Regarding docker build documentation:

Can you say where you got stuck? That could help me improve that doc.

I have improved https://github.com/guysoft/CustomPiOS/wiki/Building already, so that it works. The other page https://github.com/guysoft/CustomPiOS/wiki/Building-with-Docker was not really understandable for me.

steviehs commented 3 years ago

I stopped all deamons in end chroot script and this seems to work.