常见版本的rootfs无法使用OSTree进行管理，为了使用OSTree管理Layerscape的rootfs升级，需要针对性的对rootfs进行改造，同时需要对uboot的环境变量和initramfs进行更改，接下来，我们一步步介绍如何进行修改。

SD卡分区

升级管理基于SD卡进行，因此需要对SD卡进行分区，分区信息如下：

[sudo] password for quanwen: 
Disk /dev/sdb: 30.26 GiB, 32480690176 bytes, 63438848 sectors
Disk model: STORAGE DEVICE  
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x5dc644d9

Device     Boot   Start      End  Sectors  Size Id Type
/dev/sdb1         20480  1044479  1024000  500M  b W95 FAT32
/dev/sdb2       1044480 63438847 62394368 29.8G 83 Linux

第一个分区起始地址为20480，原因是前面的空间用于刷写uboot。第一个分区刷写制作的FIT格式的Image镜像，第二个分区存放OSTree格式的rootfs。刷写uboot： sudo dd if=./firmware_ls1046ardb_uboot_sdboot_1133_5a59.img of=/dev/sdc bs=512 seek=8 conv=fsync

制作OSTree格式的rootfs

服务器端

服务器端使用ostree创建仓库管理要更新的rootfs，这里介绍ostree服务器的搭建示例，分为几个步骤：

创建OSTree仓库
```
cd target_directory
```

创建repo文件夹用于存放ostree配置

mkdir repo ostree --repo=repo init --mode=archive-z2 # 创建模式为archive-z2的OSTree格式仓库 mkdir rfs # rfs文件夹用于存放要更新的rootfs

2. 改造编译后输出的rootfs
改造主要参考： https://ostreedev.github.io/ostree/deployment/ ,将编译好的rootfs解压到上一步创建的rfs文件夹，改造包含三个部分：
* 移动 `/etc`到`/usr`目录下
![image](https://github.com/robertwenhk/blog/assets/143626366/3f3cc193-380d-49c6-8f2a-15f18a658faf)
* 将kernel镜像、initramfs和设备树移动到`/usr/lib/modules/$version`目录下，并改名为`vmlinuz`, `initramfs.img`和`devicetree`，其中initramfs和设备树不是必须选项，但是注意，不能是符号链接文件，我这里只放了kernel和设备树
![image](https://github.com/robertwenhk/blog/assets/143626366/49c61ebe-6381-4bd4-899a-d2eff4048c96)

![image](https://github.com/robertwenhk/blog/assets/143626366/63a0b121-99bc-4c57-ba11-402177dae5d1)
* 在这个rootfs的根文件目录下创建`sysroot`目录，并初始化为OSTree的fs，缺少这个步骤，在启动时会因为无法move而导致initramfs执行失败而无法启动
```bash
# enter rfs
mkdir sysroot -p
sudo ostree admin init-fs sysroot

提交rfs到OSTree服务器仓库

sudo ostree --repo=repo commit --branch=master --subject="rootfs v1" rfs

启动http服务器，开放OSTree仓库（这里不考虑安全问题，不使能HTTPs，而只用HTTP）
```
python -m http.server 8080 --directory repo
```
这样，服务端的准备工作就完成了。

板端配置

我们的板子测试使用SD卡完成，因此这些操作现在SD卡上完成后，再通过SD卡启动测试。前面已经对SD卡分好了区，现在需要对SD卡的rootfs分区进行操作：
建立OSTree的sysroot 不能直接使用原始的rootfs，并在rootfs的基础上创建sysroot，否则会导致在initramfs启动的时候，出现双层的sysroot而无法切换，正确的做法应该是：
- 格式化rootfs分区：
```
sudo mkfs.ext4 -L "rootfs" -b 4096 /dev/sdb2
```
- 部署OSTree的sysroot，这里这个文件夹名称不影响，你也可以变更为sysroot
```
sudo ostree admin init-fs rootfs
```
  *将刚才设置的服务器 ostree repo设置为remote repo
```
sudo ostree remote add --repo=rootfs/ostree/repo --no-gpg-verify origin http://localhost:8080/ 
```
- 拉取已经改造好的rootfs
```
sudo ostree pull --repo=rootfs/ostree/repo origin:master && sync
```
- 部署系统，根据OSTree的说明，可以部署多个系统，但是我们这里只需要一个，因此，直接部署名称为TCU的OS
```
sudo ostree admin os-init tcu --sysroot=rootfs # 创建名为tcu的os
sudo ostree admin --sysroot=rootfs --os=tcu deploy origin:master # 真正部署OS，这个操作会将etc从usr拿出来
```
- 升级系统
```
sudo ostree admin --sysroot=rootfs --os=tcu upgrade
```
  至此，SD卡的rootfs已经制作完成了。在这个过程中，任何一步都可以通过sudo ostree admin --sysroot=rootfs status查看状态的变化。但是，此时仍然不能直接用于启动，因为原始的initramfs是无法解析这个OSTree格式的rootfs的，我们需要对initramfs的脚本进行替换。
  
  initramfs和uboot修改
  
  initramfs修改
  
  initramfs需要替换为meta-updater官方仓库的init脚本：
```
/* -*- c-file-style: "gnu" -*-
```
- Switch to new root directory and start init.
- Copyright 2011,2012,2013 Colin Walters walters@verbum.org
- Based on code from util-linux/sys-utils/switch_root.c,
- Copyright 2002-2009 Red Hat, Inc. All rights reserved.
- Authors:
- Peter Jones pjones@redhat.com
- Jeremy Katz katzj@redhat.com
- Relicensed with permission to LGPLv2+.
- SPDX-License-Identifier: LGPL-2.0+
- This library is free software; you can redistribute it and/or
- modify it under the terms of the GNU Lesser General Public
- License as published by the Free Software Foundation; either
- version 2 of the License, or (at your option) any later version.
- This library is distributed in the hope that it will be useful,
- but WITHOUT ANY WARRANTY; without even the implied warranty of
- MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- Lesser General Public License for more details.
- You should have received a copy of the GNU Lesser General Public
- License along with this library; if not, write to the
- Free Software Foundation, Inc., 59 Temple Place - Suite 330,
- Boston, MA 02111-1307, USA. */

include "config.h"

include <sys/mount.h>

include <sys/types.h>

include <sys/stat.h>

include <sys/param.h>

include <sys/syscall.h>

include

include "ostree-mount-util.h"

/ Initialized early in main / static bool running_as_pid1;

static char resolve_deploy_path (const char root_mountpoint) { char destpath[PATH_MAX]; struct stat stbuf; char ostree_target, deploy_path;

ostree_target = read_proc_cmdline_ostree (); if (!ostree_target) errx (EXIT_FAILURE, "No OSTree target; expected ostree=/ostree/boot.N/...");

snprintf (destpath, sizeof(destpath), "%s/%s", root_mountpoint, ostree_target); if (lstat (destpath, &stbuf) < 0) err (EXIT_FAILURE, "Couldn't find specified OSTree root '%s'", destpath); if (!S_ISLNK (stbuf.st_mode)) errx (EXIT_FAILURE, "OSTree target is not a symbolic link: %s", destpath); deploy_path = realpath (destpath, NULL); if (deploy_path == NULL) err (EXIT_FAILURE, "realpath(%s) failed", destpath); / Quiet logs if there's no journal / if (!running_as_pid1) printf ("Resolved OSTree target to: %s\n", deploy_path); return deploy_path; }

static int pivot_root(const char new_root, const char put_old) { return syscall(__NR_pivot_root, new_root, put_old); }

int main(int argc, char argv[]) { / If we're pid 1, that means there's no initramfs; in this situation

various defaults change:
- Assume that the target root is /
- Quiet logging as there's no journal
etc. */ running_as_pid1 = (getpid () == 1);

const char *root_arg = NULL; bool we_mounted_proc = false; if (running_as_pid1) { root_arg = "/"; } else { if (argc < 2) err (EXIT_FAILURE, "usage: ostree-prepare-root SYSROOT"); root_arg = argv[1]; }

struct stat stbuf; if (stat ("/proc/cmdline", &stbuf) < 0) { if (errno != ENOENT) err (EXIT_FAILURE, "stat(\"/proc/cmdline\") failed"); /* We need /proc mounted for /proc/cmdline and realpath (on musl) to
- work: */ if (mount ("proc", "/proc", "proc", 0, NULL) < 0) err (EXIT_FAILURE, "failed to mount proc on /proc"); we_mounted_proc = 1; }
const char root_mountpoint = realpath (root_arg, NULL); if (root_mountpoint == NULL) err (EXIT_FAILURE, "realpath(\"%s\")", root_arg); char deploy_path = resolve_deploy_path (root_mountpoint);

if (we_mounted_proc) { / Leave the filesystem in the state that we found it: / if (umount ("/proc")) err (EXIT_FAILURE, "failed to umount proc from /proc"); }

/* Work-around for a kernel bug: for some reason the kernel
refuses switching root if any file systems are mounted
MS_SHARED. Hence remount them MS_PRIVATE here as a
work-around.
https://bugzilla.redhat.com/show_bug.cgi?id=847418 */ if (mount (NULL, "/", NULL, MS_REC|MS_PRIVATE, NULL) < 0) err (EXIT_FAILURE, "failed to make \"/\" private mount");

/ Make deploy_path a bind mount, so we can move it later / if (mount (deploy_path, deploy_path, NULL, MS_BIND, NULL) < 0) err (EXIT_FAILURE, "failed to make initial bind mount %s", deploy_path);

/* chdir to our new root. We need to do this after bind-mounting it over
itself otherwise our cwd is still on the non-bind-mounted filesystem
below. */ if (chdir (deploy_path) < 0) err (EXIT_FAILURE, "failed to chdir to deploy_path");

/* Default to true, but in the systemd case, default to false because it's handled by
ostree-system-generator. */ bool mount_var = true;

ifdef HAVE_SYSTEMD_AND_LIBMOUNT

mount_var = false;

endif

/ file in /run can override the default behaviour so that we definitely mount /var / if (lstat (INITRAMFS_MOUNT_VAR, &stbuf) == 0) mount_var = true;

/ Link to the deployment's /var / if (mount_var && mount ("../../var", "var", NULL, MS_MGC_VAL|MS_BIND, NULL) < 0) err (EXIT_FAILURE, "failed to bind mount ../../var to var");

char srcpath[PATH_MAX]; /* If /boot is on the same partition, use a bind mount to make it visible
at /boot inside the deployment. */ snprintf (srcpath, sizeof(srcpath), "%s/boot/loader", root_mountpoint); if (lstat (srcpath, &stbuf) == 0 && S_ISLNK (stbuf.st_mode)) { if (lstat ("boot", &stbuf) == 0 && S_ISDIR (stbuf.st_mode)) { snprintf (srcpath, sizeof(srcpath), "%s/boot", root_mountpoint); if (mount (srcpath, "boot", NULL, MS_BIND, NULL) < 0) err (EXIT_FAILURE, "failed to bind mount %s to boot", srcpath); } }

/ Do we have a persistent overlayfs for /usr? If so, mount it now. / if (lstat (".usr-ovl-work", &stbuf) == 0) { const char usr_ovl_options[] = "lowerdir=usr,upperdir=.usr-ovl-upper,workdir=.usr-ovl-work";

/* Except overlayfs barfs if we try to mount it on a read-only
- filesystem. For this use case I think admins are going to be
- okay if we remount the rootfs here, rather than waiting until
- later boot and systemd-remount-fs.service. */ if (path_is_on_readonly_fs (".")) { if (mount (".", ".", NULL, MS_REMOUNT | MS_SILENT, NULL) < 0) err (EXIT_FAILURE, "failed to remount rootfs writable (for overlayfs)"); }
  
  if (mount ("overlay", "usr", "overlay", 0, usr_ovl_options) < 0) err (EXIT_FAILURE, "failed to mount /usr overlayfs"); } else { / Otherwise, a read-only bind mount for /usr / if (mount ("usr", "usr", NULL, MS_BIND, NULL) < 0) err (EXIT_FAILURE, "failed to bind mount (class:readonly) /usr"); if (mount ("usr", "usr", NULL, MS_BIND | MS_REMOUNT | MS_RDONLY, NULL) < 0) err (EXIT_FAILURE, "failed to bind mount (class:readonly) /usr"); }
/* We only stamp /run now if we're running in an initramfs, i.e. we're
not pid 1. Otherwise it's handled later via ostree-system-generator.
https://mail.gnome.org/archives/ostree-list/2018-March/msg00012.html
https://github.com/ostreedev/ostree/pull/1675 */ if (!running_as_pid1) touch_run_ostree ();

if (strcmp(root_mountpoint, "/") == 0) { /* pivot_root rotates two mount points around. In this instance . (the
- deploy location) becomes / and the existing / becomes /sysroot. We
- have to use pivot_root rather than mount --move in this instance
- because our deploy location is mounted as a subdirectory of the real
- sysroot, so moving sysroot would also move the deploy location. In
- reality attempting mount --move would fail with EBUSY. / if (pivot_root (".", "sysroot") < 0) err (EXIT_FAILURE, "failed to pivot_root to deployment"); } else { / In this instance typically we have our ready made-up up root at
- /sysroot/ostree/deploy/.../ (deploy_path) and the real rootfs at
- /sysroot (root_mountpoint). We want to end up with our made-up root at
- /sysroot/ and the real rootfs under /sysroot/sysroot as systemd will be
- responsible for moving /sysroot to /.
- We need to do this in 3 moves to avoid trying to move /sysroot under
- itself:
- 1. /sysroot/ostree/deploy/... -> /sysroot.tmp
- 1. /sysroot -> /sysroot.tmp/sysroot
- 1. /sysroot.tmp -> /sysroot */ if (mkdir ("/sysroot.tmp", 0755) < 0) err (EXIT_FAILURE, "couldn't create temporary sysroot /sysroot.tmp");
  if (mount (deploy_path, "/sysroot.tmp", NULL, MS_MOVE, NULL) < 0) err (EXIT_FAILURE, "failed to MS_MOVE '%s' to '/sysroot.tmp'", deploy_path);
  
  if (mount (root_mountpoint, "sysroot", NULL, MS_MOVE, NULL) < 0) { system("ls -al /sysroot"); system("ls -al /sysroot.tmp"); sleep(3); err (EXIT_FAILURE, "failed to MS_MOVE '%s' to 'sysroot', errno:%d", root_mountpoint, errno); }
  
  printf("last step passes\n"); if (mount (".", root_mountpoint, NULL, MS_MOVE, NULL) < 0) err (EXIT_FAILURE, "failed to MS_MOVE %s to %s", deploy_path, root_mountpoint); }
/* The /sysroot mount needs to be private to avoid having a mount for e.g. /var/cache
also propagate to /sysroot/ostree/deploy/$stateroot/var/cache
Now in reality, today this is overridden by systemd: the actual way we fix this up
is in ostree-remount.c. But let's do it here to express the semantics we want
at the very start (perhaps down the line systemd will have compile/runtime option
to say that the initramfs environment did everything right from the start). */ if (mount ("none", "sysroot", NULL, MS_PRIVATE, NULL) < 0) err (EXIT_FAILURE, "remounting 'sysroot' private");

if (running_as_pid1) { execl ("/sbin/init", "/sbin/init", NULL); err (EXIT_FAILURE, "failed to exec init inside ostree"); } else { exit (EXIT_SUCCESS); } }

具体，可以到meta-updater仓库去查看：https://github.com/advancedtelematic/meta-updater/blob/master/recipes-sota/ostree-initrd/files/init.sh
同时，因为initramfs中用到了ostree-prepare-root这个二进制，因此需要在initramfs中集成这个二进制，源码在OSTree官方仓库中：https://github.com/ostreedev/ostree/blob/v2018.8/src/switchroot/ostree-prepare-root.c
这里建议把ostree-remount也集成到initramfs中。
## uboot环境变量修改
uboot中要对`bootargs`环境变量进行修改，改为实际rootfs中的boot和哈希对应的rootfs，这里第一次需要手动去sd卡确定。

setenv bootargs "earlycon=uart8250,mmio,0x21c0500 console=ttyS0,115200 root=/dev/ram0 rw rootfstype=ext4 rootwait rootdelay=2 ostree=ostree/boot.0/tcu/cb774860fd0fe8232bae38d000e4dad985e35a39a95cf98b160bf0d9b2025590/0 ostree_root=/dev/mmcblk0p2"


集成好后，就可以启动了：
![image](https://github.com/robertwenhk/blog/assets/143626366/088ff4cf-86dd-40f9-834a-d50e12413c9c)
后续在升级过程中，每次改变的都是boot.N中的N，哈希值和最后的0是不变的，因为这是个链接路径，指向实际的rootfs，因此需要通过改变uboot的环境变量的方式来实现自动化升级，这部分工作放到下一个帖子中进行说明。

# 过程中遇到的问题
1. 双层sysroot问题
![image](https://github.com/robertwenhk/blog/assets/143626366/fb04b397-7b12-4a57-8ca2-78bbea76c0d1)
这个问题就是因为SD卡中使用了原生的rootfs，并在原生rootfs下创建sysroot导致的
2. 无法movesysroot
![image](https://github.com/robertwenhk/blog/assets/143626366/207c9421-ab0b-4ff3-9b65-2bb001d96265)
这是因为改造rootfs时，没有在编译后的rootfs中创建OSTree的sysroot，导致在move的时候找不到sysroot

quinnwencn / blog

03 制作Layerscape的OSTree管理的rootfs #11

SD卡分区

制作OSTree格式的rootfs

服务器端

创建repo文件夹用于存放ostree配置

板端配置

initramfs和uboot修改

initramfs修改

include "config.h"

include <sys/mount.h>

include <sys/types.h>

include <sys/stat.h>

include <sys/param.h>

include <sys/syscall.h>

include

include

include

include

include

include

include

include

include

include

include "ostree-mount-util.h"

ifdef HAVE_SYSTEMD_AND_LIBMOUNT

endif