A/B partition setup

[mrbluecoat]

The casync blog post indicates

An invocation like this could be typically used by IoT systems with an A/B partition setup

casync extract http://example.com/myimage.caibx --seed=/dev/sda1 /dev/sda2

Your README indicates

Extract an image directly onto a block device. The -k or --in-place option is needed.

desync extract -k -s /mnt/store image.caibx /dev/sdc

Does desync support extracting from a URL and seeding from one partition to another? Also, is there a quickstart guide for creating an image.caibx and switching the bootloader to use the new partition?

[folbricht]

There isn't really a full guide on it, but there are examples in the README.md on each step. You'd create a .caibx (index) file with the make command. Then re-assemble it from the index file (and a local or remote chunk store) with extract. Since you mentioned A/B, perhaps you're looking for the --seed option. With that, you can use an existing image (A) plus the caibx, to create an image (B). If there are chunks in B that are already present in A, it'll use those to build B and only load from the chunk store what is not already in A.

Imagine you have two files, imageA and imageA.caibx in the same directory on the target. You copied imageB.caibx to the target and want to extract imageB from it, you could run:

desync extract -s http://host/store --seed imageA.caibx  imageB.caibx imageB

It'll only load the chunks that aren't already in A from the remote chunk store.

[mrbluecoat]

Thanks @folbricht - I'm just starting with desync so I really appreciate the help and patience. What's the best way to image an existing Debian installation on eMMC? Can you just do something like desync make -s /tmp/img index.caibx / or would you need a tool like mender-convert ?

[folbricht]

Depends on what format you have the data in. make and extract operate on blobs, so in your case an image perhaps, not a filesystem. To pack a whole filesystem, you'd have to use the tar and untar sub-commands which are a bit slower.

[mrbluecoat]

I see. So basically: 1) boot "golden unit" from USB/SD 2) create image of eMMC with dd or mkosi or Armbian build 3) desync make that image file and upload to server 4) clients then desync extract -k

Is that about right for a single partition setup? Is there any danger of bricking with a partial download or extract?

In an A/B setup I'm assuming after step 4 I would 5) update u-boot to point to the updated partition and 6) reboot

Is there a way to push notify the clients to update or do they just periodically poll with cron?

[charles-dyfis-net]

.caibx files are pretty small (relative to whatever your system image size is) -- that makes them pretty straightforward to poll for updates (over whatever transport you choose) and compare against whatever version one already has.

[mrbluecoat]

That's a good point as well @charles-dyfis-net

[folbricht]

A few notes:

• If you use -k then the binary is assembled in-place, meaning if that process is interrupted, the target file is corrupted. Without -k, it'll extract into a tempfile and then atomically move that into the right place which is much safer. If you want to be sure it's correct at all times, don't use -k. desync checks the integrity of every chunk during extract

• Since you mentioned HTTP for the chunk store, you might find the index-server command useful. That one basically makes caibx files available over HTTP. Might save you from having to copy it yourself. You could have an index server and a chunk server running and use both.

• There isn't really any notification feature in desync. You'd have to wire this up separately. desync should return 0 on success and non-0 on failure.

[om26er]

The general expectation of an A/B partition setup is:

1. Read chunks from a local partition (A, lets say /dev/mmcblk0p2)
2. Read chunk from a remote store (like S3)
3. Construct and write the image on to another local partition (B, /dev/mmcblk0p3)

So this brings me to the question, what would it take to have that sort of setup working with desync ?

[om26er]

So the question really is it possible to use a block device (/dev/mmcblk0p2) as seed, if so, how ?

[folbricht]

That should also work already, but you would have to make sure that the filenames for the seed file and the seed device are named correctly. Basically when you provide a seed like /path/to/seed.caibx, desync will try to use /path/to/seed (without the extension) as the seed. In your case, it might be best to use symlinks to handle this. /path/to/A -> /dev/mmcblk0p2 /path/to/A.caibx == <index-file> Then provide --seed /path/to/A.caibx to the command

[om26er]

@folbricht That worked and in my testing desync is much faster than casync when downloading chunks from S3, probably due to it's parallel downloads.

[bearrito]

@folbricht Question. I thought my mental model was correct, but further testing on my end shows some misunderstanding.

Today we create a compressed .RAW image. We write to a partition with the below command. This is on device. SKIP_BLOCKS is the size of mbr

xz -d -c $IMAGE_PATH | sudo dd skip=$SKIP_BLOCKS of=$PARTITION

I believed that doing the following would make a corresponding index and chunks. This is on the server

 xz -d -c "$compressed_image_path" | dd skip="$SKIP_BLOCKS" of="$base_image_path"

desync make -s "$CHUNK_STORE_PATH" "$CHUNK_STORE_PATH/indices/$base_image_name.caibx" "$base_image_path"

However when we perform an verify-index I end up with a

Error: seed index for /dev/sda3 doesn't match its data

I'm trying to sort out if this a misunderstanding on my part, something about our .RAW file or code issue.

[folbricht]

Can you show the command that gives you the error message? The verify-index one?

I assume it's using a block-device as seed. There is some special handling of block devices so it's possible there's a bug in there when using it as seed. It's also possible that something has modified the seed after the index was created. Does the same happen when you try to use the device as seed for the extract command?

I'd first rule out that the content of the device has been modified by comparing it (either dd into a file or compare it to the original)

[bearrito]

@folbricht Yeah there are a few issues I think working against me, but please allow me to ask some question before I waste your time with more info.

From the OP post on this thread they quote the blogpost

casync extract http://example.com/myimage.caibx --seed=/dev/sda1 /dev/sda2

So my mental model is that this works with a single block-device, but with two partitions. Is that correct? @om26er seems to be in a similar situation, and it appeared to work?

In our case we have the below. We alternate between sda{2,3} those are our A/B. sda4 is an overlay

scrubbed ~$ lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINTS
sda      8:0    0 119.2G  0 disk 
├─sda1   8:1    0     2G  0 part /boot_scrubbed
├─sda2   8:2    0    20G  0 part 
├─sda3   8:3    0    20G  0 part /media/ro
└─sda4   8:4    0  77.2G  0 part /media/rw

I will try the DD'ing out the image and comparing. There was some belief in my team that DD'ing to a disk then back off would give a different result. That was surprising to me. I'll try that later today.

[bearrito]

@folbricht I put more work into this.

I think what I learned is that that verifying a partition generally isn't feasible. Whether written with dd or extract Namely, unless the partition is the same size as the written image there will be bytes at the end that invalidate the verification. To workaround this we are re-indexing the partition after being written. This still enables us to use the partition as a seed, at the cost of a re-index (our device is powerful enough to do this without being to problematic)

Maybe verify-index and/or even extract could support fields like --skip-bytes (for skipping some initial segments on the index similar to DD ) to write and --until to verify only to certain number of bytes of the image making it very similar to https://linux.die.net/man/1/cmp

[folbricht]

Ignoring extra bytes after the end of the index should probably even be the default for block devices. It should be fine to add this as an option to verify-index.

As for allowing an offset when using extract, I'll need to look at how complex that'd be. Seems fine at first glance. Would an option like that be of any use for files or does it only make sense for block devices?

[charles-dyfis-net]

As an alternative that doesn't require code changes, a user could wrap their underlying block device with a loop device having offset and length specified.

[bearrito]

@folbricht Thanks for looking. I guess I was mainly trying to ensure I hadn't used a foot-gun on myself. It seems odd that others seem to have this working with partitions (not block devices) but didn't encounter any issues.

If it doesn't seem high-priority to you,I could contribute it back. I've been working on another feature we discussed before which was the throttling of network traffic.

@charles-dyfis-net I'm not sure I've seen that done. Do you have an example?