search

OE4T / meta-mender-community

Community supported integration layers for Mender on various boards
Apache License 2.0
5 stars 5 forks source link

Nano updates for master-next #5

Closed madisongh closed 3 years ago

madisongh commented 3 years ago

These changes track the master branch in meta-tegra, which is now at L4T R32.4.4 and has support for the Nano 2GB devkit.

The flash layout fix for jetson-nano-qspi-sd also needs back-porting to dunfell, since the problem occurs with L4T R32.4.3 as well.

madisongh commented 3 years ago

@dwalkes I've updated tegra-demo-distro master with the R32.4.4 changes ported from my test distro. I did find one more update to the custom flash layouts here which I merged in this morning prior to the demo distro update.

I'll try to get through the rest of the flash layouts this week and work on a backport of these to dunfell and upstream PR.

Thanks!

dwalkes commented 3 years ago

I did find one more update to the custom flash layouts here

I noticed this at https://github.com/OE4T/meta-mender-community/pull/6, thanks for catching it.

I've also updated https://github.com/OE4T/meta-tegra/wiki/L4T-Integration-Issues to reference headaches with flash.xml layouts.

dwalkes commented 3 years ago

I'll try to get through the rest of the flash layouts this week

See updated sheet at https://docs.google.com/spreadsheets/d/14zVltS2LTwCNywoje6_8wzFzHQ3sYVhY6C7T0DkOnx0/edit#gid=2052166477 and r32.4.3 tabs. I think this covers all configurations we support. I've tried to make it easier to keep up up with this going forward by including some xml parsing commands that can generate the tables automagically.

Can you refresh my memory about why the tegra194 and tegra210 directories exist at https://github.com/mendersoftware/meta-mender-community/tree/dunfell/meta-mender-tegra/recipes-bsp/tegra-binaries/mender-custom-flash-layout ? I didn't check these.

I found one issue, it looks like NVIDIA moved VER partitions on tx2/nano-emmc and these now overlap the uboot area they previously advised for r32.3.1. I've added a comment at here to ask about this, I'm tempted to try to change the default layout related to VER partitions to match NVIDIA, at least for jetson-nano-emmc. Not sure if we should try this for tx2 or not. At minimum I think we should probably update the flash layouts to include the UBENV partition with start offset to make these align better with the new definitions from nvidia.

and work on a backport of these to dunfell and upstream PR.

See https://github.com/mendersoftware/meta-mender-community/pull/188

dwalkes commented 3 years ago

Can you refresh my memory about why the tegra194 and tegra210 directories exist at https://github.com/mendersoftware/meta-mender-community/tree/dunfell/meta-mender-tegra/recipes-bsp/tegra-binaries/mender-custom-flash-layout

The t194 is for AGX Xavier jetson-xavier. I added a tab for that one. Shouldn't this line be RECROOTFSSIZE? Looks like it was set with a fixed value during r32.3.1 here.

Still not sure when the tegra210 file would be used. Would that be for jetson-tx1? Do we actually support that one? I've never tested myself with this platform and don't have a TX1. It was originally added with https://github.com/mendersoftware/meta-mender-community/commit/f9e465c05b293715526ad721f702943d160c90e1

madisongh commented 3 years ago

The t194 is for AGX Xavier jetson-xavier. I added a tab for that one. Shouldn't this line be RECROOTFSSIZE? Looks like it was set with a fixed value during r32.3.1 here.

Yes, tegra194 is for any machine with the Xavier SoC (and for which we don't have a machine-specific layout). In L4T R32.3.1, the size was hard-coded. In R32.4.x NVIDIA changed to use RECROOTFSSIZE.

Still not sure when the tegra210 file would be used. Would that be for jetson-tx1?

Yes, and I have tested with a TX1 dev kit.

dwalkes commented 3 years ago

Yes, and I have tested with a TX1 dev kit.

OK I've added a tx1 worksheet as well. Looks like it has the same issue as AGX regarding RECROOTFSSIZE and same issue as tx2 and nano-emmc regarding swapping of the VER locations.

For the RECROOTFSSIZE change on AGX and TX1 it seems we should update both of these and I can't think of a downside.

For the VER and UENV changes I think this is going to be more complicated. No response yet on my question about relocating. For nano I think it's easier to relocate, since we don't update the bootloader I wouldn't expect this to really be something anyone will notice unless they are flashing the sdcard and paying very close attention to which rootfs they are using. See related discussion here. For TX1 since support is new, not advertised on mender hub and we don't know of anyone actually using I'm tempted to move these to match NVIDIA as well.

For TX2 if we change the uboot environment and VER location this would introduce incompatibility in the location of VER across mender u-boot bootloader updates and corrupt the VER location with u-boot environment content if booting an old version of u-boot bootloader on a new flash layout. The response to this post may help determine the urgency or whether we need to attempt to match. For the tx2 cboot install it seems it would be safer to move VER partitions, since we don't have the concern about uboot environment, however this would require special logic to select a different partition layout based on PREFERRED_PROVIDER_virtual/bootloader. This is another reason why encouraging folks starting out with TX2/mender to use PREFERRED_PROVIDER_virtual/bootloader selecting cboot might be a good approach too. See related headaches in https://github.com/OE4T/meta-mender-community/issues/7.

I'm tempted to fix the TX2 issue by just moving the VER location to match NVIDIAs r32.4.3 layout for cboot based builds and making the default configuration for tegrademo-mender use cboot and having yet another version of flash layout for cboot based tx2 builds. What do you think?

madisongh commented 3 years ago

OK I've added a tx1 worksheet as well. Looks like it has the same issue as AGX regarding RECROOTFSSIZE and same issue as tx2 and nano-emmc regarding swapping of the VER locations.

For the RECROOTFSSIZE change on AGX and TX1 it seems we should update both of these and I can't think of a downside.

meta-tegra sets RECROOTFSSIZE to 314572800 (300MiB) by default, so as long as folks keep that default, it should be compatible. L4T R32.3.1 did not have a RECROOTFSSIZE setting at all, and when NV added it in R32.4.x, they set it to 100MiB in their config files (flash.sh still uses 300MiB as a default if unset).

Since it's unlikely we'll ever use the L4T recovery rootfs in a Mender setup (since we're doing full A/B), the size probably doesn't matter - in fact, we could get rid of it completely. The only consideration is compatibility for upgrades and not surprising anyone with the change in layout. Documenting the change could handle that.

For the VER and UENV changes I think this is going to be more complicated. No response yet on my question about relocating. For nano I think it's easier to relocate, since we don't update the bootloader I wouldn't expect this to really be something anyone will notice unless they are flashing the sdcard and paying very close attention to which rootfs they are using. See related discussion here. For TX1 since support is new, not advertised on mender hub and we don't know of anyone actually using I'm tempted to move these to match NVIDIA as well.

That should be fine. TX1 is like Nano in that it has no A/B support in the bootloader.

For TX2 if we change the uboot environment and VER location this would introduce incompatibility in the location of VER across mender u-boot bootloader updates and corrupt the VER location with u-boot environment content if booting an old version of u-boot bootloader on a new flash layout. The response to this post may help determine the urgency or whether we need to attempt to match. For the tx2 cboot install it seems it would be safer to move VER partitions, since we don't have the concern about uboot environment, however this would require special logic to select a different partition layout based on PREFERRED_PROVIDER_virtual/bootloader. This is another reason why encouraging folks starting out with TX2/mender to use PREFERRED_PROVIDER_virtual/bootloader selecting cboot might be a good approach too. See related headaches in #7.

I'm tempted to fix the TX2 issue by just moving the VER location to match NVIDIAs r32.4.3 layout for cboot based builds and making the default configuration for tegrademo-mender use cboot and having yet another version of flash layout for cboot based tx2 builds. What do you think?

The only programs that locate and parse the VER partitions are the updating tools that look them up by partition name rather than a hard-coded offset, so moving them should not be a big deal, and there shouldn't be a problem with those partitions on a downgrade. They aren't used by the bootloaders themselves. I'd be more concerned about losing U-Boot environment due to relocation/resizing across an upgrade/downgrade.

dwalkes commented 3 years ago

The only consideration is compatibility for upgrades and not surprising anyone with the change in layout. Documenting the change could handle that.

So would your preference be to match NVIDIA's layout regarding RECROOTFSSIZE and document the fact that this changed in the pull request as well as in some mender tegra release notes we add?

I'd be more concerned about losing U-Boot environment due to relocation/resizing across an upgrade/downgrade.

Agreed. I was thinking uboot environment location could possibly be handled with state scripts like the ones you wrote earlier but it would be more difficult to handle any dependencies on VER location for bootloader tools.

It seems you are suggesting we don't bother to change our tx2 layout for cboot or uboot based builds and just stick with our current VER location from r32.3.1?

madisongh commented 3 years ago

So would your preference be to match NVIDIA's layout regarding RECROOTFSSIZE and document the fact that this changed in the pull request as well as in some mender tegra release notes we add?

Sort of. Since we've kept the same 300MiB default for RECROOTFSSIZE in meta-tegra across versions (which differs from L4T's defaults, as it turns out), it's not exactly matching, other than to use the RECROOTFSSIZE token in the XML file instead of the hard-coded number. But making that change is fine, and yes, it should be mentioned in the release notes.

It seems you are suggesting we don't bother to change our tx2 layout for cboot or uboot based builds and just stick with our current VER location from r32.3.1?

I'm just saying that we should try to preserve the U-Boot environment across upgrades/downgrades as part of the integration -- the upgrade/downgrade considerations are, IMO, more important than alignment with stock L4T layouts in this specific case, since the VER partitions are not actually used by NV's bootloaders, just the update tools that run in Linux (and can look up the partitions by name).

Upgrades/downgrades will be easier for the U-Boot TX2 if the environment area remains at the same location with the same size... otherwise we'd have to hack in a way to transfer the environment settings like I did in those state scripts I originally put together.

This is a non-issue for the cboot-only platforms (or TX2 configured for cboot only), of course, and aligning with the L4T layout makes more sense for those cases. (Although I'd suggest that using a single layout for TX2 that works for either cboot-only or cboot+U-Boot would be better than having different ones.)