siemens / meta-iot2050

SIMATIC IOT2050 Isar/Debian Board Support Package
MIT License
129 stars 76 forks source link

Problem with cpio and swupdate example image #436

Closed SCordibella closed 1 year ago

SCordibella commented 1 year ago

Hi all, I am trying to create my own image based on the swupdate example.

It is not clear to me where to checkout kas referred to my own layer. My idea is to checkout kas at the same level of my layer ie

    work
    ├── kas
    ├── meta-mybuild
    │   └── kas-mybuild.yml

and then call it with

 ../kas-container build kas-mybuild.yml  

inside my layer.

To try it out I make a very simple yml file:

header:
  version: 11
  includes:
   - repo: meta-iot2050
     file: kas-iot2050-swupdate.yml

build_system: isar

repos:
  meta-mybuild:

  meta-iot2050:
    url: https://github.com/siemens/meta-iot2050
    refspec: 845a732c6769841e7cbdb11e5561e3b6877da5bb

Unfortunately this doesn't work and report an error about cpio, here is the log:

2023-04-21 14:31:20 - INFO     - /build$ /work/isar/bitbake/bin/bitbake -c build iot2050-image-swu-example
Loading cache: 100% |###############################################################################################################################################################################| Time: 0:00:00
Loaded 104 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:00
Parsing of 104 .bb files complete (101 cached, 3 parsed). 104 targets, 0 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 0 Local 0 Network 0 Missed 0 Current 49 (0% match, 100% complete)
NOTE: Executing Tasks
ERROR: iot2050-image-swu-example-1.0-r0 do_swupdate_image: Execution of '/build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/run.do_swupdate_image.808' failed with exit code 127:
/build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/run.do_swupdate_image.808: 93: cpio: not found
WARNING: exit code 127 from a shell command.

ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/log.do_swupdate_image.808
Log data follows:
| DEBUG: Executing shell function do_swupdate_image
| /build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/run.do_swupdate_image.808: 93: cpio: not found
| WARNING: exit code 127 from a shell command.
| ERROR: Execution of '/build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/run.do_swupdate_image.808' failed with exit code 127:
| /build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050-wic-swu-img/1.0-r0/temp/run.do_swupdate_image.808: 93: cpio: not found
| WARNING: exit code 127 from a shell command.
|
ERROR: Task (/build/../work/meta-iot2050/recipes-core/images/iot2050-image-swu-example.bb:do_swupdate_image) failed with exit code '1'
NOTE: Tasks Summary: Attempted 554 tasks of which 544 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
/build/../work/meta-iot2050/recipes-core/images/iot2050-image-swu-example.bb:do_swupdate_image
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
2023-04-21 14:32:53 - ERROR    - Command returned non-zero exit status 1

It seems a path error with cpio, but if I look in the build tree cpio is at the same place as the meta-iot2050 build that is working.

jan-kiszka commented 1 year ago

First, you own layer seems to be missing a conf/layer.conf. Then you are using a old meta-iot2050 version - although that was a release, its swupdate capabilities were limited, and it had issues, including the one you see now. Try master instead.

SCordibella commented 1 year ago

Hi @jan-kiszka , thank you for your quick reply.

I don't add the complete tree view of my layer, I am sorry for that, of course I have a conf/layer.conf in my project.

For the master checkout do you have any particular commit to point or can I use the head? Are there any plan for the next release? Since we want to base a product on this OS I guess to start from a stable release.

SCordibella commented 1 year ago

The above error is solved using the latest commit on main branch in meta-iot2050.

However there is still a problem with my own image:

2023-04-27 13:31:41 - INFO     - /build$ /work/isar/bitbake/bin/bitbake -c build iot2050-image-ckm
Loading cache: 100% |                                                                                                                                                                              | ETA:  --:--:--
Loaded 0 entries from dependency cache.
Parsing recipes: 100% |#############################################################################################################################################################################| Time: 0:00:03
Parsing of 116 .bb files complete (0 cached, 116 parsed). 333 targets, 0 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Initialising tasks: 100% |##########################################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 66 Local 65 Mirrors 0 Missed 1 Current 0 (98% match, 0% complete)
NOTE: Executing Tasks
ERROR: iot2050-image-swu-example-1.0-r0 do_copy_boot_files_setscene: The recipe iot2050-image-swu-example is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced-pg2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6528-iot2050-basic.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6528-iot2050-basic-pg2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced-m2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
Please verify which recipe should provide the above files.

The build has stopped, as continuing in this scenario WILL break things - if not now, possibly in the future (we've seen builds fail several months later). If the system knew how to recover from this automatically it would, however there are several different scenarios which can result in this and we don't know which one this is. It may be you have switched providers of something like virtual/kernel (e.g. from linux-yocto to linux-yocto-dev), in that case you need to execute the clean task for both recipes and it will resolve this error. It may be you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning those recipes should again resolve this error, however switching DISTRO_FEATURES on an existing build directory is not supported - you should really clean out tmp and rebuild (reusing sstate should be safe). It could be the overlapping files detected are harmless in which case adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could also be your build is including two different conflicting versions of things (e.g. bluez 4 and bluez 5 and the correct solution for that would be to resolve the conflict. If in doubt, please ask on the mailing list, sharing the error and filelist above.
ERROR: iot2050-image-swu-example-1.0-r0 do_copy_boot_files_setscene: If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.
WARNING: Logfile for failed setscene task is /build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050/1.0-r0/temp/log.do_copy_boot_files_setscene.2527
WARNING: Setscene task (/build/../work/meta-iot2050/recipes-core/images/iot2050-image-swu-example.bb:do_copy_boot_files_setscene) failed with exit code '1' - real task will be run instead
ERROR: iot2050-image-swu-example-1.0-r0 do_copy_boot_files: The recipe iot2050-image-swu-example is trying to install files into a shared area when those files already exist. Those files and their manifest location are:
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced-pg2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6528-iot2050-basic.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6528-iot2050-basic-pg2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
  /build/tmp/deploy/images/iot2050/k3-am6548-iot2050-advanced-m2.dtb
    (matched in manifest-arm64-iot2050-image-ckm.copy_boot_files)
Please verify which recipe should provide the above files.

The build has stopped, as continuing in this scenario WILL break things - if not now, possibly in the future (we've seen builds fail several months later). If the system knew how to recover from this automatically it would, however there are several different scenarios which can result in this and we don't know which one this is. It may be you have switched providers of something like virtual/kernel (e.g. from linux-yocto to linux-yocto-dev), in that case you need to execute the clean task for both recipes and it will resolve this error. It may be you changed DISTRO_FEATURES from systemd to udev or vice versa. Cleaning those recipes should again resolve this error, however switching DISTRO_FEATURES on an existing build directory is not supported - you should really clean out tmp and rebuild (reusing sstate should be safe). It could be the overlapping files detected are harmless in which case adding them to SSTATE_ALLOW_OVERLAP_FILES may be the correct solution. It could also be your build is including two different conflicting versions of things (e.g. bluez 4 and bluez 5 and the correct solution for that would be to resolve the conflict. If in doubt, please ask on the mailing list, sharing the error and filelist above.
ERROR: iot2050-image-swu-example-1.0-r0 do_copy_boot_files: If the above message is too much, the simpler version is you're advised to wipe out tmp and rebuild (reusing sstate is fine). That will likely fix things in most (but not all) cases.
ERROR: Logfile of failure stored in: /build/tmp/work/iot2050-debian-arm64/iot2050-image-swu-example-iot2050/1.0-r0/temp/log.do_copy_boot_files.38118
ERROR: Task (/build/../work/meta-iot2050/recipes-core/images/iot2050-image-swu-example.bb:do_copy_boot_files) failed with exit code '1'
NOTE: Tasks Summary: Attempted 536 tasks of which 462 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /build/../work/meta-iot2050/recipes-core/images/iot2050-image-swu-example.bb:do_copy_boot_files
Summary: There were 4 ERROR messages, returning a non-zero exit code.
2023-04-27 13:33:44 - ERROR    - Command returned non-zero exit status 1

here is my image recipe:

require recipes-core/images/iot2050-image-swu-example.bb

# Disable Coral support
IOT2050_CORAL_SUPPORT = "0"

IMAGE_INSTALL:remove = " \
    node-red \
    node-red-gpio \
    node-red-preinstalled-nodes \
"
jan-kiszka commented 1 year ago

I'm currently also getting such errors from time to time, usually (or only?) over unclean build/ folders while switching / changing configurations and rebuilding. Try removing those listed dtb files from the deploy dir and rebuild.

Regarding a stable release: Work in progress, but I cannot give you concrete dates.

SCordibella commented 1 year ago

My case is different, I get the error on a clean build. Are there any tool like oe-pkgdata-util (https://stackoverflow.com/a/42293849) to show which package provide that files? I guess that I make some mistake with image inheritance, but I follow the same approach of the swu example image over the sample image.

The actual workaround is to delete them and rebuild, but of course it can't work in production.

jan-kiszka commented 1 year ago

The files are listed in https://github.com/siemens/meta-iot2050/blob/master/conf/machine/iot2050.conf#L20 and are being deployed by an Isar-specific image task. Something changed in Isar recently that seem to trigger it. If you know a pattern to reproduce it reliably, I'm all ears. It can't be a clean build alone as our CI is running fine.

SCordibella commented 1 year ago

It is not a clean build of the meta-iot2050 image, I recall the swu example image from my own image on my own layer. Here is a recap of my setup.

I create a minimal layer with a layer.conf file and an image recipe (iot2050-image-custom.bb) as follow:

require recipes-core/images/iot2050-image-swu-example.bb

# Disable Coral support
IOT2050_CORAL_SUPPORT = "0"

IMAGE_INSTALL:remove = " \
    node-red \
    node-red-gpio \
    node-red-preinstalled-nodes \
"

Then I checkout the latest release of kas as a git submodule of my layer and I create the following kas-iot2050-custom.yml file

header:
  version: 11
  includes:
    - repo: meta-iot2050
      file: kas-iot2050-swupdate.yml

build_system: isar

target: iot2050-image-custom

repos:
  meta-mylayer:

  meta-iot2050:
    url: https://github.com/siemens/meta-iot2050
    refspec: 77ded9343348008e4da67cdd9a694615edd8d750

When I run ./kas/kas-container build kas-iot2050-custom I get the error above about .dtb files.

jan-kiszka commented 1 year ago

While trying to reproduce the dtb issue, I think I understood your original problem: You also need to set ABROOTFS_IMAGE_RECIPE = "iot2050-image-custom", or initramfs-abrootfs-hook will use iot2050-image-swu-example rather than your own image recipe.

SCordibella commented 1 year ago

Thank you @jan-kiszka for the update, now I can try to use my own image.