search

apache / buildstream

BuildStream, the software integration tool
https://buildstream.build/
Apache License 2.0
85 stars 28 forks source link

[bst-1] Stack trace while removing device files, fuse related #161

Open BuildStream-Migration-Bot opened 3 years ago

BuildStream-Migration-Bot commented 3 years ago

See original issue on GitLab In GitLab by [Gitlab user @tristanvb] on Nov 24, 2017, 05:35

As usual, this is almost impossible to reproduce.

Interestingly in this case we have two stack traces, one of them is incomplete because it's overwritten by the status area, and the remainder is properly delivered through the logging mechanism:

fusermount: failed to access mountpoint /home/tristan/.cache/buildstream/build/apps-lightsoff-l4dzm2ki/scratch/_/mount: Permission denied
Process Process-358:4:
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in _bootstrap
    self.run()
  File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
    self._target(*self._args, **self._kwargs)
  File "/codethink/GNOME/buildstream/buildstream/_fuse/mount.py", line 187, in __run_fuse
[00:01:20][44fe0704][build:apps/lightsoff.bst            ] BUG     gnome/apps-lightsoff/44fe0704-build.7666.log

    An unhandled exception occured:

    Traceback (most recent call last):
      File "/codethink/GNOME/buildstream/buildstream/_scheduler/job.py", line 265, in child_action
        result = self.action(element)
      File "/codethink/GNOME/buildstream/buildstream/_scheduler/buildqueue.py", line 34, in process
        element._assemble()
      File "/codethink/GNOME/buildstream/buildstream/element.py", line 1092, in _assemble
        self.stage(sandbox)
      File "/codethink/GNOME/buildstream/buildstream/buildelement.py", line 174, in stage
        dep.integrate(sandbox)
      File "/codethink/GNOME/buildstream/buildstream/element.py", line 484, in integrate
        exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
      File "/codethink/GNOME/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 179, in run
        self.try_remove_device(device_path)
      File "/codethink/GNOME/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 342, in try_remove_device
        os.unlink(device_path)
    OSError: [Errno 107] Transport endpoint is not connected: '/home/tristan/.cache/buildstream/build/apps-lightsoff-l4dzm2ki/scratch/_/mount/dev/full'

It looks like the first overwritten stack trace is not from the main process, but from the fuse subprocess.

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @juergbi] on Feb 16, 2018, 10:27

mentioned in merge request !278

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @toscalix] on May 23, 2018, 17:34

assigned to [Gitlab user @tristanvb]

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @toscalix] on May 23, 2018, 17:34

[Gitlab user @tristanvb] is this bug still relevant/valid?

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @franred] on Jul 11, 2018, 16:23

[Gitlab user @toscalix] [Gitlab user @tristanvb] This issue is still happening, I've seen it when building

https://gitlab.com/baserock/definitions/blob/master/elements/bootstrap/stage3-sysroot.bst in arm64 when bootstrapping. See the following log:

[--:--:--][][] START   Build
[--:--:--][][] START   Loading pipeline
[00:00:01][][] SUCCESS Loading pipeline
[--:--:--][][] START   Resolving pipeline
[00:00:00][][] SUCCESS Resolving pipeline
[--:--:--][][] START   Initializing remote caches
[00:00:00][][] SUCCESS Initializing remote caches
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources

BuildStream Version 1.1.3+176.g9defbb62
  Session Start: Wednesday, 11-07-2018 at 09:33:39
  Project:       baserock (/mnt/buildstream-workspace/projects/definitions)
  Targets:       bootstrap/stage3-sysroot.bst

User Configuration
  Configuration File:      Default Configuration
  Log Files:               /home/ubuntu/.cache/buildstream/logs
  Source Mirrors:          /home/ubuntu/.cache/buildstream/sources
  Build Area:              /home/ubuntu/.cache/buildstream/build
  Artifact Cache:          /home/ubuntu/.cache/buildstream/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Project Options
  arch:       armv8l64
  build_arch: armv8l64

Pipeline
      cached e08e7b1e973dd40253eb14fd03e2aca29d7271def6c0cf27a880f7c22bb78bae gnu-toolchain/base.bst 
      cached 3f010e97121e37cbd040be5fae087f4f2d00f9a73d2affb62807821b82f2834c gnu-toolchain/stage1-binutils.bst 
      cached adb5a3ef8fb123375cf9d7e5a8a4fb91709992e382de56ea3c1ee02227c8e7c6 gnu-toolchain/stage1-gcc.bst 
      cached 8c87f31ee95447d37e21d024c67af1b2405f7f4c89f9303d2ff5ec77a9dfad7b gnu-toolchain/stage1.bst 
     waiting 443de3b436e285570ce15e30f35a918cf50d675f8ab8783be33f88d50d39662d gnu-toolchain/stage2-fhs-dirs.bst 
      cached a035bab22170b3055d7b800fe75801082046bcefb5765ae8420c33e7e7956f23 gnu-toolchain/stage2-linux-api-headers.bst 
     waiting ca15fd73f57d1314434be88620107926416f8953b7ce77a71aa494ad72f3de84 gnu-toolchain/stage2-glibc.bst 
     waiting 7b5db0002b2db58eb704aae8ab5f4d75fdb89f13522aa4e5667cba0bbef16d1a gnu-toolchain/stage2-binutils.bst 
     waiting 7950bb172fd7c787f50d0deb369a11101f4fc49a1a927f7693271fad04948653 gnu-toolchain/stage2-busybox.bst 
      cached 7bc7b98f7d29fa710c6ecdb4d4f471f5e6eeeb9fd461f6faedc6c3eae35d5eb4 gnu-toolchain/stage2-fake-bash.bst 
     waiting 5f3b7fee2be039355fa32a4ecd83d9c8e8c25a5d04f38dfc52544f41abef37ee gnu-toolchain/stage2-gawk.bst 
     waiting e91cda85a49fd49e78f2b38bcd02e4f222dcc18cadc101268cd6e1c0c7f67b46 gnu-toolchain/stage2-gcc-fixed-headers.bst 
     waiting 00f547acc5794ab8a41e6aa319753981c65307dfa1b69c50e7f3feaba601f4a5 gnu-toolchain/stage2-libstdcxx.bst 
     waiting 77ee7e211bc2eb6af9dc4c3d970d18ef6959b6d62df20c21b8ff7976795091a3 gnu-toolchain/stage2-gcc.bst 
     waiting 11a4e288f97c03e9fa3f84830cf738d07184acc2cd1bee4e38cc9d3096459082 gnu-toolchain/stage2-make.bst 
     waiting cf81807416d830efe54e51bc9301f381372400e55dd4e185a3481356f000b8f6 gnu-toolchain/stage2-reset-specs.bst 
     waiting 4469b040741baae5b2f9879dbfc42b612e72e98d0824bd1480abf94ee1041353 gnu-toolchain/stage2.bst 
     waiting ce6dca5c02581cb9488944bd71692ad1b6f9023f799fe749990e85f3520fb4bf gnu-toolchain/fhs-dirs.bst 
     waiting 8e603caeb1fdab4fec27544514c30b549018a9d78b31256c9d23a33a5c3043b9 gnu-toolchain/linux-api-headers.bst 
     waiting 44d39afbb8b59c3e205937e6d0815d96c83c83d674e21781b75ebeab9acc6db4 gnu-toolchain/glibc.bst 
     waiting 8a1f1cc2cbc8c8270e538fa6c715bdc5757eb5d18b4a742dbedb9ed7b4de5c1c gnu-toolchain/zlib.bst 
     waiting 0878b4542f55bcc37a404b2c6ef5e72163b1c6b04c6653313592bca7d70ab657 gnu-toolchain/binutils.bst 
     waiting 198ac3481c2e9474f37349863d9dd2968e461973d261deec294a0f26ace052da gnu-toolchain/busybox.bst 
     waiting b1da0eb6e9db27e89ee701d18e121d6146b71ee59cc220b0229f3c2b85c3eb9d gnu-toolchain/ccache.bst 
     waiting 806a94f40a5ae2ee39419ee5ee08887a0afb0d0e9a755fae8060eb7fb219e9c7 gnu-toolchain/gawk.bst 
     waiting d75c12a543a3c0964d59ccb5c25a2d11ebc729f58d3d5b49a1c32c6962845d3d gnu-toolchain/m4-tarball.bst 
     waiting 3e8ebec67d894067a7890ab6f19aebce3ac5e897f5007e12afe78b4b14a27848 gnu-toolchain/gcc.bst 
     waiting fb7c3cc78126dc4a0c4675279ad24a40913532a7234b704375cd3db0a037feaf gnu-toolchain/make.bst 
     waiting 6f4dca3d24ce439688acdb0dd3ca4c1d31e9b1390491e11234baaaa8bb482224 gnu-toolchain.bst 
     waiting 946bb9f729f663f49140e507d7a53d53f391e2911857f5d31ad620618771dbaa bootstrap/stage3-sysroot.bst 
===============================================================================
[--:--:--][946bb9f7][ pull:bootstrap/stage3-sysroot.bst  ] START   baserock/bootstrap-stage3-sysroot/946bb9f7-pull.17166.log
[00:00:00][946bb9f7][ pull:bootstrap/stage3-sysroot.bst  ] SUCCESS baserock/bootstrap-stage3-sysroot/946bb9f7-pull.17166.log
[--:--:--][6f4dca3d][ pull:gnu-toolchain.bst             ] START   baserock/gnu-toolchain/6f4dca3d-pull.17171.log
[--:--:--][fb7c3cc7][ pull:gnu-toolchain/make.bst        ] START   baserock/gnu-toolchain-make/fb7c3cc7-pull.17172.log
[--:--:--][3e8ebec6][ pull:gnu-toolchain/gcc.bst         ] START   baserock/gnu-toolchain-gcc/3e8ebec6-pull.17175.log
[--:--:--][806a94f4][ pull:gnu-toolchain/gawk.bst        ] START   baserock/gnu-toolchain-gawk/806a94f4-pull.17178.log
[--:--:--][b1da0eb6][ pull:gnu-toolchain/ccache.bst      ] START   baserock/gnu-toolchain-ccache/b1da0eb6-pull.17180.log
[--:--:--][198ac348][ pull:gnu-toolchain/busybox.bst     ] START   baserock/gnu-toolchain-busybox/198ac348-pull.17186.log
[--:--:--][0878b454][ pull:gnu-toolchain/binutils.bst    ] START   baserock/gnu-toolchain-binutils/0878b454-pull.17191.log
[--:--:--][d75c12a5][ pull:gnu-toolchain/m4-tarball.bst  ] START   baserock/gnu-toolchain-m4-tarball/d75c12a5-pull.17192.log
[--:--:--][8a1f1cc2][ pull:gnu-toolchain/zlib.bst        ] START   baserock/gnu-toolchain-zlib/8a1f1cc2-pull.17194.log
[--:--:--][44d39afb][ pull:gnu-toolchain/glibc.bst       ] START   baserock/gnu-toolchain-glibc/44d39afb-pull.17203.log
[00:00:00][3e8ebec6][ pull:gnu-toolchain/gcc.bst         ] SUCCESS baserock/gnu-toolchain-gcc/3e8ebec6-pull.17175.log
[--:--:--][8e603cae][ pull:gnu-toolchain/linux-api-headers.bst] START   baserock/gnu-toolchain-linux-api-headers/8e603cae-pull.17224.log
[00:00:00][fb7c3cc7][ pull:gnu-toolchain/make.bst        ] SUCCESS baserock/gnu-toolchain-make/fb7c3cc7-pull.17172.log
[00:00:00][6f4dca3d][ pull:gnu-toolchain.bst             ] SUCCESS baserock/gnu-toolchain/6f4dca3d-pull.17171.log
[--:--:--][ce6dca5c][ pull:gnu-toolchain/fhs-dirs.bst    ] START   baserock/gnu-toolchain-fhs-dirs/ce6dca5c-pull.17230.log
[--:--:--][4469b040][ pull:gnu-toolchain/stage2.bst      ] START   baserock/gnu-toolchain-stage2/4469b040-pull.17232.log
[00:00:00][8e603cae][ pull:gnu-toolchain/linux-api-headers.bst] SUCCESS baserock/gnu-toolchain-linux-api-headers/8e603cae-pull.17224.log
[--:--:--][cf818074][ pull:gnu-toolchain/stage2-reset-specs.bst] START   baserock/gnu-toolchain-stage2-reset-specs/cf818074-pull.17240.log
[00:00:00][ce6dca5c][ pull:gnu-toolchain/fhs-dirs.bst    ] SUCCESS baserock/gnu-toolchain-fhs-dirs/ce6dca5c-pull.17230.log
[00:00:01][0878b454][ pull:gnu-toolchain/binutils.bst    ] SUCCESS baserock/gnu-toolchain-binutils/0878b454-pull.17191.log
[--:--:--][11a4e288][ pull:gnu-toolchain/stage2-make.bst ] START   baserock/gnu-toolchain-stage2-make/11a4e288-pull.17244.log
[--:--:--][77ee7e21][ pull:gnu-toolchain/stage2-gcc.bst  ] START   baserock/gnu-toolchain-stage2-gcc/77ee7e21-pull.17249.log
[00:00:00][4469b040][ pull:gnu-toolchain/stage2.bst      ] SUCCESS baserock/gnu-toolchain-stage2/4469b040-pull.17232.log
[00:00:01][806a94f4][ pull:gnu-toolchain/gawk.bst        ] SUCCESS baserock/gnu-toolchain-gawk/806a94f4-pull.17178.log
[--:--:--][5f3b7fee][ pull:gnu-toolchain/stage2-gawk.bst ] START   baserock/gnu-toolchain-stage2-gawk/5f3b7fee-pull.17254.log
[--:--:--][7950bb17][ pull:gnu-toolchain/stage2-busybox.bst] START   baserock/gnu-toolchain-stage2-busybox/7950bb17-pull.17256.log
[00:00:02][d75c12a5][ pull:gnu-toolchain/m4-tarball.bst  ] SUCCESS baserock/gnu-toolchain-m4-tarball/d75c12a5-pull.17192.log
[00:00:02][198ac348][ pull:gnu-toolchain/busybox.bst     ] SUCCESS baserock/gnu-toolchain-busybox/198ac348-pull.17186.log
[00:00:02][44d39afb][ pull:gnu-toolchain/glibc.bst       ] SUCCESS baserock/gnu-toolchain-glibc/44d39afb-pull.17203.log
[--:--:--][7b5db000][ pull:gnu-toolchain/stage2-binutils.bst] START   baserock/gnu-toolchain-stage2-binutils/7b5db000-pull.17264.log
[--:--:--][00f547ac][ pull:gnu-toolchain/stage2-libstdcxx.bst] START   baserock/gnu-toolchain-stage2-libstdcxx/00f547ac-pull.17269.log
[00:00:02][b1da0eb6][ pull:gnu-toolchain/ccache.bst      ] SUCCESS baserock/gnu-toolchain-ccache/b1da0eb6-pull.17180.log
[--:--:--][e91cda85][ pull:gnu-toolchain/stage2-gcc-fixed-headers.bst] START   baserock/gnu-toolchain-stage2-gcc-fixed-headers/e91cda85-pull.17273.log
[--:--:--][ca15fd73][ pull:gnu-toolchain/stage2-glibc.bst] START   baserock/gnu-toolchain-stage2-glibc/ca15fd73-pull.17275.log
[00:00:02][8a1f1cc2][ pull:gnu-toolchain/zlib.bst        ] SUCCESS baserock/gnu-toolchain-zlib/8a1f1cc2-pull.17194.log
[00:00:01][cf818074][ pull:gnu-toolchain/stage2-reset-specs.bst] SUCCESS baserock/gnu-toolchain-stage2-reset-specs/cf818074-pull.17240.log
[--:--:--][443de3b4][ pull:gnu-toolchain/stage2-fhs-dirs.bst] START   baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-pull.17282.log
[00:00:00][11a4e288][ pull:gnu-toolchain/stage2-make.bst ] SUCCESS baserock/gnu-toolchain-stage2-make/11a4e288-pull.17244.log
[00:00:00][5f3b7fee][ pull:gnu-toolchain/stage2-gawk.bst ] SUCCESS baserock/gnu-toolchain-stage2-gawk/5f3b7fee-pull.17254.log
[00:00:01][77ee7e21][ pull:gnu-toolchain/stage2-gcc.bst  ] SUCCESS baserock/gnu-toolchain-stage2-gcc/77ee7e21-pull.17249.log
[00:00:00][7b5db000][ pull:gnu-toolchain/stage2-binutils.bst] SUCCESS baserock/gnu-toolchain-stage2-binutils/7b5db000-pull.17264.log
[00:00:00][00f547ac][ pull:gnu-toolchain/stage2-libstdcxx.bst] SUCCESS baserock/gnu-toolchain-stage2-libstdcxx/00f547ac-pull.17269.log
[00:00:00][e91cda85][ pull:gnu-toolchain/stage2-gcc-fixed-headers.bst] SUCCESS baserock/gnu-toolchain-stage2-gcc-fixed-headers/e91cda85-pull.17273.log
[00:00:00][443de3b4][ pull:gnu-toolchain/stage2-fhs-dirs.bst] SUCCESS baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-pull.17282.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-build.17291.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   Staging dependencies
[00:00:01][ca15fd73][ pull:gnu-toolchain/stage2-glibc.bst] SUCCESS baserock/gnu-toolchain-stage2-glibc/ca15fd73-pull.17275.log
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   baserock/gnu-toolchain-stage2-glibc/ca15fd73-build.17293.log
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Staging dependencies
[00:00:01][7950bb17][ pull:gnu-toolchain/stage2-busybox.bst] SUCCESS baserock/gnu-toolchain-stage2-busybox/7950bb17-pull.17256.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] WARNING Non-whitelisted overlaps detected

    Staged files overwrite existing files in staging area:
    /usr/bin/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/include/gnumake.h: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/lib/debug/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-2: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/man/man1/make.1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /tools/lib/debug/make: gnu-toolchain/stage1-gcc.bst is not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst 

[00:00:01][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] SUCCESS Staging dependencies
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   Integrating sandbox
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] STATUS  Running integration command

    /usr/bin/ldconfig

[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] WARNING Non-whitelisted overlaps detected

    Staged files overwrite existing files in staging area:
    /usr/bin/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/include/gnumake.h: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/lib/debug/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-2: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/man/man1/make.1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /tools/lib/debug/make: gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst 

[00:00:02][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Staging dependencies
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Integrating sandbox
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running integration command

    /usr/bin/ldconfig

[00:00:03][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] BUG     Build

    An unhandled exception occured:

    Traceback (most recent call last):
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 148, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 207, in run
        self.try_remove_device(device_path)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 386, in try_remove_device
        os.unlink(device_path)
    OSError: [Errno 107] Transport endpoint is not connected: '/home/ubuntu/.cache/buildstream/build/gnu-toolchain-stage2-fhs-dirs-k5ih53a6/scratch/_/mount/dev/full'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/job.py", line 340, in _child_action
        result = self._action_cb(element)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/buildqueue.py", line 33, in process
        element._assemble()
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 1439, in _assemble
        self.stage(sandbox)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/buildelement.py", line 170, in stage
        dep.integrate(sandbox)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 782, in integrate
        exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 242, in run
        pass
      File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 150, in mounted
        self.unmount()
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 128, in unmount
        .format(type(self).__name__, self.__process.exitcode))
    buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

[00:00:04][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Integrating sandbox
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Staging sources
[00:00:17][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Staging sources
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running configure-commands

    mkdir o

[00:00:01][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running configure-commands

    export CFLAGS="-O2 $CFLAGS"; export CXX=false; \
    cd o && ../configure  \
      --build=$(../scripts/config.guess) --host=aarch64-bootstrap-linux-gnu \
      --prefix="/tools" \
      `# [1]` --libdir="/tools/lib" \
      `# [2]` --sysconfdir="/etc" \
      `# [3]` --with-headers="/tools/include" \
      `# [4]` --disable-profile  --enable-kernel=2.6.25 \
      `# [5]` libc_cv_c_cleanup=yes libc_cv_ctors_header=yes \
              libc_cv_forced_unwind=yes libc_cv_ssp=no

[--:--:--][ca15fd73][ main:gnu-toolchain/stage2-glibc.bst] STATUS  Build terminating
[--:--:--][ca15fd73][ main:gnu-toolchain/stage2-glibc.bst] STATUS  Build terminating
[00:01:20][][] FAILURE Build

Failure Summary
  gnu-toolchain/stage2-fhs-dirs.bst:
    [00:00:03][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] BUG     Build

        An unhandled exception occured:

        Traceback (most recent call last):
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 148, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 207, in run
            self.try_remove_device(device_path)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 386, in try_remove_device
            os.unlink(device_path)
        OSError: [Errno 107] Transport endpoint is not connected: '/home/ubuntu/.cache/buildstream/build/gnu-toolchain-stage2-fhs-dirs-k5ih53a6/scratch/_/mount/dev/full'

        During handling of the above exception, another exception occurred:

        Traceback (most recent call last):
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/job.py", line 340, in _child_action
            result = self._action_cb(element)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/buildqueue.py", line 33, in process
            element._assemble()
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 1439, in _assemble
            self.stage(sandbox)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/buildelement.py", line 170, in stage
            dep.integrate(sandbox)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 782, in integrate
            exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 242, in run
            pass
          File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
            raise exc_details[1]
          File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
            if cb(*exc_details):
          File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
            return cm_exit(cm, *exc_details)
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
            yield
          File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
            raise exc_details[1]
          File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
            if cb(*exc_details):
          File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
            return cm_exit(cm, *exc_details)
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
            yield
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 150, in mounted
            self.unmount()
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 128, in unmount
            .format(type(self).__name__, self.__process.exitcode))
        buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

Pipeline Summary
  Total:       30
  Session:     24
  Pull Queue:  processed 0, skipped 24, failed 0 
  Fetch Queue: processed 0, skipped 24, failed 0 
  Build Queue: processed 0, skipped 0,  failed 1

In this case is also producing a core dump in libffi which is not shown in this log but I've seen it in the terminal and it seems it is the cause of buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @franred] on Jul 11, 2018, 16:25

[Gitlab user @toscalix] [Gitlab user @tristanvb] This issue is still happening, I've seen it when building

https://gitlab.com/baserock/definitions/blob/master/elements/bootstrap/stage3-sysroot.bst in arm64 when bootstrapping. See the following log:

[--:--:--][][] START   Build
[--:--:--][][] START   Loading pipeline
[00:00:01][][] SUCCESS Loading pipeline
[--:--:--][][] START   Resolving pipeline
[00:00:00][][] SUCCESS Resolving pipeline
[--:--:--][][] START   Initializing remote caches
[00:00:00][][] SUCCESS Initializing remote caches
[--:--:--][][] START   Resolving cached state
[00:00:00][][] SUCCESS Resolving cached state
[--:--:--][][] START   Checking sources
[00:00:00][][] SUCCESS Checking sources

BuildStream Version 1.1.3+176.g9defbb62
  Session Start: Wednesday, 11-07-2018 at 09:33:39
  Project:       baserock (/mnt/buildstream-workspace/projects/definitions)
  Targets:       bootstrap/stage3-sysroot.bst

User Configuration
  Configuration File:      Default Configuration
  Log Files:               /home/ubuntu/.cache/buildstream/logs
  Source Mirrors:          /home/ubuntu/.cache/buildstream/sources
  Build Area:              /home/ubuntu/.cache/buildstream/build
  Artifact Cache:          /home/ubuntu/.cache/buildstream/artifacts
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Project Options
  arch:       armv8l64
  build_arch: armv8l64

Pipeline
      cached e08e7b1e973dd40253eb14fd03e2aca29d7271def6c0cf27a880f7c22bb78bae gnu-toolchain/base.bst 
      cached 3f010e97121e37cbd040be5fae087f4f2d00f9a73d2affb62807821b82f2834c gnu-toolchain/stage1-binutils.bst 
      cached adb5a3ef8fb123375cf9d7e5a8a4fb91709992e382de56ea3c1ee02227c8e7c6 gnu-toolchain/stage1-gcc.bst 
      cached 8c87f31ee95447d37e21d024c67af1b2405f7f4c89f9303d2ff5ec77a9dfad7b gnu-toolchain/stage1.bst 
     waiting 443de3b436e285570ce15e30f35a918cf50d675f8ab8783be33f88d50d39662d gnu-toolchain/stage2-fhs-dirs.bst 
      cached a035bab22170b3055d7b800fe75801082046bcefb5765ae8420c33e7e7956f23 gnu-toolchain/stage2-linux-api-headers.bst 
     waiting ca15fd73f57d1314434be88620107926416f8953b7ce77a71aa494ad72f3de84 gnu-toolchain/stage2-glibc.bst 
     waiting 7b5db0002b2db58eb704aae8ab5f4d75fdb89f13522aa4e5667cba0bbef16d1a gnu-toolchain/stage2-binutils.bst 
     waiting 7950bb172fd7c787f50d0deb369a11101f4fc49a1a927f7693271fad04948653 gnu-toolchain/stage2-busybox.bst 
      cached 7bc7b98f7d29fa710c6ecdb4d4f471f5e6eeeb9fd461f6faedc6c3eae35d5eb4 gnu-toolchain/stage2-fake-bash.bst 
     waiting 5f3b7fee2be039355fa32a4ecd83d9c8e8c25a5d04f38dfc52544f41abef37ee gnu-toolchain/stage2-gawk.bst 
     waiting e91cda85a49fd49e78f2b38bcd02e4f222dcc18cadc101268cd6e1c0c7f67b46 gnu-toolchain/stage2-gcc-fixed-headers.bst 
     waiting 00f547acc5794ab8a41e6aa319753981c65307dfa1b69c50e7f3feaba601f4a5 gnu-toolchain/stage2-libstdcxx.bst 
     waiting 77ee7e211bc2eb6af9dc4c3d970d18ef6959b6d62df20c21b8ff7976795091a3 gnu-toolchain/stage2-gcc.bst 
     waiting 11a4e288f97c03e9fa3f84830cf738d07184acc2cd1bee4e38cc9d3096459082 gnu-toolchain/stage2-make.bst 
     waiting cf81807416d830efe54e51bc9301f381372400e55dd4e185a3481356f000b8f6 gnu-toolchain/stage2-reset-specs.bst 
     waiting 4469b040741baae5b2f9879dbfc42b612e72e98d0824bd1480abf94ee1041353 gnu-toolchain/stage2.bst 
     waiting ce6dca5c02581cb9488944bd71692ad1b6f9023f799fe749990e85f3520fb4bf gnu-toolchain/fhs-dirs.bst 
     waiting 8e603caeb1fdab4fec27544514c30b549018a9d78b31256c9d23a33a5c3043b9 gnu-toolchain/linux-api-headers.bst 
     waiting 44d39afbb8b59c3e205937e6d0815d96c83c83d674e21781b75ebeab9acc6db4 gnu-toolchain/glibc.bst 
     waiting 8a1f1cc2cbc8c8270e538fa6c715bdc5757eb5d18b4a742dbedb9ed7b4de5c1c gnu-toolchain/zlib.bst 
     waiting 0878b4542f55bcc37a404b2c6ef5e72163b1c6b04c6653313592bca7d70ab657 gnu-toolchain/binutils.bst 
     waiting 198ac3481c2e9474f37349863d9dd2968e461973d261deec294a0f26ace052da gnu-toolchain/busybox.bst 
     waiting b1da0eb6e9db27e89ee701d18e121d6146b71ee59cc220b0229f3c2b85c3eb9d gnu-toolchain/ccache.bst 
     waiting 806a94f40a5ae2ee39419ee5ee08887a0afb0d0e9a755fae8060eb7fb219e9c7 gnu-toolchain/gawk.bst 
     waiting d75c12a543a3c0964d59ccb5c25a2d11ebc729f58d3d5b49a1c32c6962845d3d gnu-toolchain/m4-tarball.bst 
     waiting 3e8ebec67d894067a7890ab6f19aebce3ac5e897f5007e12afe78b4b14a27848 gnu-toolchain/gcc.bst 
     waiting fb7c3cc78126dc4a0c4675279ad24a40913532a7234b704375cd3db0a037feaf gnu-toolchain/make.bst 
     waiting 6f4dca3d24ce439688acdb0dd3ca4c1d31e9b1390491e11234baaaa8bb482224 gnu-toolchain.bst 
     waiting 946bb9f729f663f49140e507d7a53d53f391e2911857f5d31ad620618771dbaa bootstrap/stage3-sysroot.bst 
===============================================================================
[--:--:--][946bb9f7][ pull:bootstrap/stage3-sysroot.bst  ] START   baserock/bootstrap-stage3-sysroot/946bb9f7-pull.17166.log
[00:00:00][946bb9f7][ pull:bootstrap/stage3-sysroot.bst  ] SUCCESS baserock/bootstrap-stage3-sysroot/946bb9f7-pull.17166.log
[--:--:--][6f4dca3d][ pull:gnu-toolchain.bst             ] START   baserock/gnu-toolchain/6f4dca3d-pull.17171.log
[--:--:--][fb7c3cc7][ pull:gnu-toolchain/make.bst        ] START   baserock/gnu-toolchain-make/fb7c3cc7-pull.17172.log
[--:--:--][3e8ebec6][ pull:gnu-toolchain/gcc.bst         ] START   baserock/gnu-toolchain-gcc/3e8ebec6-pull.17175.log
[--:--:--][806a94f4][ pull:gnu-toolchain/gawk.bst        ] START   baserock/gnu-toolchain-gawk/806a94f4-pull.17178.log
[--:--:--][b1da0eb6][ pull:gnu-toolchain/ccache.bst      ] START   baserock/gnu-toolchain-ccache/b1da0eb6-pull.17180.log
[--:--:--][198ac348][ pull:gnu-toolchain/busybox.bst     ] START   baserock/gnu-toolchain-busybox/198ac348-pull.17186.log
[--:--:--][0878b454][ pull:gnu-toolchain/binutils.bst    ] START   baserock/gnu-toolchain-binutils/0878b454-pull.17191.log
[--:--:--][d75c12a5][ pull:gnu-toolchain/m4-tarball.bst  ] START   baserock/gnu-toolchain-m4-tarball/d75c12a5-pull.17192.log
[--:--:--][8a1f1cc2][ pull:gnu-toolchain/zlib.bst        ] START   baserock/gnu-toolchain-zlib/8a1f1cc2-pull.17194.log
[--:--:--][44d39afb][ pull:gnu-toolchain/glibc.bst       ] START   baserock/gnu-toolchain-glibc/44d39afb-pull.17203.log
[00:00:00][3e8ebec6][ pull:gnu-toolchain/gcc.bst         ] SUCCESS baserock/gnu-toolchain-gcc/3e8ebec6-pull.17175.log
[--:--:--][8e603cae][ pull:gnu-toolchain/linux-api-headers.bst] START   baserock/gnu-toolchain-linux-api-headers/8e603cae-pull.17224.log
[00:00:00][fb7c3cc7][ pull:gnu-toolchain/make.bst        ] SUCCESS baserock/gnu-toolchain-make/fb7c3cc7-pull.17172.log
[00:00:00][6f4dca3d][ pull:gnu-toolchain.bst             ] SUCCESS baserock/gnu-toolchain/6f4dca3d-pull.17171.log
[--:--:--][ce6dca5c][ pull:gnu-toolchain/fhs-dirs.bst    ] START   baserock/gnu-toolchain-fhs-dirs/ce6dca5c-pull.17230.log
[--:--:--][4469b040][ pull:gnu-toolchain/stage2.bst      ] START   baserock/gnu-toolchain-stage2/4469b040-pull.17232.log
[00:00:00][8e603cae][ pull:gnu-toolchain/linux-api-headers.bst] SUCCESS baserock/gnu-toolchain-linux-api-headers/8e603cae-pull.17224.log
[--:--:--][cf818074][ pull:gnu-toolchain/stage2-reset-specs.bst] START   baserock/gnu-toolchain-stage2-reset-specs/cf818074-pull.17240.log
[00:00:00][ce6dca5c][ pull:gnu-toolchain/fhs-dirs.bst    ] SUCCESS baserock/gnu-toolchain-fhs-dirs/ce6dca5c-pull.17230.log
[00:00:01][0878b454][ pull:gnu-toolchain/binutils.bst    ] SUCCESS baserock/gnu-toolchain-binutils/0878b454-pull.17191.log
[--:--:--][11a4e288][ pull:gnu-toolchain/stage2-make.bst ] START   baserock/gnu-toolchain-stage2-make/11a4e288-pull.17244.log
[--:--:--][77ee7e21][ pull:gnu-toolchain/stage2-gcc.bst  ] START   baserock/gnu-toolchain-stage2-gcc/77ee7e21-pull.17249.log
[00:00:00][4469b040][ pull:gnu-toolchain/stage2.bst      ] SUCCESS baserock/gnu-toolchain-stage2/4469b040-pull.17232.log
[00:00:01][806a94f4][ pull:gnu-toolchain/gawk.bst        ] SUCCESS baserock/gnu-toolchain-gawk/806a94f4-pull.17178.log
[--:--:--][5f3b7fee][ pull:gnu-toolchain/stage2-gawk.bst ] START   baserock/gnu-toolchain-stage2-gawk/5f3b7fee-pull.17254.log
[--:--:--][7950bb17][ pull:gnu-toolchain/stage2-busybox.bst] START   baserock/gnu-toolchain-stage2-busybox/7950bb17-pull.17256.log
[00:00:02][d75c12a5][ pull:gnu-toolchain/m4-tarball.bst  ] SUCCESS baserock/gnu-toolchain-m4-tarball/d75c12a5-pull.17192.log
[00:00:02][198ac348][ pull:gnu-toolchain/busybox.bst     ] SUCCESS baserock/gnu-toolchain-busybox/198ac348-pull.17186.log
[00:00:02][44d39afb][ pull:gnu-toolchain/glibc.bst       ] SUCCESS baserock/gnu-toolchain-glibc/44d39afb-pull.17203.log
[--:--:--][7b5db000][ pull:gnu-toolchain/stage2-binutils.bst] START   baserock/gnu-toolchain-stage2-binutils/7b5db000-pull.17264.log
[--:--:--][00f547ac][ pull:gnu-toolchain/stage2-libstdcxx.bst] START   baserock/gnu-toolchain-stage2-libstdcxx/00f547ac-pull.17269.log
[00:00:02][b1da0eb6][ pull:gnu-toolchain/ccache.bst      ] SUCCESS baserock/gnu-toolchain-ccache/b1da0eb6-pull.17180.log
[--:--:--][e91cda85][ pull:gnu-toolchain/stage2-gcc-fixed-headers.bst] START   baserock/gnu-toolchain-stage2-gcc-fixed-headers/e91cda85-pull.17273.log
[--:--:--][ca15fd73][ pull:gnu-toolchain/stage2-glibc.bst] START   baserock/gnu-toolchain-stage2-glibc/ca15fd73-pull.17275.log
[00:00:02][8a1f1cc2][ pull:gnu-toolchain/zlib.bst        ] SUCCESS baserock/gnu-toolchain-zlib/8a1f1cc2-pull.17194.log
[00:00:01][cf818074][ pull:gnu-toolchain/stage2-reset-specs.bst] SUCCESS baserock/gnu-toolchain-stage2-reset-specs/cf818074-pull.17240.log
[--:--:--][443de3b4][ pull:gnu-toolchain/stage2-fhs-dirs.bst] START   baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-pull.17282.log
[00:00:00][11a4e288][ pull:gnu-toolchain/stage2-make.bst ] SUCCESS baserock/gnu-toolchain-stage2-make/11a4e288-pull.17244.log
[00:00:00][5f3b7fee][ pull:gnu-toolchain/stage2-gawk.bst ] SUCCESS baserock/gnu-toolchain-stage2-gawk/5f3b7fee-pull.17254.log
[00:00:01][77ee7e21][ pull:gnu-toolchain/stage2-gcc.bst  ] SUCCESS baserock/gnu-toolchain-stage2-gcc/77ee7e21-pull.17249.log
[00:00:00][7b5db000][ pull:gnu-toolchain/stage2-binutils.bst] SUCCESS baserock/gnu-toolchain-stage2-binutils/7b5db000-pull.17264.log
[00:00:00][00f547ac][ pull:gnu-toolchain/stage2-libstdcxx.bst] SUCCESS baserock/gnu-toolchain-stage2-libstdcxx/00f547ac-pull.17269.log
[00:00:00][e91cda85][ pull:gnu-toolchain/stage2-gcc-fixed-headers.bst] SUCCESS baserock/gnu-toolchain-stage2-gcc-fixed-headers/e91cda85-pull.17273.log
[00:00:00][443de3b4][ pull:gnu-toolchain/stage2-fhs-dirs.bst] SUCCESS baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-pull.17282.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   baserock/gnu-toolchain-stage2-fhs-dirs/443de3b4-build.17291.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   Staging dependencies
[00:00:01][ca15fd73][ pull:gnu-toolchain/stage2-glibc.bst] SUCCESS baserock/gnu-toolchain-stage2-glibc/ca15fd73-pull.17275.log
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   baserock/gnu-toolchain-stage2-glibc/ca15fd73-build.17293.log
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Staging dependencies
[00:00:01][7950bb17][ pull:gnu-toolchain/stage2-busybox.bst] SUCCESS baserock/gnu-toolchain-stage2-busybox/7950bb17-pull.17256.log
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] WARNING Non-whitelisted overlaps detected

    Staged files overwrite existing files in staging area:
    /usr/bin/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/include/gnumake.h: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/lib/debug/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-2: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/man/man1/make.1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst are not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /tools/lib/debug/make: gnu-toolchain/stage1-gcc.bst is not permitted to overlap other elements, order gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst 

[00:00:01][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] SUCCESS Staging dependencies
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] START   Integrating sandbox
[--:--:--][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] STATUS  Running integration command

    /usr/bin/ldconfig

[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] WARNING Non-whitelisted overlaps detected

    Staged files overwrite existing files in staging area:
    /usr/bin/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/include/gnumake.h: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/lib/debug/make: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/info/make.info-2: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /usr/share/man/man1/make.1: gnu-toolchain/stage1-binutils.bst and gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst above gnu-toolchain/base.bst 
    /tools/lib/debug/make: gnu-toolchain/stage1-gcc.bst and gnu-toolchain/stage2-linux-api-headers.bst are not permitted to overlap other elements, order gnu-toolchain/stage2-linux-api-headers.bst above gnu-toolchain/stage1-gcc.bst above gnu-toolchain/stage1-binutils.bst 

[00:00:02][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Staging dependencies
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Integrating sandbox
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running integration command

    /usr/bin/ldconfig

[00:00:03][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] BUG     Build

    An unhandled exception occured:

    Traceback (most recent call last):
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 148, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 207, in run
        self.try_remove_device(device_path)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 386, in try_remove_device
        os.unlink(device_path)
    OSError: [Errno 107] Transport endpoint is not connected: '/home/ubuntu/.cache/buildstream/build/gnu-toolchain-stage2-fhs-dirs-k5ih53a6/scratch/_/mount/dev/full'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/job.py", line 340, in _child_action
        result = self._action_cb(element)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/buildqueue.py", line 33, in process
        element._assemble()
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 1439, in _assemble
        self.stage(sandbox)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/buildelement.py", line 170, in stage
        dep.integrate(sandbox)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 782, in integrate
        exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 242, in run
        pass
      File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
        yield
      File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
        raise exc_details[1]
      File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
        if cb(*exc_details):
      File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
        return cm_exit(cm, *exc_details)
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
        yield
      File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
        self.gen.throw(type, value, traceback)
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 150, in mounted
        self.unmount()
      File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 128, in unmount
        .format(type(self).__name__, self.__process.exitcode))
    buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

[00:00:04][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Integrating sandbox
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Staging sources
[00:00:17][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Staging sources
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running configure-commands

    mkdir o

[00:00:01][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] SUCCESS Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] START   Running configure-commands
[--:--:--][ca15fd73][build:gnu-toolchain/stage2-glibc.bst] STATUS  Running configure-commands

    export CFLAGS="-O2 $CFLAGS"; export CXX=false; \
    cd o && ../configure  \
      --build=$(../scripts/config.guess) --host=aarch64-bootstrap-linux-gnu \
      --prefix="/tools" \
      `# [1]` --libdir="/tools/lib" \
      `# [2]` --sysconfdir="/etc" \
      `# [3]` --with-headers="/tools/include" \
      `# [4]` --disable-profile  --enable-kernel=2.6.25 \
      `# [5]` libc_cv_c_cleanup=yes libc_cv_ctors_header=yes \
              libc_cv_forced_unwind=yes libc_cv_ssp=no

[--:--:--][ca15fd73][ main:gnu-toolchain/stage2-glibc.bst] STATUS  Build terminating
[--:--:--][ca15fd73][ main:gnu-toolchain/stage2-glibc.bst] STATUS  Build terminating
[00:01:20][][] FAILURE Build

Failure Summary
  gnu-toolchain/stage2-fhs-dirs.bst:
    [00:00:03][443de3b4][build:gnu-toolchain/stage2-fhs-dirs.bst] BUG     Build

        An unhandled exception occured:

        Traceback (most recent call last):
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 148, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
            yield
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 207, in run
            self.try_remove_device(device_path)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 386, in try_remove_device
            os.unlink(device_path)
        OSError: [Errno 107] Transport endpoint is not connected: '/home/ubuntu/.cache/buildstream/build/gnu-toolchain-stage2-fhs-dirs-k5ih53a6/scratch/_/mount/dev/full'

        During handling of the above exception, another exception occurred:

        Traceback (most recent call last):
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/job.py", line 340, in _child_action
            result = self._action_cb(element)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_scheduler/buildqueue.py", line 33, in process
            element._assemble()
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 1439, in _assemble
            self.stage(sandbox)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/buildelement.py", line 170, in stage
            dep.integrate(sandbox)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/element.py", line 782, in integrate
            exitcode = sandbox.run(['sh', '-e', '-c', cmd], 0, env=environment, cwd='/')
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_sandboxbwrap.py", line 242, in run
            pass
          File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
            raise exc_details[1]
          File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
            if cb(*exc_details):
          File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
            return cm_exit(cm, *exc_details)
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 142, in mounted
            yield
          File "/usr/lib/python3.6/contextlib.py", line 380, in __exit__
            raise exc_details[1]
          File "/usr/lib/python3.6/contextlib.py", line 365, in __exit__
            if cb(*exc_details):
          File "/usr/lib/python3.6/contextlib.py", line 284, in _exit_wrapper
            return cm_exit(cm, *exc_details)
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/sandbox/_mount.py", line 84, in mounted
            yield
          File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
            self.gen.throw(type, value, traceback)
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 150, in mounted
            self.unmount()
          File "/mnt/buildstream-workspace/projects/buildstream/buildstream/_fuse/mount.py", line 128, in unmount
            .format(type(self).__name__, self.__process.exitcode))
        buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

Pipeline Summary
  Total:       30
  Session:     24
  Pull Queue:  processed 0, skipped 24, failed 0 
  Fetch Queue: processed 0, skipped 24, failed 0 
  Build Queue: processed 0, skipped 0,  failed 1

In this case is also producing a core dump in libffi which is not shown in this log but I've seen it in the terminal and it seems it is the cause of buildstream._fuse.mount.FuseMountError: SafeHardlinks reported exit code -11 when unmounting

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @franred] on Jul 11, 2018, 16:31

mentioned in merge request !539

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @franred] on Jul 12, 2018, 17:56

!539 doesn't fix or minimize the libffi crash in aarch64 when creating the SafeHardLinks. I will try to get some logs and more investigation.

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @aevri] on Mar 21, 2019, 11:08

mentioned in merge request !1250

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @cs-shadow] on Feb 21, 2020, 20:55

We should be able to close this once https://gitlab.com/BuildStream/buildstream/issues/719 is finished as we will not be interacting with fuse directly.

For the same reason, it will be interesting to see if this still happens with buildbox-run-bubblewrap.

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @cs-shadow] on Jun 22, 2020, 22:33

We don't have this code anymore (see !1952).

In case someone hits this in future, please create an issue on the buildbox-run-bubblewrap repository.

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @cs-shadow] on Jun 22, 2020, 22:33

closed

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @tristanvb] on Nov 5, 2020, 07:10

marked #748 as a duplicate of this issue

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @tristanvb] on Nov 5, 2020, 07:10

marked this issue as related to #748

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @nanonyme] on Nov 5, 2020, 11:29

It probably makes sense to keep this open until 2.0.0 tag since this affects stable versions and is targeted against those.

BuildStream-Migration-Bot commented 3 years ago

In GitLab by [Gitlab user @tristanvb] on Nov 6, 2020, 06:54

changed title from Stack trace while removing device files, fuse related to {+[bst-1] +}Stack trace while removing device files, fuse related