Issue with refs accros junctions #519

Open Cynical-Optimist opened 4 years ago

See original issue on GitLab In GitLab by [Gitlab user @willsalmon] on Jul 26, 2018, 18:04

Summary

There seems to be a issue with junctions.

Steps to reproduce

I clone https://gitlab.com/willsalmon/buildstream-kicad

I run: bst track kicad.bst bst build kicad.bst

Interestingly if you track the junctions before running

bst build kicad.bst

Then you get different errors.

What is the current bug behavior?

Bst exits because that the ref's do not match.

What is the expected correct behavior?

It should down load all of the dependencies and try to build them

(I dont think the project will build atm but it should at least try. And it is not getting close.)

Relevant logs and/or screenshots

https://hastebin.com/odegehogaq.md

[00:00:24][784d4223][fetch:freedesktop-sdk.bst:base/gnutls.bst] WARNING Try #2 failed, retrying
[--:--:--][784d4223][fetch:freedesktop-sdk.bst:base/gnutls.bst] START   base-sdk/base-gnutls/784d4223-fetch.15971.log
[--:--:--][784d4223][fetch:freedesktop-sdk.bst:base/gnutls.bst] START   Fetching https://www.gnupg.org/ftp/gcrypt/gnutls/v3.6/gnutls-3.6.2.tar.xz
[00:00:26][593520fe][fetch:freedesktop-sdk.bst:base/gnupg.bst] FAILURE Fetching https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.8.tar.bz2
[00:00:26][593520fe][fetch:freedesktop-sdk.bst:base/gnupg.bst] FAILURE File downloaded from https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.8.tar.bz2 has sha256sum '777b4cb8ced21965a5053d4fa20fe11484f0a478f3d011cef508a1a49db50dcd', not '747e1632271bd6aa852b9c40be15a12614adad257e4a765261355812d110558e'!

    Printing the last 20 lines from log file:
    /home/will/.cache/buildstream/logs/base-sdk/base-gnupg/593520fe-fetch.15969.log
    ======================================================================

        CFLAGS: -O2 -g -fstack-protector-strong
        CPPFLAGS: -O2 -D_FORTIFY_SOURCE=2
        CXXFLAGS: -O2 -g -fstack-protector-strong
        HOME: /tmp
        LC_ALL: en_US.UTF-8
        LDFLAGS: -fstack-protector-strong -Wl,-z,relro,-z,now
        LOGNAME: tomjon
        MAKEFLAGS: -j4
        PATH: /usr/bin:/bin:/usr/sbin:/sbin
        SHELL: /bin/sh
        SOURCE_DATE_EPOCH: '1320937200'
        TERM: dumb
        TZ: UTC
        USER: tomjon
        USERNAME: tomjon
        V: '1'
    [--:--:--] START   base/gnupg.bst-0: Fetching https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.8.tar.bz2
    [00:00:26] FAILURE base/gnupg.bst-0: Fetching https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.8.tar.bz2
    [00:00:26] FAILURE base/gnupg.bst: File downloaded from https://www.gnupg.org/ftp/gcrypt/gnupg/gnupg-2.2.8.tar.bz2 has sha256sum '777b4cb8ced21965a5053d4fa20fe11484f0a478f3d011cef508a1a49db50dcd', not '747e1632271bd6aa852b9c40be15a12614adad257e4a765261355812d110558e'!
    ======================================================================

Other relevant information

BuildStream version affected: current master ie. f62b6cb7420c91c2f87b3ebba4dcfcf16b4d6f0a

In GitLab by [Gitlab user @abderrahimk] on Jul 26, 2018, 22:47

changed the description

In GitLab by [Gitlab user @tristanvb] on Aug 5, 2018, 10:28

mentioned in commit 27f2bb210dcbd409b033e8260911061fa3b975e0

In GitLab by [Gitlab user @cs-shadow] on Oct 10, 2019, 17:43

[Gitlab user @willsalmon] I'm not sure if I understand this correctly. Seems to like the checksums in the bst file don't match what's being returned by the server. I am not sure what BuildStream needs to do differently?

In GitLab by [Gitlab user @willsalmon] on Nov 11, 2019, 10:33

[Gitlab user @cs-shadow], the issue is that because the miss match is inside the junction, it make everything very tricky to fix.

One solution is to make a workspace and then update inside there.

I am not sure if we can currently run bst source track workspace:workspacedelement.bst but if that is what is needed it is not clear to a new user to run such a thing.

In GitLab by [Gitlab user @BenjaminSchubert] on Nov 11, 2019, 10:43

If the junction you depend on isn't correct, you can always use patch or quilt to add the the changes you want to apply on top.

This has the advantage of keeping the junction unmodified and will fail to apply once it has been fixed upstream.

My usual workflow for that is:

bst workspace open junction.bst
# iterate over the junction until it is correct and generate the quilt patches out of it
# copy the quilt patches directory to my own project
# register a local source on the junction pointing to the patches
# register a quilt source
bst workspace close --remove-dir junction.bst
# Everything works!

In GitLab by [Gitlab user @willsalmon] on Nov 11, 2019, 10:50

[Gitlab user @BenjaminSchubert] this work flow seems sensible to me.

But if this is what we are recommending then i think this should be included in the doc's in a way that is accessible to new users. And that the error message should be much more helpful, maybe pointing to the new docs.

[Gitlab user @cs-shadow] I think this issue could be resolved by a Doc patch detailing the above workflow, and ideally also updating the error message.

In GitLab by [Gitlab user @cs-shadow] on Nov 11, 2019, 10:51

Just to note - bst source track workspace:workspacedelement.bst can work currently, however only if the project is using project.refs ref-storage. This is certainly a limitation of BuildStream.

That said, I do agree that we can improve our documentation in the meantime, while we don't have that feature.

buildstream-migration / bst-staging