search

buildstream-migration / bst-staging

GNU Lesser General Public License v2.1
0 stars 0 forks source link

Standalone test failure for buildtrack #988

Closed Cynical-Optimist closed 4 years ago

Cynical-Optimist commented 4 years ago

See original issue on GitLab In GitLab by [Gitlab user @raoul].hidalgocharman on Apr 3, 2019, 17:15

Summary

The first two parametrized tests for test/frontend/buildtrack::test_build_track, fail if they are run with no test before them.

Steps to reproduce

Running either of the following will result in a failure:

tox -- "tests/frontend/buildtrack.py::test_build_track[track_targets0-exceptions0-tracked0-inline]"
tox -- "tests/frontend/buildtrack.py::test_build_track[track_targets0-exceptions0-tracked0-project.refs]"

But running this will result in tests passing:

tox -- "tests/frontend/buildcheckout.py::test_build_checkout[strict-copies]" "tests/frontend/buildtrack.py::test_build_track[track_targets0-exceptions0-tracked0-inline]"

What is the current bug behavior?

Issue seems to be creating an extra element it tracks called 0.bst-0, which the test then fails against because it's not expecting it.

________________________________________________________________________________________ test_build_track[track_targets0-exceptions0-tracked0-inline] _________________________________________________________________________________________

cli = <buildstream.plugintestutils.runcli.Cli object at 0x7ff97f431f98>, datafiles = local('/home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0')
tmpdir = local('/home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0'), ref_storage = 'inline', track_targets = ['0.bst'], exceptions = []
tracked = ['0.bst', '2.bst', '3.bst', '4.bst', '5.bst', '6.bst', ...]

    [[Gitlab user @pytest]](https://gitlab.com/pytest).mark.datafiles(os.path.join(DATA_DIR))
    [[Gitlab user @pytest]](https://gitlab.com/pytest).mark.parametrize("ref_storage", [('inline'), ('project.refs')])
    [[Gitlab user @pytest]](https://gitlab.com/pytest).mark.parametrize("track_targets,exceptions,tracked", [
        # Test with no exceptions
        (['0.bst'], [], ['0.bst', '2.bst', '3.bst', '4.bst', '5.bst', '6.bst', '7.bst']),
        (['3.bst'], [], ['3.bst', '4.bst', '5.bst', '6.bst']),
        (['2.bst', '3.bst'], [], ['2.bst', '3.bst', '4.bst', '5.bst', '6.bst', '7.bst']),

        # Test excepting '2.bst'
        (['0.bst'], ['2.bst'], ['0.bst', '3.bst', '4.bst', '5.bst', '6.bst']),
        (['3.bst'], ['2.bst'], []),
        (['2.bst', '3.bst'], ['2.bst'], ['3.bst', '4.bst', '5.bst', '6.bst']),

        # Test excepting '2.bst' and '3.bst'
        (['0.bst'], ['2.bst', '3.bst'], ['0.bst']),
        (['3.bst'], ['2.bst', '3.bst'], []),
        (['2.bst', '3.bst'], ['2.bst', '3.bst'], [])
    ])
    def test_build_track(cli, datafiles, tmpdir, ref_storage,
                         track_targets, exceptions, tracked):
        project = str(datafiles)
        dev_files_path = os.path.join(project, 'files', 'dev-files')
        element_path = os.path.join(project, 'elements')

        repo = create_repo('git', str(tmpdir))
        ref = repo.create(dev_files_path)

        configure_project(project, {
            'ref-storage': ref_storage
        })

        create_elements = {
            '0.bst': [
                '2.bst',
                '3.bst'
            ],
            '2.bst': [
                '3.bst',
                '7.bst'
            ],
            '3.bst': [
                '4.bst',
                '5.bst',
                '6.bst'
            ],
            '4.bst': [],
            '5.bst': [],
            '6.bst': [
                '5.bst'
            ],
            '7.bst': []
        }

        initial_project_refs = {}
        for element, dependencies in create_elements.items():
            # Test the element inconsistency resolution by ensuring that
            # only elements that aren't tracked have refs
            if element in set(tracked):
                # Elements which should not have a ref set
                #
                create_element(repo, element, element_path, dependencies)
            elif ref_storage == 'project.refs':
                # Store a ref in project.refs
                #
                create_element(repo, element, element_path, dependencies)
                initial_project_refs[element] = [{'ref': ref}]
            else:
                # Store a ref in the element itself
                #
                create_element(repo, element, element_path, dependencies, ref=ref)

        # Generate initial project.refs
        if ref_storage == 'project.refs':
            project_refs = {
                'projects': {
                    'test': initial_project_refs
                }
            }
            _yaml.dump(project_refs, os.path.join(project, 'project.refs'))

        args = ['build']
        args += itertools.chain.from_iterable(zip(itertools.repeat('--track'), track_targets))
        args += itertools.chain.from_iterable(zip(itertools.repeat('--track-except'), exceptions))
        args += ['0.bst']

        result = cli.run(project=project, silent=True, args=args)
        result.assert_success()
        tracked_elements = result.get_tracked_elements()

>       assert set(tracked_elements) == set(tracked)
E       AssertionError: assert {'0.bst', '0.... '5.bst', ...} == {'0.bst', '2.b... '6.bst', ...}
E         Extra items in the left set:
E         '0.bst-0'
E         Full diff:
E         - {'2.bst', '3.bst', '0.bst', '4.bst', '5.bst', '6.bst', '0.bst-0', '7.bst'}
E         ?                                                        -----------
E         + {'2.bst', '3.bst', '0.bst', '4.bst', '5.bst', '6.bst', '7.bst'}

tests/frontend/buildtrack.py:126: AssertionError
------------------------------------------------------------------------------------------------------------ Captured stdout call -------------------------------------------------------------------------------------------------------------
Initialized empty Git repository in /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo/.git/
[master (root-commit) 2abde3f] Initial commit
 1 file changed, 12 insertions(+)
 create mode 100644 usr/include/pony.h
BuildStream exited with code 0 for invocation:
    bst --no-colors --no-verbose --config /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache/buildstream.conf --directory /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0 build --track 0.bst 0.bst
Program stderr was:
[--:--:--][        ][    main:core activity                 ] START   Build
[--:--:--][        ][    main:core activity                 ] START   Loading elements
[00:00:00][        ][    main:core activity                 ] SUCCESS Loading elements
[--:--:--][        ][    main:core activity                 ] START   Resolving elements
[00:00:00][        ][    main:core activity                 ] SUCCESS Resolving elements
[--:--:--][        ][    main:core activity                 ] START   Resolving cached state
[00:00:00][        ][    main:core activity                 ] SUCCESS Resolving cached state
[--:--:--][        ][    main:core activity                 ] START   Checking sources
[00:00:00][        ][    main:core activity                 ] SUCCESS Checking sources

BuildStream Version 1.3.0+2013.g9605f798
  Session Start: Wednesday, 03-04-2019 at 17:10:09
  Project:       test (/home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0)
  Targets:       0.bst
  Cache Usage:   12K / None (0%)

User Configuration
  Configuration File:      /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache/buildstream.conf
  Cache Directory:         /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache
  Log Files:               /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache/logs
  Source Mirrors:          /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache/sources
  Build Area:              /home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/cache/build
  Strict Build Plan:       Yes
  Maximum Fetch Tasks:     10
  Maximum Build Tasks:     4
  Maximum Push Tasks:      4
  Maximum Network Retries: 2

Pipeline
no reference ???????????????????????????????????????????????????????????????? 4.bst 
no reference ???????????????????????????????????????????????????????????????? 5.bst 
no reference ???????????????????????????????????????????????????????????????? 6.bst 
no reference ???????????????????????????????????????????????????????????????? 3.bst 
no reference ???????????????????????????????????????????????????????????????? 7.bst 
no reference ???????????????????????????????????????????????????????????????? 2.bst 
no reference ???????????????????????????????????????????????????????????????? 0.bst 
===============================================================================
[--:--:--][????????][   track:4.bst                         ] START   test/4/????????-track.3178.log
[--:--:--][????????][   track:6.bst                         ] START   test/6/????????-track.3180.log
[--:--:--][????????][   track:5.bst                         ] START   test/5/????????-track.3179.log
[--:--:--][????????][   track:4.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:5.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:6.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:0.bst                         ] START   test/0/????????-track.3186.log
[--:--:--][????????][   track:7.bst                         ] START   test/7/????????-track.3183.log
[--:--:--][????????][   track:3.bst                         ] START   test/3/????????-track.3181.log
[--:--:--][        ][   track:0.bst-0                       ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:2.bst                         ] START   test/2/????????-track.3184.log
[--:--:--][????????][   track:7.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:3.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:2.bst                         ] START   Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[00:00:00][????????][   track:6.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:6.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:6.bst                         ] SUCCESS test/6/????????-track.3180.log
[00:00:00][????????][   track:7.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:7.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:7.bst                         ] SUCCESS test/7/????????-track.3183.log
[00:00:00][????????][   track:4.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:4.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:4.bst                         ] SUCCESS test/4/????????-track.3178.log
[00:00:00][????????][   track:5.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:5.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:5.bst                         ] SUCCESS test/5/????????-track.3179.log
[00:00:00][????????][   track:3.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:3.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:3.bst                         ] SUCCESS test/3/????????-track.3181.log
[00:00:00][????????][   track:2.bst                         ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][????????][   track:2.bst                         ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:2.bst                         ] SUCCESS test/2/????????-track.3184.log
[00:00:00][        ][   track:0.bst-0                       ] SUCCESS Tracking master from file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[--:--:--][        ][   track:0.bst-0                       ] INFO    Found new revision: ('2abde3f5dd1611de18c48ad51dc16457f5c47f55', [])
[00:00:00][????????][   track:0.bst                         ] SUCCESS test/0/????????-track.3186.log
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][2a925a30][   fetch:7.bst                         ] START   test/7/2a925a30-fetch.3275.log
[--:--:--][2a925a30][   fetch:7.bst                         ] START   Staging file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[00:00:00][2a925a30][   fetch:7.bst                         ] SUCCESS Staging file:///home/raoulhidalgo/Documents/buildstream/.tox/py37/tmp/test_build_track_track_targets0/repo
[00:00:00][2a925a30][   fetch:7.bst                         ] SUCCESS test/7/2a925a30-fetch.3275.log
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][2a925a30][   build:4.bst                         ] START   test/4/2a925a30-build.3320.log
[--:--:--][2a925a30][   build:4.bst                         ] START   Staging sources
[00:00:00][2a925a30][   build:4.bst                         ] SUCCESS Staging sources
[--:--:--][2a925a30][   build:4.bst                         ] START   Caching artifact
[--:--:--][2a925a30][   build:5.bst                         ] START   test/5/2a925a30-build.3327.log
[00:00:00][2a925a30][   build:4.bst                         ] SUCCESS Caching artifact
[00:00:00][2a925a30][   build:4.bst                         ] SUCCESS test/4/2a925a30-build.3320.log
[--:--:--][2a925a30][   build:7.bst                         ] START   test/7/2a925a30-build.3328.log
[--:--:--][2a925a30][   build:5.bst                         ] START   Staging sources
[00:00:00][2a925a30][   build:5.bst                         ] SUCCESS Staging sources
[--:--:--][2a925a30][   build:5.bst                         ] START   Caching artifact
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][2a925a30][   build:7.bst                         ] START   Staging sources
[00:00:00][2a925a30][   build:5.bst                         ] SUCCESS Caching artifact
[00:00:00][2a925a30][   build:7.bst                         ] SUCCESS Staging sources
[00:00:00][2a925a30][   build:5.bst                         ] SUCCESS test/5/2a925a30-build.3327.log
[--:--:--][2a925a30][   build:7.bst                         ] START   Caching artifact
[00:00:00][2a925a30][   build:7.bst                         ] SUCCESS Caching artifact
[00:00:00][2a925a30][   build:7.bst                         ] SUCCESS test/7/2a925a30-build.3328.log
Coverage.py warning: No data was collected. (no-data-collected)
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][99ba38ae][   build:6.bst                         ] START   test/6/99ba38ae-build.3333.log
[--:--:--][99ba38ae][   build:6.bst                         ] START   Staging sources
[00:00:00][99ba38ae][   build:6.bst                         ] SUCCESS Staging sources
[--:--:--][99ba38ae][   build:6.bst                         ] START   Caching artifact
[00:00:00][99ba38ae][   build:6.bst                         ] SUCCESS Caching artifact
[00:00:00][99ba38ae][   build:6.bst                         ] SUCCESS test/6/99ba38ae-build.3333.log
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][83536fb4][   build:3.bst                         ] START   test/3/83536fb4-build.3336.log
[--:--:--][83536fb4][   build:3.bst                         ] START   Staging sources
[00:00:00][83536fb4][   build:3.bst                         ] SUCCESS Staging sources
[--:--:--][83536fb4][   build:3.bst                         ] START   Caching artifact
[00:00:00][83536fb4][   build:3.bst                         ] SUCCESS Caching artifact
[00:00:00][83536fb4][   build:3.bst                         ] SUCCESS test/3/83536fb4-build.3336.log
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][b96050f8][   build:2.bst                         ] START   test/2/b96050f8-build.3339.log
[--:--:--][b96050f8][   build:2.bst                         ] START   Staging sources
[00:00:00][b96050f8][   build:2.bst                         ] SUCCESS Staging sources
[--:--:--][b96050f8][   build:2.bst                         ] START   Caching artifact
[00:00:00][b96050f8][   build:2.bst                         ] SUCCESS Caching artifact
[00:00:00][b96050f8][   build:2.bst                         ] SUCCESS test/2/b96050f8-build.3339.log
Coverage.py warning: No data was collected. (no-data-collected)
[--:--:--][d7447441][   build:0.bst                         ] START   test/0/d7447441-build.3342.log
[--:--:--][d7447441][   build:0.bst                         ] START   Staging sources
[00:00:00][d7447441][   build:0.bst                         ] SUCCESS Staging sources
[--:--:--][d7447441][   build:0.bst                         ] START   Caching artifact
[00:00:00][d7447441][   build:0.bst                         ] SUCCESS Caching artifact
[00:00:00][d7447441][   build:0.bst                         ] SUCCESS test/0/d7447441-build.3342.log
Coverage.py warning: No data was collected. (no-data-collected)
[00:00:01][        ][    main:core activity                 ] SUCCESS Build

Resolved key Summary
      cached d7447441faeb11229a79170b95823c3c059fd7291a1ecec18ad9b5108eff4640 0.bst 
      cached b96050f8cefb6de7641c6ea07d264806412e07cd502e4727f75129019c08240e 2.bst 
      cached 83536fb4fe767c37b4d73d7b68aa1d6ef899186f89e36263d8da4d22df78f4c0 3.bst 
      cached 2a925a301a5d9bdfa5d8481679c3d2e05c3db9a5ccd54e32b07fff374122cec2 4.bst 
      cached 2a925a301a5d9bdfa5d8481679c3d2e05c3db9a5ccd54e32b07fff374122cec2 5.bst 
      cached 99ba38aedbc083c10fe4813b267016b8d0193206c74b46dd2de43a6605e99325 6.bst 
      cached 2a925a301a5d9bdfa5d8481679c3d2e05c3db9a5ccd54e32b07fff374122cec2 7.bst 

Pipeline Summary
  Total:       7
  Session:     7
  Track Queue: processed 7, skipped 0, failed 0 
  Fetch Queue: processed 1, skipped 6, failed 0 
  Build Queue: processed 7, skipped 0, failed 0

Other relevant information

A git bisect points to this commit being the culprit bd7e75f99c28fbebedfe8d5db98a922b0e140a62 which introduces using a WeakValueDictionary to store plugin ID's. Unsure why this causes the bug.

Cynical-Optimist commented 4 years ago

In GitLab by [Gitlab user @jennis] on Sep 5, 2019, 15:27

I've been unable to reproduce this with current master (but the tests have slightly changed), so I think it's fixed but I'm unfortunately unable to tell you what has exactly fixed it.

I will close it for now. If you disagree, please reopen.

Cynical-Optimist commented 4 years ago

In GitLab by [Gitlab user @jennis] on Sep 5, 2019, 15:27

closed