BuildStream is now completely broken on MacOS #1408

Open BuildStream-Migration-Bot opened 3 years ago

See original issue on GitLab In GitLab by [Gitlab user @cs-shadow] on Nov 5, 2020, 19:55

Summary

Last time I was using BuildStream on Mac (a good few months ago now) I was able of running most (if not all) non-integration tests locally. And the CLI also used to work fine for the most part. Of course, anything that needed a real sandbox wouldn't work, but import elements etc would work fine.

It seems like now, things are completely broken and a bst build on our first example project results in a failure with a BUG message.

Steps to reproduce

pre-requisites:

Find a MacOS machine
Clone and install BuildStream master

then:

# this works fine
$ bst -C doc/examples/first-project show
...
   buildable b7a8b07d97bafb0ac1029d7f6164a49b636b1dcea34efa89ca5eb951aeeef401 hello.bst

# build fails miserably, pages and pages of the same error is repeated
$ bst -C doc/examples/first-project build
...
[--:--:--][b7a8b07d][    main:hello.bst                     ] STATUS  Fetch terminating
[--:--:--][        ][    main:core activity                 ] BUG     'NoneType' object has no attribute 'terminate'

    Traceback (most recent call last):
      File "/usr/local/Cellar/python[[Gitlab user @3]](https://gitlab.com/3).8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
        self._context.run(self._callback, *self._args)
      File "/Users/enigma/git/buildstream/buildstream/src/buildstream/_scheduler/scheduler.py", line 496, in _terminate_jobs_real
        job.terminate()
      File "/Users/enigma/git/buildstream/buildstream/src/buildstream/_scheduler/jobs/job.py", line 216, in terminate
        self._process.terminate()
      File "/usr/local/Cellar/python[[Gitlab user @3]](https://gitlab.com/3).8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 133, in terminate
        self._popen.terminate()
    AttributeError: 'NoneType' object has no attribute 'terminate'

[--:--:--][b7a8b07d][    main:hello.bst                     ] STATUS  Fetch terminating
[--:--:--][        ][    main:core activity                 ] BUG     'NoneType' object has no attribute 'terminate'

    Traceback (most recent call last):
      File "/usr/local/Cellar/python[[Gitlab user @3]](https://gitlab.com/3).8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/events.py", line 81, in _run
        self._context.run(self._callback, *self._args)
      File "/Users/enigma/git/buildstream/buildstream/src/buildstream/_scheduler/scheduler.py", line 496, in _terminate_jobs_real
        job.terminate()
      File "/Users/enigma/git/buildstream/buildstream/src/buildstream/_scheduler/jobs/job.py", line 216, in terminate
        self._process.terminate()
      File "/usr/local/Cellar/python[[Gitlab user @3]](https://gitlab.com/3).8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 133, in terminate
        self._popen.terminate()
    AttributeError: 'NoneType' object has no attribute 'terminate'

What is the current bug behavior?

Pages and pages of bug messages repeated over and over again, that is even hard to ctrl-c.

What is the expected correct behavior?

Ideally, we'd get back to the previous behavior of being able to build elements that don't require a sandbox.

I'm not sure when this started happening but is at least an issue since 9f205147b5461cc29486f2c1830e88913a8d098e (which is what I tried today).

In GitLab by [Gitlab user @cs-shadow] on Nov 5, 2020, 20:14

mentioned in merge request !1982

In GitLab by [Gitlab user @nanonyme] on Nov 5, 2020, 20:46

I wonder if this has to do with Mac defaulting to spawn start method for multiprocessing unlike all other POSIX platforms. (which use fork)

apache / buildstream