Race condition in public data handling

Noticed in freedesktop-sdk CI

Failure Summary
    integration/platform-integration.bst:
    [02:39:04][00:00:01][3e0961e6][   build:integration/platform-integration.bst] BUG     Build
        An unhandled exception occured:

        Traceback (most recent call last):
          File "/usr/lib/python3.11/site-packages/buildstream/_scheduler/jobs/job.py", line 346, in child_action
            result = self.child_process()  # pylint: disable=assignment-from-no-return
                     ^^^^^^^^^^^^^^^^^^^^
          File "/usr/lib/python3.11/site-packages/buildstream/_scheduler/jobs/elementjob.py", line 81, in child_process
            return self._action_cb(self._element)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/usr/lib/python3.11/site-packages/buildstream/_scheduler/queues/buildqueue.py", line 55, in _assemble_element
            element._assemble()
          File "/usr/lib/python3.11/site-packages/buildstream/element.py", line 1687, in _assemble
            collect = self.assemble(sandbox)  # pylint: disable=assignment-from-no-return
                      ^^^^^^^^^^^^^^^^^^^^^^
          File "/builds/freedesktop-sdk/freedesktop-sdk/.bst/staged-junctions/plugins/buildstream-plugins-community.bst/378f21cb745cc234d69c13a78f0b1dbac94ecaeeb7544c463df174723f2d5224/src/buildstream_plugins_community/elements/collect_integration.py", line 93, in assemble
            bstdata = dependency.get_public_data("bst")
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/usr/lib/python3.11/site-packages/buildstream/_elementproxy.py", line 138, in get_public_data
            return cast("Element", self._plugin).get_public_data(domain)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
          File "/usr/lib/python3.11/site-packages/buildstream/element.py", line 735, in get_public_data
            self.__load_public_data()
          File "/usr/lib/python3.11/site-packages/buildstream/element.py", line 3201, in __load_public_data
            assert self.__dynamic_public is None
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        AssertionError

@abderrahim hinted that it's because the code has segments like

        if self.__dynamic_public is None:
            self.__load_public_data()

which are not protected with explicit locks. GIL doesn't thread-safety here as operation is done over multiple statements.

apache / buildstream

Race condition in public data handling #1955