Support parallel and lazy connect

[coretl]

https://github.com/DiamondLightSource/dodal/issues/415#issuecomment-2079651830 summarizes a discussion on Device connection. https://github.com/bluesky/ophyd-async/issues/265 will implement the ophyd-async supporting features of idempotent connect. https://github.com/DiamondLightSource/dodal/issues/483 will implement introspectable device factories.

The strategy we would like for blueapi is:

• Import the dodal.beamlines.<bl> module
• Call dodal.beamlines.beamline_utils.get_device_factories() to get all the device factories and whether they are lazy or not
• Create devices using all the factories
• For the non-lazy factories kick off connection in a background task (parallel to start off, revisit this if there are performance problems)
• Allow plans to be run
• When a plan is run, call ophyd_async.plan_stubs.ensure_connected(*devices) on all plan args and default args that are devices

[stan-dot]

    def with_dodal_module(self, module: ModuleType, **kwargs) -> None:
        devices, exceptions = make_all_devices(module, **kwargs)

        for device in devices.values():
            self.device(device)

        # If exceptions have occurred, we log them but we do not make blueapi
        # fall over
        if len(exceptions) > 0:
            LOGGER.warning(
                f"{len(exceptions)} exceptions occurred while instantiating devices"
            )
            LOGGER.exception(NotConnected(exceptions))

change the call to make_all_devices inside context.py

[stan-dot]

not sure if we should support the old and new instantiation behaviour or make it behind a flag and go beamline-by-beamline

[callumforrester]

@stan-dot I would favour supporting both, deprecating one and making issues to migrate as-and-when. Migrating beamline-by-beamline is still ideal but not mandated.

[stan-dot]

same