Port: Pr keep ref to scene valid

[sjahr]

Description

Port https://github.com/ros-planning/moveit/pull/3538 over from moveit1

Checklist

• [ ] Required by CI: Code is auto formatted using clang-format
• [ ] Extend the tutorials / documentation reference
• [ ] Document API changes relevant to the user in the MIGRATION.md notes
• [ ] Create tests, which fail without this PR reference
• [ ] Include a screenshot if changing a GUI
• [ ] While waiting for someone to review your request, please help review another open pull request to support the maintainers

[rhaschke]

Unfortunately, CI is broken due to a broken rolling setup. Do the tests pass locally?

[sjahr]

Unfortunately, CI is broken due to a broken rolling setup. Do the tests pass locally?

Yes, but CI wont do for a little longer :/ The rolling transition to Ubuntu 24.04 is blocking us here. The Humble CI failure is fixed by https://github.com/ros-planning/moveit2/pull/2750