`runpy` engine import 'unpredictable behaviour' warnings

[redsquare]

First check

• [X] I added a descriptive title to this issue.
• [X] I used the GitHub search to find a similar issue and didn't find it.
• [X] I searched the Prefect documentation for this issue.
• [X] I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

When running the job the following log message is seen

/usr/local/lib/python3.10/runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour

Reproduction

simple flow

Error

+pip install s3fs datadog python-dotenv r7insight_python s3fs python-dotenv slack_sdk r7insight_python datadog clickhouse-driver[lz4] pandas
Collecting s3fs
  Downloading s3fs-2022.11.0-py3-none-any.whl (27 kB)
Collecting datadog
  Downloading datadog-0.44.0-py2.py3-none-any.whl (111 kB)
Collecting python-dotenv
  Downloading python_dotenv-0.21.0-py3-none-any.whl (18 kB)
Collecting r7insight_python
  Downloading r7insight_python-1.0.0-py3-none-any.whl (6.4 kB)
Collecting slack_sdk
  Downloading slack_sdk-3.19.5-py2.py3-none-any.whl (273 kB)
Collecting clickhouse-driver[lz4]
  Downloading clickhouse_driver-0.2.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (736 kB)
Collecting pandas
  Downloading pandas-1.5.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.1 MB)
Collecting aiohttp!=4.0.0a0,!=4.0.0a1
  Downloading aiohttp-3.8.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.0 MB)
Collecting aiobotocore~=2.4.0
  Downloading aiobotocore-2.4.1-py3-none-any.whl (66 kB)
Requirement already satisfied: fsspec==2022.11.0 in /usr/local/lib/python3.10/site-packages (from s3fs) (2022.11.0)
Requirement already satisfied: requests>=2.6.0 in /usr/local/lib/python3.10/site-packages (from datadog) (2.28.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.10/site-packages (from r7insight_python) (2022.12.7)
Requirement already satisfied: pytz in /usr/local/lib/python3.10/site-packages (from clickhouse-driver[lz4]) (2022.6)
Collecting tzlocal
  Downloading tzlocal-4.2-py3-none-any.whl (19 kB)
Collecting clickhouse-cityhash>=1.0.2.1
  Downloading clickhouse_cityhash-1.0.2.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (96 kB)
Collecting lz4
  Downloading lz4-4.0.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.2 MB)
Requirement already satisfied: python-dateutil>=2.8.1 in /usr/local/lib/python3.10/site-packages (from pandas) (2.8.2)
Collecting numpy>=1.21.0
  Downloading numpy-1.23.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)
Collecting wrapt>=1.10.10
  Downloading wrapt-1.14.1-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)
Collecting botocore<1.27.60,>=1.27.59
  Downloading botocore-1.27.59-py3-none-any.whl (9.1 MB)
Collecting aioitertools>=0.5.1
  Downloading aioitertools-0.11.0-py3-none-any.whl (23 kB)
Collecting attrs>=17.3.0
  Downloading attrs-22.1.0-py2.py3-none-any.whl (58 kB)
Collecting aiosignal>=1.1.2
  Downloading aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Collecting yarl<2.0,>=1.0
  Downloading yarl-1.8.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)
Collecting async-timeout<5.0,>=4.0.0a3
  Downloading async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting multidict<7.0,>=4.5
  Downloading multidict-6.0.3-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.10/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (2.1.1)
Collecting frozenlist>=1.1.1
  Downloading frozenlist-1.3.3-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (149 kB)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/site-packages (from python-dateutil>=2.8.1->pandas) (1.16.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.10/site-packages (from requests>=2.6.0->datadog) (1.26.13)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.10/site-packages (from requests>=2.6.0->datadog) (3.4)
Collecting pytz-deprecation-shim
  Downloading pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)
Collecting jmespath<2.0.0,>=0.7.1
  Downloading jmespath-1.0.1-py3-none-any.whl (20 kB)
Collecting tzdata
  Downloading tzdata-2022.7-py2.py3-none-any.whl (340 kB)
Installing collected packages: tzdata, multidict, frozenlist, yarl, pytz-deprecation-shim, jmespath, attrs, async-timeout, aiosignal, wrapt, tzlocal, botocore, aioitertools, aiohttp, numpy, lz4, clickhouse-driver, clickhouse-cityhash, aiobotocore, slack-sdk, s3fs, r7insight-python, python-dotenv, pandas, datadog
Successfully installed aiobotocore-2.4.1 aiohttp-3.8.3 aioitertools-0.11.0 aiosignal-1.3.1 async-timeout-4.0.2 attrs-22.1.0 botocore-1.27.59 clickhouse-cityhash-1.0.2.4 clickhouse-driver-0.2.5 datadog-0.44.0 frozenlist-1.3.3 jmespath-1.0.1 lz4-4.0.2 multidict-6.0.3 numpy-1.23.5 pandas-1.5.2 python-dotenv-0.21.0 pytz-deprecation-shim-0.1.0.post0 r7insight-python-1.0.0 s3fs-2022.11.0 slack-sdk-3.19.5 tzdata-2022.7 tzlocal-4.2 wrapt-1.14.1 yarl-1.8.2
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
WARNING: You are using pip version 21.3.1; however, version 22.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.
/usr/local/lib/python3.10/runpy.py:126: RuntimeWarning: 'prefect.engine' found in sys.modules after import of package 'prefect', but prior to execution of 'prefect.engine'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
14:01:01.387 | INFO    | Flow run 'armored-macaw' - No clickhouse errors recently
14:01:01.774 | INFO    | Flow run 'armored-macaw' - Finished in state Completed()
Adding prefect.flow_runs
Adding prefect.task_runs
LE: Starting Rapid7 Insight Asynchronous Socket Appender

Versions

Version:             2.7.1
API version:         0.8.3
Python version:      3.9.5
Git commit:          33f7de50
Built:               Thu, Dec 8, 2022 4:09 PM
OS/Arch:             linux/x86_64
Profile:             test
Server type:         cloud

Additional context

No response

[zanieb]

Hi! I've updated the title to reflect that this affects other infrastructure types as well. This warning is safe to ignore and nebulous in it's meaning and source — we will try to fix it though!