search

PrefectHQ / prefect

Prefect is a workflow orchestration framework for building resilient data pipelines in Python.
https://prefect.io
Apache License 2.0
17.68k stars 1.65k forks source link

Not Connecting with Azure Databricks #13129

Open keshavksingh opened 1 year ago

keshavksingh commented 1 year ago

https://adb-1**********87.7.azuredatabricks.net/

desertaxle commented 1 year ago

Hey @keshavksingh, sorry you're having trouble connecting to your Databricks instance! Do you have any additional information about the error you're seeing? A stack trace of the error would help diagnose the issue.

keshavksingh commented 1 year ago

Code Executed from prefect import flow from prefect_databricks import DatabricksCredentials from prefect_databricks.jobs import jobs_list

@flow def example_execute_endpoint_flow(): databricks_credentials = DatabricksCredentials.load("databricks") jobs = jobs_list( databricks_credentials, limit=5 ) return jobs

example_execute_endpoint_flow()

RESULT

10:06:46.765 | INFO | prefect.engine - Created flow run 'rebel-bandicoot' for flow 'example-execute-endpoint-flow' 10:06:48.399 | INFO | Flow run 'rebel-bandicoot' - Created task run 'jobs_list-0' for task 'jobs_list' 10:06:48.404 | INFO | Flow run 'rebel-bandicoot' - Executing 'jobs_list-0' immediately... 10:06:51.747 | ERROR | Task run 'jobs_list-0' - Encountered exception during execution: Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 189, in connect_tcp gai_res = await getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 496, in getaddrinfo gai_res = await get_asynclib().getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 1754, in getaddrinfo result = await get_running_loop().getaddrinfo( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 825, in getaddrinfo return await self.run_in_executor( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\thread.py", line 57, in run result = self.fn(*self.args, **self.kwargs) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 253, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 237, in handle_async_request response = await connection.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 86, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 63, in handle_async_request stream = await self._connect(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 111, in _connect stream = await self._network_backend.connect_tcp(**kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\auto.py", line 29, in connect_tcp return await self._backend.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) httpcore.ConnectError: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 1526, in orchestrate_task_run result = await task.fn(*args, kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\jobs.py", line 550, in jobs_list response = await execute_endpoint.fn( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\rest.py", line 134, in execute_endpoint response = await getattr(client, http_method)( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1757, in get return await self.request( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1533, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1620, in send response = await self._send_handling_auth( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1648, in _send_handling_auth response = await self._send_handling_redirects( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1685, in _send_handling_redirects response = await self._send_single_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1722, in _send_single_request response = await transport.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno 11001] getaddrinfo failed 10:06:52.116 | ERROR | Task run 'jobs_list-0' - Finished in state Failed('Task run encountered an exception: Traceback (most recent call last):\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_exceptions.py", line 10, in map_exceptions\n yield\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp\n stream: anyio.abc.ByteStream = await anyio.connect_tcp(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_core\_sockets.py", line 189, in connect_tcp\n gai_res = await getaddrinfo(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_core\_sockets.py", line 496, in getaddrinfo\n gai_res = await get_asynclib().getaddrinfo(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 1754, in getaddrinfo\n result = await get_running_loop().getaddrinfo(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 825, in getaddrinfo\n return await self.run_in_executor(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\thread.py", line 57, in run\n result = self.fn(self.args, self.kwargs)\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\socket.py", line 918, in getaddrinfo\n for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\nsocket.gaierror: [Errno 11001] getaddrinfo failed\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpx\_transports\default.py", line 60, in map_httpcore_exceptions\n yield\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpx\_transports\default.py", line 353, in handle_async_request\n resp = await self._pool.handle_async_request(req)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection_pool.py", line 253, in handle_async_request\n raise exc\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection_pool.py", line 237, in handle_async_request\n response = await connection.handle_async_request(request)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 86, in handle_async_request\n raise exc\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 63, in handle_async_request\n stream = await self._connect(request)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 111, in _connect\n stream = await self._network_backend.connect_tcp(kwargs)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\auto.py", line 29, in connect_tcp\n return await self._backend.connect_tcp(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp\n stream: anyio.abc.ByteStream = await anyio.connect_tcp(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit\n self.gen.throw(type, value, traceback)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions\n raise to_exc(exc)\nhttpcore.ConnectError: [Errno 11001] getaddrinfo failed\n\nThe above exception was the direct cause of the following exception:\n\nhttpx.ConnectError: [Errno 11001] getaddrinfo failed\n') 10:06:52.132 | ERROR | Flow run 'rebel-bandicoot' - Encountered exception during execution: Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 189, in connect_tcp gai_res = await getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 496, in getaddrinfo gai_res = await get_asynclib().getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 1754, in getaddrinfo result = await get_running_loop().getaddrinfo( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 825, in getaddrinfo return await self.run_in_executor( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\thread.py", line 57, in run result = self.fn(self.args, self.kwargs) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 253, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 237, in handle_async_request response = await connection.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 86, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 63, in handle_async_request stream = await self._connect(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 111, in _connect stream = await self._network_backend.connect_tcp(**kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\auto.py", line 29, in connect_tcp return await self._backend.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) httpcore.ConnectError: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 665, in orchestrate_flow_run result = await run_sync(flow_call) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 156, in run_sync_in_interruptible_worker_thread tg.start_soon( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 662, in aexit raise exceptions[0] File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 135, in capture_worker_thread_and_result result = __fn(args, kwargs) File "c:/Users/home/Downloads/pythonprojects/prefect/TrainDatabricks.py", line 26, in example_execute_endpoint_flow jobs = jobs_list( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\tasks.py", line 469, in call return enter_task_run_engine( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 965, in enter_task_run_engine return run_async_from_worker_thread(begin_run) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 177, in run_async_from_worker_thread return anyio.from_thread.run(call) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio\from_thread.py", line 49, in run return asynclib.run_async_from_thread(func, args) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 970, in run_async_from_thread return f.result() File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures_base.py", line 439, in result return self.get_result() File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures_base.py", line 388, in get_result raise self._exception File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 1114, in get_task_call_return_value return await future._result() File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\futures.py", line 237, in _result return await final_state.result(raise_on_failure=raise_on_failure, fetch=True) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\states.py", line 91, in _get_state_result raise await get_state_exception(state) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 1526, in orchestrate_task_run result = await task.fn(args, kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\jobs.py", line 550, in jobs_list response = await execute_endpoint.fn( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\rest.py", line 134, in execute_endpoint response = await getattr(client, http_method)( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1757, in get return await self.request( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1533, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1620, in send response = await self._send_handling_auth( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1648, in _send_handling_auth response = await self._send_handling_redirects( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1685, in _send_handling_redirects response = await self._send_single_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1722, in _send_single_request response = await transport.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno 11001] getaddrinfo failed 10:06:52.498 | ERROR | Flow run 'rebel-bandicoot' - Finished in state Failed('Flow run encountered an exception. Traceback (most recent call last):\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_exceptions.py", line 10, in map_exceptions\n yield\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp\n stream: anyio.abc.ByteStream = await anyio.connect_tcp(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_core\_sockets.py", line 189, in connect_tcp\n gai_res = await getaddrinfo(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_core\_sockets.py", line 496, in getaddrinfo\n gai_res = await get_asynclib().getaddrinfo(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\anyio\_backends\_asyncio.py", line 1754, in getaddrinfo\n result = await get_running_loop().getaddrinfo(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 825, in getaddrinfo\n return await self.run_in_executor(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\thread.py", line 57, in run\n result = self.fn(self.args, self.kwargs)\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\socket.py", line 918, in getaddrinfo\n for res in _socket.getaddrinfo(host, port, family, type, proto, flags):\nsocket.gaierror: [Errno 11001] getaddrinfo failed\n\nDuring handling of the above exception, another exception occurred:\n\nTraceback (most recent call last):\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpx\_transports\default.py", line 60, in map_httpcore_exceptions\n yield\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpx\_transports\default.py", line 353, in handle_async_request\n resp = await self._pool.handle_async_request(req)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection_pool.py", line 253, in handle_async_request\n raise exc\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection_pool.py", line 237, in handle_async_request\n response = await connection.handle_async_request(request)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 86, in handle_async_request\n raise exc\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 63, in handle_async_request\n stream = await self._connect(request)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_async\connection.py", line 111, in _connect\n stream = await self._network_backend.connect_tcp(kwargs)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\auto.py", line 29, in connect_tcp\n return await self._backend.connect_tcp(\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp\n stream: anyio.abc.ByteStream = await anyio.connect_tcp(\n File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit\n self.gen.throw(type, value, traceback)\n File "C:\Users\home\Downloads\pythonprojects\prefect\.venv\lib\site-packages\httpcore\_exceptions.py", line 14, in map_exceptions\n raise to_exc(exc)\nhttpcore.ConnectError: [Errno 11001] getaddrinfo failed\n\nThe above exception was the direct cause of the following exception:\n\nhttpx.ConnectError: [Errno 11001] getaddrinfo failed\n') Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 10, in map_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 189, in connect_tcp gai_res = await getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_sockets.py", line 496, in getaddrinfo gai_res = await get_asynclib().getaddrinfo( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 1754, in getaddrinfo result = await get_running_loop().getaddrinfo( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 825, in getaddrinfo return await self.run_in_executor( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures\thread.py", line 57, in run result = self.fn(self.args, **self.kwargs) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\socket.py", line 918, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno 11001] getaddrinfo failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 60, in map_httpcore_exceptions yield File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 253, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection_pool.py", line 237, in handle_async_request response = await connection.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 86, in handle_async_request raise exc File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 63, in handle_async_request stream = await self._connect(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_async\connection.py", line 111, in _connect stream = await self._network_backend.connect_tcp(**kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\auto.py", line 29, in connect_tcp return await self._backend.connect_tcp( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore\backends\asyncio.py", line 111, in connect_tcp stream: anyio.abc.ByteStream = await anyio.connect_tcp( File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpcore_exceptions.py", line 14, in map_exceptions raise to_exc(exc) httpcore.ConnectError: [Errno 11001] getaddrinfo failed

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "c:/Users/home/Downloads/pythonprojects/prefect/TrainDatabricks.py", line 32, in example_execute_endpoint_flow() File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\flows.py", line 468, in call return enter_flow_run_engine_from_flow_call( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 175, in enter_flow_run_engine_from_flow_call return anyio.run(begin_run) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_core_eventloop.py", line 70, in run return asynclib.run(func, *args, backend_options) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 292, in run return native_run(wrapper(), debug=debug) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\runners.py", line 43, in run return loop.run_until_complete(main) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\asyncio\base_events.py", line 616, in run_until_complete return future.result() File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 287, in wrapper return await func(args) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\client\utilities.py", line 47, in with_injected_client return await fn(args, kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 256, in create_then_begin_flow_run return await state.result(fetch=True) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\states.py", line 91, in _get_state_result raise await get_state_exception(state) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 665, in orchestrate_flow_run result = await run_sync(flow_call) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 156, in run_sync_in_interruptible_worker_thread tg.start_soon( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 662, in aexit raise exceptions[0] File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, args) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 135, in capture_worker_thread_and_result result = __fn(args, kwargs) File "c:/Users/home/Downloads/pythonprojects/prefect/TrainDatabricks.py", line 26, in example_execute_endpoint_flow jobs = jobs_list( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\tasks.py", line 469, in call return enter_task_run_engine( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 965, in enter_task_run_engine return run_async_from_worker_thread(begin_run) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\utilities\asyncutils.py", line 177, in run_async_from_worker_thread return anyio.from_thread.run(call) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio\from_thread.py", line 49, in run return asynclib.run_async_from_thread(func, args) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\anyio_backends_asyncio.py", line 970, in run_async_from_thread return f.result() File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures_base.py", line 439, in result return self.get_result() File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\concurrent\futures_base.py", line 388, in get_result raise self._exception File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 1114, in get_task_call_return_value return await future._result() File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\futures.py", line 237, in _result return await final_state.result(raise_on_failure=raise_on_failure, fetch=True) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\states.py", line 91, in _get_state_result raise await get_state_exception(state) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect\engine.py", line 1526, in orchestrate_task_run result = await task.fn(args, kwargs) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\jobs.py", line 550, in jobs_list response = await execute_endpoint.fn( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\prefect_databricks\rest.py", line 134, in execute_endpoint response = await getattr(client, http_method)( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1757, in get return await self.request( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1533, in request return await self.send(request, auth=auth, follow_redirects=follow_redirects) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1620, in send response = await self._send_handling_auth( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1648, in _send_handling_auth response = await self._send_handling_redirects( File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1685, in _send_handling_redirects response = await self._send_single_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_client.py", line 1722, in _send_single_request response = await transport.handle_async_request(request) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 353, in handle_async_request resp = await self._pool.handle_async_request(req) File "C:\Users\home\AppData\Local\Programs\Python\Python38\lib\contextlib.py", line 131, in exit self.gen.throw(type, value, traceback) File "C:\Users\home\Downloads\pythonprojects\prefect.venv\lib\site-packages\httpx_transports\default.py", line 77, in map_httpcore_exceptions raise mapped_exc(message) from exc httpx.ConnectError: [Errno 11001] getaddrinfo failed

LvffY commented 4 months ago

I'm having the exact same errors on my side from my Azure Databricks instance and using Prefect cloud (trial). Is there any update on this ?