search

Arize-ai / phoenix

AI Observability & Evaluation
https://docs.arize.com/phoenix
Other
3.98k stars 294 forks source link

[ENHANCEMENT] Ability to specify phoenix host for phoenix.experiments.run_experiment() #4228

Open chrishart0 opened 3 months ago

chrishart0 commented 3 months ago

Is your feature request related to a problem? Please describe. phoenix.experiments.run_experiment() appears to default to localhost:6006 for reaching out to Phoenix. I am running phoenix as a container and running the experiment from another container, I would like to reach out to phoenix at a non-default endpoint such as http://phoenix:6006.

Describe the solution you'd like I would like a new parameter added to phoenix.experiments.run_experiment() which allows the specifying of an endpoint the same way you can for px.Client. Possibly a client could be passed to run_experiments.

Describe alternatives you've considered I tried setting the env vars ENV_PHOENIX_COLLECTOR_ENDPOINT and ENV_PHOENIX_HOST but neither seemed to effect effect running run_experiment()

Additional context

In my code, I am successfully uploading a dataframe using a px.Client(endpoint=...)I just need functionality to specify my non-default url to resolve this error below.

Here is my stack trace when I try to run an experiment. 

Traceback (most recent call last):
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 69, in map_httpcore_exceptions
    yield
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 233, in handle_request
    resp = self._pool.handle_request(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 216, in handle_request
    raise exc from None
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection_pool.py", line 196, in handle_request
    response = connection.handle_request(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 99, in handle_request
    raise exc
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 76, in handle_request
    stream = self._connect(request)
             ^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_sync/connection.py", line 122, in _connect
    stream = self._network_backend.connect_tcp(**kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_backends/sync.py", line 205, in connect_tcp
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/app/.venv/lib/python3.12/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc) from exc
httpcore.ConnectError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/app/tests/evals/eval_manager.py", line 337, in <module>
    eval_manager.run_phoenix_experiment()
  File "/app/tests/evals/eval_manager.py", line 318, in run_phoenix_experiment
    experiment = run_experiment(
                 ^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/phoenix/experiments/functions.py", line 186, in run_experiment
    experiment_response = sync_client.post(
                          ^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1145, in post
    return self.request(
           ^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/phoenix/utilities/client.py", line 61, in request
    response = super().request(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 827, in request
    return self.send(request, auth=auth, follow_redirects=follow_redirects)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 914, in send
    response = self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 942, in _send_handling_auth
    response = self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 979, in _send_handling_redirects
    response = self._send_single_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_client.py", line 1015, in _send_single_request
    response = transport.handle_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 232, in handle_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.12/contextlib.py", line 158, in __exit__
    self.gen.throw(value)
  File "/app/.venv/lib/python3.12/site-packages/httpx/_transports/default.py", line 86, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ConnectError: [Errno 111] Connection refused
mikeldking commented 3 months ago

@chrishart0 sorry for the late response. there is a client inside of the experiment running code that should respect the PHOENIX_COLLECTOR_ENDPOINT. See https://github.com/Arize-ai/phoenix/blob/1459dbccdc19d1ff54c04f520d8d371d2a5fe874/src/phoenix/experiments/functions.py#L82

the env var should be PHOENIX_COLLECTOR_ENDPOINT - can you try this one more time for us?