requests.exceptions.RequestException: Failed to create environment:

ishapuri commented 5 months ago

I try to follow the directions to set up an evaluation for WebShop (I assume the way to run other tasks is similar but you just activate those servers and then change the task name in the script)

I followed the instructions here and launched the server: https://github.com/WooooDyy/AgentGym/blob/main/docs/tutorials/en/01-evaluation.md

But when I try to run the code provided to get the actual evaluation, I get this error: requests.exceptions.RequestException: Failed to create environment: <Response [500]>

how do I fix this? I followed all the instructions.

python theirEvalCode.py 
/u/isha/miniconda3/envs/agentGym/lib/python3.12/site-packages/transformers/utils/hub.py:124: FutureWarning: Using `TRANSFORMERS_CACHE` is deprecated and will be removed in v5 of Transformers. Use `HF_HOME` instead.
  warnings.warn(
Downloading shards: 100%|█████████████████| 3/3 [00:00<00:00, 448.76it/s]
Loading checkpoint shards: 100%|███████████| 3/3 [00:24<00:00,  8.16s/it]
Traceback (most recent call last):
  File "/u/isha/agents_new/AgentGym/outputs/theirEvalCode.py", line 18, in <module>
    WebshopTask(
  File "/u/isha/agents_new/AgentGym/agentenv/agentenv/envs/webshop.py", line 108, in __init__
    super().__init__(client_args, n_clients, *args, **kwargs)
  File "/u/isha/agents_new/AgentGym/agentenv/agentenv/controller/task.py", line 54, in __init__
    self.clients = [self.env_client_cls(**client_args) for _ in range(n_clients)]
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/u/isha/agents_new/AgentGym/agentenv/agentenv/envs/webshop.py", line 34, in __init__
    raise RequestException(f"Failed to create environment: {ok}")
requests.exceptions.RequestException: Failed to create environment: <Response [500]>

Yiwen-Ding commented 5 months ago

Hi, Thank you for providing the details. The error message <Response [500]> typically indicates a server-side error. Could you please provide the output or logs from the server after you launched it?

arman-kazemi commented 4 months ago

@Yiwen-Ding I have the same issue with the webshop server. I believe some files are missing and the server tries to access them. Here is the error stack from the server:

(agentenv-webshop) ubuntu@ip-172-31-9-208:~/arman/AgentGym/agentenv-webshop$ webshop --host 127.0.0.1 --port 36004
/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/requests/__init__.py:43: DeprecationWarning: 'urllib3[secure]' extra is deprecated and will be removed in a future release of urllib3 2.x. Read more in this issue: https://github.com/urllib3/urllib3/issues/2680
  import urllib3
/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/jnius_config.py:87: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
  from pkg_resources import resource_filename
/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/pkg_resources/__init__.py:2825: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('mpl_toolkits')`.
Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
  declare_namespace(pkg)
/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/faiss/loader.py:28: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  if LooseVersion(numpy.__version__) >= "1.19":
/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/setuptools/_distutils/version.py:337: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.
  other = LooseVersion(other)
INFO:     Started server process [2762860]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:36004 (Press CTRL+C to quit)
-------Env 8189444 created--------
Products loaded.
Keys cleaned.
Attributes loaded.
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [00:00<00:00, 126616.68it/s]
INFO:     127.0.0.1:48990 - "POST /create HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/base.py", line 78, in call_next
    message = await recv_stream.receive()
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/anyio/streams/memory.py", line 98, in receive
    return self.receive_nowait()
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/anyio/streams/memory.py", line 91, in receive_nowait
    raise EndOfStream
anyio.EndOfStream

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py", line 399, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py", line 70, in __call__
    return await self.app(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/fastapi/applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/errors.py", line 184, in __call__
    raise exc
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/base.py", line 108, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/server.py", line 23, in log_request_response_time
    response = await call_next(request)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/base.py", line 84, in call_next
    raise app_exc
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/base.py", line 70, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 79, in __call__
    raise exc
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 20, in __call__
    raise e
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/fastapi/middleware/asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/fastapi/routing.py", line 273, in app
    raw_response = await run_endpoint_function(
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/fastapi/routing.py", line 190, in run_endpoint_function
    return await dependant.call(**values)
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/server.py", line 46, in create
    env = webshop_env_server.create()
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/environment.py", line 37, in create
    self.env[idx] = gym.make(
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/gym/envs/registration.py", line 676, in make
    return registry.make(id, **kwargs)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/gym/envs/registration.py", line 520, in make
    return spec.make(**kwargs)
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/gym/envs/registration.py", line 140, in make
    env = cls(**_kwargs)
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/../webshop/web_agent_site/envs/web_agent_text_env.py", line 61, in __init__
    self.server = SimServer(
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/../webshop/web_agent_site/envs/web_agent_text_env.py", line 299, in __init__
    self.search_engine = init_search_engine(num_products=num_products)
  File "/home/ubuntu/arman/AgentGym/agentenv-webshop/agentenv_webshop/../webshop/web_agent_site/engine/engine.py", line 206, in init_search_engine
    search_engine = LuceneSearcher(os.path.join(BASE_DIR, f'../search_engine/{indexes}'))
  File "/opt/conda/envs/agentenv-webshop/lib/python3.8/site-packages/pyserini/search/lucene/_searcher.py", line 51, in __init__
    self.object = JLuceneSearcher(index_dir)
  File "jnius/jnius_export_class.pxi", line 269, in jnius.JavaClass.__init__
  File "jnius/jnius_export_class.pxi", line 383, in jnius.JavaClass.call_constructor
  File "jnius/jnius_utils.pxi", line 79, in jnius.check_exception
jnius.JavaException: JVM exception occurred: /home/ubuntu/arman/AgentGym/agentenv-webshop/webshop/web_agent_site/../search_engine/indexes_1k does not exist or is not a directory. java.lang.IllegalArgumentException

arman-kazemi commented 4 months ago

I found out that the issue is due to a missing folder in webshop directory and solved it as follows:

copy https://github.com/princeton-nlp/WebShop/tree/a557a208d03b93c83f4075e66e8746922606e60f/search_engine and all the files in it into https://github.com/WooooDyy/AgentGym/tree/main/agentenv-webshop/webshop
run lines 50-56 from setup.sh in webshop https://github.com/WooooDyy/AgentGym/blob/0e89a615adee9112c5d4e8bc36b6c92cb17ab973/agentenv-webshop/webshop/setup.sh#L50
might need to change "./run_indexing.sh" to "bash run_indexing.sh"

You're good to go.

Yiwen-Ding commented 4 months ago

Thanks for your feeedback. We have fixed this problem in the latest commit. Please git pull and retry. fix: webshop setup.sh and search_engine folder

WooooDyy / AgentGym

requests.exceptions.RequestException: Failed to create environment: #20