OpenRobotLab / PointLLM

[ECCV 2024] PointLLM: Empowering Large Language Models to Understand Point Clouds
https://runsenxu.com/projects/PointLLM
450 stars 22 forks source link

The provided api code example didn't work! #15

Closed auniquesun closed 6 months ago

auniquesun commented 6 months ago

Thanks for sharing the project!

When I try to use the provided api to generate 3d objects related descriptions, the .npy path is valid on my computer, the example.json is as follows {"raw":{"a":1,"b":2},"serialized":null}. I am not clear what the exact format of the json file, so I just set a very simple one.

from gradio_client import Client

client = Client("http://101.230.144.196/")
result = client.predict(
    "Object ID",    # str in 'Input Method' Radio component
    "b4bbf2116b1a41a5a3b9d3622b07074c", # str in 'Object ID Input' Textbox component
    "data/modelnet40_c/data_background_1.npy",  # str (filepath or URL to file) in 'Upload Point Cloud File (PLY, NPY, etc.)' File component
    "example.json", # str (filepath to JSON file) in 'parameter_15' Chatbot component
    fn_index=4
)
print(result)

The server returned the following error.

Loaded as API: http://101.230.144.196/ ✔
Traceback (most recent call last):
  File "pointllm.py", line 15, in <module>
    result = client.predict(
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/site-packages/gradio_client/client.py", line 392, in predict
    return self.submit(*args, api_name=api_name, fn_index=fn_index).result()
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/site-packages/gradio_client/client.py", line 1573, in result
    return super().result(timeout=timeout)
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/concurrent/futures/_base.py", line 444, in result
    return self.__get_result()
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
    raise self._exception
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/site-packages/gradio_client/client.py", line 1245, in _inner
    predictions = _predict(*data)
  File "/home/jerry/miniconda3/envs/dassl/lib/python3.8/site-packages/gradio_client/client.py", line 1275, in _predict
    raise ValueError(result["error"])
ValueError: None

I am not sure why the error ocurred.

  1. Can you provide an example json file, including all neccessary keys and values? Since the json file is required in most api call functions.
  2. Can you provide some insights to help me figure out the error?

Thanks.

RunsenXu commented 6 months ago

Hi,

I am afraid the online demo cannot be used as API (I haven't tried this). You may download the model checkpoints and run the model on your local machine.

As for the return json format, you can check the code of our gradio demo: pointllm/eval/chat_gradio.py

Best, Runsen

auniquesun commented 6 months ago

But in your project page, you provided the API document, shown in below. I followed the instruction but didn't succeed.

58d426df27138d964ac88a15838e0eb

Also, I didn't find anything related to json format in pointllm/eval/chat_gradio.py.

RunsenXu commented 6 months ago

That’s provided by Gradio by default and my demo is not intended to be used as API. You may check Gradio’s doc and I am not sure if it will work or not.

auniquesun commented 6 months ago

@RunsenXu Thanks for your reply. Before that I thought the doc was written by yourself. Now I get that the doc was generated by the Gradio by default. Thanks.