search

AI-Northstar-Tech / vector-io

Use the universal VDF format for vector datasets to easily export and import data from all vector databases
Apache License 2.0
158 stars 22 forks source link

KDB insert problem #58

Open dhruv-anand-aintech opened 4 months ago

dhruv-anand-aintech commented 4 months ago

Facing this issue while trying to upsert some data to kdb.ai

@qynikos, could you have a look? It's on v0.1.101

with this command:

import_vdf \        --id_column PMID \--subset \
        --max_num_rows 200 \
        --hf_dataset somewheresystems/dataclysm-pubmed \
        --vector_columns title_embedding,abstract_embedding \
        kdbai \
        --url https://cloud.kdb.ai/instance/n6qap7ddvz
Error: Error inserting chunk: Failed to insert data in table named: dataclysm_pubmed, because of: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty/1.19.9.1</center>
</body>
</html>
.
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/kdbai_client/api.py", line 536, in insert
    return self.session._rest_post_qipc(Session.INSERT_PATH, self.name, data, True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/kdbai_client/api.py", line 341, in _rest_post_qipc
    res = request.urlopen(req)
          ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 216, in urlopen
    return opener.open(url, data, timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 525, in open
    response = meth(req, response)
               ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 634, in http_response
    response = self.parent.error(
               ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 563, in error
    return self._call_chain(*args)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 496, in _call_chain
    result = func(*args)
             ^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 502: Bad Gateway

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/vdf_io/import_vdf/kdbai_import.py", line 203, in upsert_data
    table.insert(chunk)
  File "/opt/homebrew/lib/python3.11/site-packages/kdbai_client/api.py", line 538, in insert
    raise KDBAIException(f'Failed to insert data in table named: {self.name}.', e=e)
kdbai_client.api.KDBAIException: Failed to insert data in table named: dataclysm_pubmed, because of: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty/1.19.9.1</center>
</body>
</html>
.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/vdf_io/import_vdf_cli.py", line 53, in main
    run_import(span)
  File "/opt/homebrew/lib/python3.11/site-packages/vdf_io/import_vdf_cli.py", line 131, in run_import
    import_obj = slug_to_import_func[args["vector_database"]](args)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/vdf_io/import_vdf/kdbai_import.py", line 47, in import_vdb
    kdbai_import.upsert_data()
  File "/opt/homebrew/lib/python3.11/site-packages/vdf_io/import_vdf/kdbai_import.py", line 213, in upsert_data
    raise RuntimeError(f"Error inserting chunk: {e}")
RuntimeError: Error inserting chunk: Failed to insert data in table named: dataclysm_pubmed, because of: <html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>openresty/1.19.9.1</center>
</body>
</html>
.
qynikos commented 4 months ago

That is an error coming from the endpoint itself. Please drop a mail to support@cloud.kdb.ai to check the health of the endpoint.