search

aperture-data / aperturedb-python

ApertureDB Python Client
Other
7 stars 3 forks source link

Long running Connector has no logic to reauthenticate. #484

Closed gsaluja9 closed 2 months ago

gsaluja9 commented 2 months ago

Connector, which is called before a refresh token expires can issue a refresh token, which also extends the ttl of the refresh token. A Connector that leads to invalid session response from Refresh Token does not ever kick off a authenticate to try to re create session.

Observed the following error log.

2024-09-18 08:12:16,581 : WARNING : aperturedb.Connector : 139877719430720 : 472 : Session expired while query was sent. Retrying... [semanticsearch.datasets.gcp.cloud.aperturedata.io:55555 as reader using TCP with SSL=True]
2024-09-18 08:12:16,606 : WARNING : aperturedb.Connector : 139877719430720 : 492 : [Attempt 1 of 3] Failed to refresh token.
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 489, in _renew_session
    self._check_session_status()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 250, in _check_session_status
    self._refresh_token()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 265, in _refresh_token
    raise UnauthorizedException(response)
aperturedb.Connector.UnauthorizedException: [{'RefreshToken': {'info': 'Invalid Session', 'status': -1}}]
Stack (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 194, in invoke
    return self.get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 148, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 316, in get_relevant_documents
    result = self._get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 705, in _get_relevant_documents
    docs = self.vectorstore.max_marginal_relevance_search(
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 292, in max_marginal_relevance_search
    return self.max_marginal_relevance_search_by_vector(embedding, k, fetch_k, lambda_mult, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 305, in max_marginal_relevance_search_by_vector
    descriptors.find_similar_mmr(
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 113, in find_similar_mmr
    self.metric = self._descriptorset_metric(set)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 75, in _descriptorset_metric
    response, _ = self.db.query(query)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 474, in query
    self._renew_session()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 492, in _renew_session
    logger.warning(
2024-09-18 08:12:17,636 : WARNING : aperturedb.Connector : 139877719430720 : 492 : [Attempt 2 of 3] Failed to refresh token.
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 489, in _renew_session
    self._check_session_status()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 250, in _check_session_status
    self._refresh_token()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 265, in _refresh_token
    raise UnauthorizedException(response)
aperturedb.Connector.UnauthorizedException: [{'RefreshToken': {'info': 'Invalid Session', 'status': -1}}]
Stack (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 194, in invoke
    return self.get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 148, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 316, in get_relevant_documents
    result = self._get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 705, in _get_relevant_documents
    docs = self.vectorstore.max_marginal_relevance_search(
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 292, in max_marginal_relevance_search
    return self.max_marginal_relevance_search_by_vector(embedding, k, fetch_k, lambda_mult, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 305, in max_marginal_relevance_search_by_vector
    descriptors.find_similar_mmr(
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 113, in find_similar_mmr
    self.metric = self._descriptorset_metric(set)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 75, in _descriptorset_metric
    response, _ = self.db.query(query)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 474, in query
    self._renew_session()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 492, in _renew_session
    logger.warning(
2024-09-18 08:12:18,662 : WARNING : aperturedb.Connector : 139877719430720 : 492 : [Attempt 3 of 3] Failed to refresh token.
Traceback (most recent call last):
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 489, in _renew_session
    self._check_session_status()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 250, in _check_session_status
    self._refresh_token()
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 265, in _refresh_token
    raise UnauthorizedException(response)
aperturedb.Connector.UnauthorizedException: [{'RefreshToken': {'info': 'Invalid Session', 'status': -1}}]
Stack (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.10/threading.py", line 953, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
    work_item.run()
  File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 194, in invoke
    return self.get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/_api/deprecation.py", line 148, in warning_emitting_wrapper
    return wrapped(*args, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/retrievers.py", line 316, in get_relevant_documents
    result = self._get_relevant_documents(
  File "/opt/venv/lib/python3.10/site-packages/langchain_core/vectorstores.py", line 705, in _get_relevant_documents
    docs = self.vectorstore.max_marginal_relevance_search(
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 292, in max_marginal_relevance_search
    return self.max_marginal_relevance_search_by_vector(embedding, k, fetch_k, lambda_mult, **kwargs)
  File "/opt/venv/lib/python3.10/site-packages/langchain_community/vectorstores/aperturedb.py", line 305, in max_marginal_relevance_search_by_vector
    descriptors.find_similar_mmr(
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 113, in find_similar_mmr
    self.metric = self._descriptorset_metric(set)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Descriptors.py", line 75, in _descriptorset_metric
    response, _ = self.db.query(query)
  File "/opt/venv/lib/python3.10/site-packages/aperturedb/Connector.py", line 474, in query
    self._renew_session()
gsaluja9 commented 2 months ago

This also explains why watchdog does not get affected similarly: it issues timely queries.