Auto recover from: UnavailableShardsException

[chicco785]

Is your feature request related to a problem? Please describe.

If for what ever reason shards for a table are not available, injection fails:

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.8/site-packages/connexion/decorators/decorator.py", line 48, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.8/site-packages/connexion/decorators/uri_parsing.py", line 144, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.8/site-packages/connexion/decorators/validation.py", line 184, in wrapper
    response = function(request)
  File "/usr/local/lib/python3.8/site-packages/connexion/decorators/validation.py", line 384, in wrapper
    return function(request)
  File "/usr/local/lib/python3.8/site-packages/connexion/decorators/parameter.py", line 121, in wrapper
    return function(**kwargs)
  File "/src/ngsi-timeseries-api/src/reporter/reporter.py", line 189, in notify
    trans.insert(payload, fiware_s, fiware_sp)
  File "/src/ngsi-timeseries-api/src/translators/sql_translator.py", line 179, in insert
    res = self._insert_entities_of_type(et,
  File "/src/ngsi-timeseries-api/src/translators/sql_translator.py", line 285, in _insert_entities_of_type
    self._insert_entity_rows(table_name, col_names, entries, entities)
  File "/src/ngsi-timeseries-api/src/translators/sql_translator.py", line 305, in _insert_entity_rows
    self._insert_original_entities_in_failed_batch(
  File "/src/ngsi-timeseries-api/src/translators/sql_translator.py", line 359, in _insert_original_entities_in_failed_batch
    self.cursor.executemany(stmt, rows)
  File "/usr/local/lib/python3.8/site-packages/crate/client/cursor.py", line 67, in executemany
    self.execute(sql, bulk_parameters=seq_of_parameters)
  File "/usr/local/lib/python3.8/site-packages/crate/client/cursor.py", line 53, in execute
    self._result = self.connection.client.sql(sql, parameters,
  File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 331, in sql
    content = self._json_request('POST', self.path, data=data)
  File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 458, in _json_request
    _raise_for_status(response)
  File "/usr/local/lib/python3.8/site-packages/crate/client/http.py", line 187, in _raise_for_status
    raise ProgrammingError(error.get('message', ''),
crate.client.exceptions.ProgrammingError: SQLActionException[UnavailableShardsException: [mtekz.etairqualityobserved][4] Not enough active copies to meet shard count of [ALL] (have 2, needed 3). Timeout: [1m], request: [ShardRequest{, shardId=[mtekz.etairqualityobserved][4], timeout=1m}]]

This can be recovered with ALTER CLUSTER REROUTE RETRY FAILED query. For details check here

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

[github-actions[bot]]

Stale issue message