OpenDRR / opendrr-api

REST API for OpenDRR data / API REST pour les données OpenDRR
MIT License
4 stars 7 forks source link

Delay pygeoapi start (was: Investigate pygeoapi container restart problem) #93

Open anthonyfok opened 3 years ago

anthonyfok commented 3 years ago

See also:

@jvanulde noticed that the pygeoapi container stops trying to spawn after n number of hours when the add_data.sh takes too long to run.

Need to figure out:

Consider adding a quick test, or use a tool such as wait-for-it (available in the Debian sid based pygeoapi Docker image), rather than running pygeoapi generate-openapi-document -c ${PYGEOAPI_CONFIG} > ${PYGEOAPI_OPENAPI} full-on in /entrypoint.sh:

echo "Trying to generate openapi.yml"
pygeoapi generate-openapi-document -c ${PYGEOAPI_CONFIG} > ${PYGEOAPI_OPENAPI}

from https://github.com/geopython/pygeoapi/blob/master/docker/entrypoint.sh#L62-L63 at the time of writing.

This will hopefully makes pygeoapi container restarts more gracefully and less verbose, unlike the current harmless but scary-looking messages below:

pygeoapi-opendrr_1       | START /entrypoint.sh
pygeoapi-opendrr_1       | Trying to generate openapi.yml
pygeoapi-opendrr_1       | NotFoundError(404, 'index_not_found_exception', 'no such index [dsra_acm6p5_beaufort_all_indicators_b]', dsra_acm6p5_beaufort_all_indicators_b, index_or_alias)
pygeoapi-opendrr_1       | Traceback (most recent call last):
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/provider/elasticsearch_.py", line 104, in __init__
pygeoapi-opendrr_1       |     self.fields = self.get_fields()
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/provider/elasticsearch_.py", line 117, in get_fields
pygeoapi-opendrr_1       |     ii = ic.get(self.index_name)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/elasticsearch/client/utils.py", line 84, in _wrapped
pygeoapi-opendrr_1       |     return func(*args, params=params, **kwargs)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/elasticsearch/client/indices.py", line 166, in get
pygeoapi-opendrr_1       |     return self.transport.perform_request(
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/elasticsearch/transport.py", line 351, in perform_request
pygeoapi-opendrr_1       |     status, headers_response, data = connection.perform_request(
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/elasticsearch/connection/http_urllib3.py", line 257, in perform_request
pygeoapi-opendrr_1       |     self._raise_error(response.status, raw_data)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/elasticsearch/connection/base.py", line 181, in _raise_error
pygeoapi-opendrr_1       |     raise HTTP_EXCEPTIONS.get(status_code, TransportError)(
pygeoapi-opendrr_1       | elasticsearch.exceptions.NotFoundError: NotFoundError(404, 'index_not_found_exception', 'no such index [dsra_acm6p5_beaufort_all_indicators_b]', dsra_acm6p5_beaufort_all_indicators_b, index_or_alias)
pygeoapi-opendrr_1       | 
pygeoapi-opendrr_1       | During handling of the above exception, another exception occurred:
pygeoapi-opendrr_1       | 
pygeoapi-opendrr_1       | Traceback (most recent call last):
pygeoapi-opendrr_1       |   File "/usr/local/bin/pygeoapi", line 11, in <module>
pygeoapi-opendrr_1       |     load_entry_point('pygeoapi', 'console_scripts', 'pygeoapi')()
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/core.py", line 764, in __call__
pygeoapi-opendrr_1       |     return self.main(*args, **kwargs)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/core.py", line 717, in main
pygeoapi-opendrr_1       |     rv = self.invoke(ctx)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/core.py", line 1137, in invoke
pygeoapi-opendrr_1       |     return _process_result(sub_ctx.command.invoke(sub_ctx))
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/core.py", line 956, in invoke
pygeoapi-opendrr_1       |     return ctx.invoke(self.callback, **ctx.params)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/core.py", line 555, in invoke
pygeoapi-opendrr_1       |     return callback(*args, **kwargs)
pygeoapi-opendrr_1       |   File "/usr/lib/python3/dist-packages/click/decorators.py", line 17, in new_func
pygeoapi-opendrr_1       |     return f(get_current_context(), *args, **kwargs)
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/openapi.py", line 948, in generate_openapi_document
pygeoapi-opendrr_1       |     click.echo(yaml.safe_dump(get_oas(s), default_flow_style=False))
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/openapi.py", line 933, in get_oas
pygeoapi-opendrr_1       |     return get_oas_30(cfg)
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/openapi.py", line 420, in get_oas_30
pygeoapi-opendrr_1       |     p = load_plugin('provider', get_provider_by_type(
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/plugin.py", line 103, in load_plugin
pygeoapi-opendrr_1       |     plugin = class_(plugin_def)
pygeoapi-opendrr_1       |   File "/pygeoapi/pygeoapi/provider/elasticsearch_.py", line 107, in __init__
pygeoapi-opendrr_1       |     raise ProviderQueryError(err)
pygeoapi-opendrr_1       | pygeoapi.provider.base.ProviderQueryError: NotFoundError(404, 'index_not_found_exception', 'no such index [dsra_acm6p5_beaufort_all_indicators_b]', dsra_acm6p5_beaufort_all_indicators_b, index_or_alias)
pygeoapi-opendrr_1       | ERROR: openapi.yml could not be generated ERROR
opendrr-api_pygeoapi-opendrr_1 exited with code 255
anthonyfok commented 3 years ago

Some possible solutions: