search

o19s / hello-nlp

A natural language search microservice
Other
95 stars 12 forks source link

Unable to index content #22

Closed binarymax closed 3 years ago

binarymax commented 3 years ago

I wasn't sure where to start, and decided to run some scripts in the example/, in particular index-one.sh (taking the Elasticsearch route for starters).

The following one-liner returns some response (with vector information):

curl --header "Content-Type: application/json" \
  --request POST \
  --data @blog-posts-one.json \
  http://localhost:5055/enrich/osc-blog

But this one fails:

curl --header "Content-Type: application/json" \
  --request POST \
  --data @blog-posts-one.json \
  http://localhost:5055/index/osc-blog

Not sure why? Then I tried to reindex:

curl --header "Content-Type: application/json"   --request POST   http://localhost:5055/reindex/osc-blog

But I get {"error":{}} and in the logs seeing this:

('NETWORK ERROR! Could not connect to Elasticsearch server on', 'http://localhost:9200/', ' ... Have a nice day.')
INFO:     127.0.0.1:54416 - "POST /reindex/osc-blog HTTP/1.1" 200 OK

The CORS is on and configured like so (elasticsearch 7.10.1):

# CORS
http.cors.allow-origin: "/https?:\\/\\/(.*?\\.)?(quepid\\.com|splainer\\.io)|localhost/"
http.cors.enabled: true

Originally posted by @DmitryKey in https://github.com/o19s/hello-nlp/issues/19#issuecomment-744032592

binarymax commented 3 years ago

Hi @DmitryKey I think I know what the problem is - that the osc-blog index does not exist, and is not automatically being created because there is an issue in a dependency that is meant to handle it. The dependency issue was fixed today, so I will to upgrade the dependency and add some better logic to take care of this.

llermaly commented 2 years ago

Hello @binarymax I'm having a similar problem. The osc-blog index is not being created. I can enrich a document but not index it.

This is the output:

hello-nlp-elasticsearch_1  | INFO:     172.31.0.1:58390 - "POST /enrich/osc-blog HTTP/1.1" 200 OK
hello-nlp-elasticsearch_1  | INFO:     172.31.0.1:58394 - "POST /index/osc-blog HTTP/1.1" 500 Internal Server Error
hello-nlp-elasticsearch_1  | ERROR:    Exception in ASGI application
hello-nlp-elasticsearch_1  | Traceback (most recent call last):
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 390, in run_asgi
hello-nlp-elasticsearch_1  |     result = await app(self.scope, self.receive, self.send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
hello-nlp-elasticsearch_1  |     return await self.app(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/fastapi/applications.py", line 179, in __call__
hello-nlp-elasticsearch_1  |     await super().__call__(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/applications.py", line 111, in __call__
hello-nlp-elasticsearch_1  |     await self.middleware_stack(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in __call__
hello-nlp-elasticsearch_1  |     raise exc from None
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in __call__
hello-nlp-elasticsearch_1  |     await self.app(scope, receive, _send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/middleware/cors.py", line 78, in __call__
hello-nlp-elasticsearch_1  |     await self.app(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/exceptions.py", line 82, in __call__
hello-nlp-elasticsearch_1  |     raise exc from None
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in __call__
hello-nlp-elasticsearch_1  |     await self.app(scope, receive, sender)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 566, in __call__
hello-nlp-elasticsearch_1  |     await route.handle(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 227, in handle
hello-nlp-elasticsearch_1  |     await self.app(scope, receive, send)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 41, in app
hello-nlp-elasticsearch_1  |     response = await func(request)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/fastapi/routing.py", line 183, in app
hello-nlp-elasticsearch_1  |     dependant=dependant, values=values, is_coroutine=is_coroutine
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/fastapi/routing.py", line 133, in run_endpoint_function
hello-nlp-elasticsearch_1  |     return await dependant.call(**values)
hello-nlp-elasticsearch_1  |   File "./hello_nlp/main.py", line 190, in index_document
hello-nlp-elasticsearch_1  |     iq.indexDocument(enriched)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/skipchunk/indexquery.py", line 29, in indexDocument
hello-nlp-elasticsearch_1  |     return self.engine.index([document],timeout=timeout)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/skipchunk/elastic.py", line 182, in index
hello-nlp-elasticsearch_1  |     isIndex = self.indexExists(self.name)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/skipchunk/elastic.py", line 107, in indexExists
hello-nlp-elasticsearch_1  |     r = requests.get(uri)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 75, in get
hello-nlp-elasticsearch_1  |     return request('get', url, params=params, **kwargs)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/requests/api.py", line 61, in request
hello-nlp-elasticsearch_1  |     return session.request(method=method, url=url, **kwargs)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
hello-nlp-elasticsearch_1  |     resp = self.send(prep, **send_kwargs)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
hello-nlp-elasticsearch_1  |     r = adapter.send(request, **kwargs)
hello-nlp-elasticsearch_1  |   File "/usr/local/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
hello-nlp-elasticsearch_1  |     raise ConnectionError(e, request=request)
hello-nlp-elasticsearch_1  | requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=9200): Max retries exceeded with url: /osc-blog (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x42b3774950>: Failed to establish a new connection: [Errno 111] Connection refused'))

I tried with and without creating a osc-blog index manually , my elasticsearch cluster is up and running at localhost:9200

Any idea? Thanks