Connection Error. Is Haystack running? JSON Issue with Docker Compose On Fresh Repo and Image Pulls

[grahamschuckman]

Discussed in https://github.com/deepset-ai/haystack/discussions/1617

^{Originally posted by **sieu-tran** October 19, 2021} Between October 18, 2021 and October 19, 2021, something has changed and the docker returns the error: "Connection Error. Is Haystack running?" when we start running it.

[grahamschuckman]

Terminal output when running docker-compose up from clean repo download and image pulls:

haystack-api_1 | ML Logging is turned off. No parameters, metrics or artifacts will be logged to MLFlow. haystack-api_1 | [2021-10-19 19:16:54 +0000] [10] [INFO] Started server process [10] haystack-api_1 | [2021-10-19 19:16:54 +0000] [10] [INFO] Waiting for application startup. haystack-api_1 | [2021-10-19 19:16:54 +0000] [10] [INFO] Application startup complete. haystack-api_1 | [2021-10-19 19:16:54 +0000] [10] [ERROR] Exception in ASGI application haystack-api_1 | Traceback (most recent call last): haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/protocols/http/httptools_impl.py", line 375, in run_asgi haystack-api_1 | result = await app(self.scope, self.receive, self.send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in call haystack-api_1 | return await self.app(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/fastapi/applications.py", line 208, in call haystack-api_1 | await super().call(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/applications.py", line 112, in call haystack-api_1 | await self.middleware_stack(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/middleware/errors.py", line 181, in call haystack-api_1 | raise exc haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/middleware/errors.py", line 159, in call haystack-api_1 | await self.app(scope, receive, _send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/middleware/cors.py", line 84, in call haystack-api_1 | await self.app(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/exceptions.py", line 82, in call haystack-api_1 | raise exc haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/exceptions.py", line 71, in call haystack-api_1 | await self.app(scope, receive, sender) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 656, in call haystack-api_1 | await route.handle(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 259, in handle haystack-api_1 | await self.app(scope, receive, send) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/routing.py", line 61, in app haystack-api_1 | response = await func(request) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/fastapi/routing.py", line 227, in app haystack-api_1 | dependant=dependant, values=values, is_coroutine=is_coroutine haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/fastapi/routing.py", line 161, in run_endpoint_function haystack-api_1 | return await run_in_threadpool(dependant.call, *values) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/starlette/concurrency.py", line 39, in run_in_threadpool haystack-api_1 | return await anyio.to_thread.run_sync(func, args) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/anyio/to_thread.py", line 29, in run_sync haystack-api_1 | limiter=limiter) haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 805, in run_sync_in_worker_thread haystack-api_1 | return await future haystack-api_1 | File "/usr/local/lib/python3.7/site-packages/anyio/_backends/_asyncio.py", line 743, in run haystack-api_1 | result = func(args) haystack-api_1 | File "/home/user/rest_api/controller/search.py", line 48, in query haystack-api_1 | result = _process_request(PIPELINE, request) haystack-api_1 | File "/home/user/rest_api/controller/search.py", line 66, in _process_request haystack-api_1 | result = pipeline.run(query=request.query, params=params) haystack-api_1 | File "/home/user/haystack/pipeline.py", line 300, in run haystack-api_1 | raise ValueError(f"No node(s) or global parameter(s) named {', '.join(invalid_keys)} found in pipeline.") haystack-api_1 | ValueError: No node(s) or global parameter(s) named ESRetriever found in pipeline. ui_1 | 2021-10-19 19:16:54.134 Expecting value: line 1 column 1 (char 0) ui_1 | Traceback (most recent call last): ui_1 | File "/home/user/webapp.py", line 134, in main ui_1 | results, raw_json = retrieve_doc(question, top_k_reader=top_k_reader, top_k_retriever=top_k_retriever) ui_1 | File "/usr/local/lib/python3.7/site-packages/streamlit/caching.py", line 545, in wrapped_func ui_1 | return get_or_create_cached_value() ui_1 | File "/usr/local/lib/python3.7/site-packages/streamlit/caching.py", line 527, in get_or_create_cached_value ui_1 | return_value = func(args, kwargs) ui_1 | File "/home/user/utils.py", line 30, in retrieve_doc ui_1 | response_raw = requests.post(url, json=req).json() ui_1 | File "/usr/local/lib/python3.7/site-packages/requests/models.py", line 910, in json ui_1 | return complexjson.loads(self.text, kwargs) ui_1 | File "/usr/local/lib/python3.7/json/init.py", line 348, in loads ui_1 | return _default_decoder.decode(s) ui_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode ui_1 | obj, end = self.raw_decode(s, idx=_w(s, 0).end()) ui_1 | File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode ui_1 | raise JSONDecodeError("Expecting value", s, err.value) from None ui_1 | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[ZanSara]

@grahamschuckman @sieu-tran The fix is merged, have a look if it now works for you. If not, please re-open.

[grahamschuckman]

@ZanSara Thanks for getting back to us so quickly! I've relaunched the containers and it appears to be working again. I'll let you know if anything else comes up! Also, I noticed that the repo has been incredibly active with the release of Haystack 1.0 coming up. Can you share when that might occur and what other significant code changes might be made before then?

[ZanSara]

Well that's a very broad question :smile: We've made a lot of changes for the 1.0 release and on top of my mind is quite hard to recall them all. A big breaking change is mentioned here (#1590), and another big structural refactoring of the haystack package is coming, hopefully backwards-compatible, but probably not at 100%. So if you're using the latest master for your production usecases, watch out in the next days :slightly_smiling_face: At some point we're going to draft some release notes, but for now it's not done yet.

[grahamschuckman]

@ZanSara Got it, we'll keep an eye on the repo and pull as needed for the new commits. Not sure if this is in your domain, but we've been searching for some resources on switching the Docker tutorial from an ExtractiveQA pipeline to a DocumentSearch pipeline. I've found plenty of information on how to switch the Reader and DocumentStore, but not a lot on whether or not the default Docker demo can actually be done with a different Retriever. Any guidance or people you can point us to for this?

[brandenchan]

Hi @grahamschuckman, I'm gonna jump in here and try to help you out. Yes, in the default Docker demo you can switch out whichever components you'd like. You'll want to make changes to haystack/rest_api/pipeline/pipelines.yaml( here). Here's a blog article that gives some pointers on how to work with these yaml config files.

Hope this helps! Let me know how you get on with this :)

[grahamschuckman]

Hi @brandenchan, I actually had a meeting with @Timoeller yesterday to discuss how to change the pipelines.yaml file you mentioned so that our Docker environment (based off the demo) could use a DocumentSearch pipeline instead of an ExtractiveQA pipeline. He was able to help me understand that we needed to remove the Reader nodes and add a DocsConverter node and component of type Docs2Answers to still work with the API. I've attached the document as reference, though I believe it would be incredibly beneficial to the community to have additional documentation or tutorials on how to modify that pipelines.yaml file for other pipelines besides ExtractiveQA, which is what is present in all other articles and notebooks. pipelines.txt

[tholor]

The original issue seems resolved. The missing support & documentation for DocumentSearch pipelines is definitely good feedback and should be handled together with #1863.