Closed arnaudsj closed 3 years ago
Thanks for the bug report. Spontaneously, I'd say that if some documents could be indexed (which looks like it got up to 5228000 docs), then the problem might be that the particular document differs in structure and may not conform to the mapping elasticsearch expects.
I've seen this with real-world data, even it comes with some "schema", there may be slight problems, e.g. a particular value expressed 99% as int
and 1% as str
or the like - which will confuse the indexer and ES will return a 400.
I wish, I had already implemented a JSON "schema" inference tool, that would surface these kind of problems - but I haven't (yet).
Updates:
$ go get -u -v github.com/miku/jsoninf/cmd/jsoninf
$ jsoninf < file.jsonlines > /dev/null
This command should report type inconsistencies in the input.
@miku thank you for taking a look and writing quickly jsoninf
- I tested it out and confirmed that all my jsonl files were in good shape. It took me a few days to track down the original problem, but I can safely said that the issue is not related to esbulk (at least not directly).
It turned out that we upgraded our ES cluster in AWS (Open Distro) to the latest version in our QA environment (ES 7.9) and it appears that there is some instability/bug that has not 100% been resolved yet. When I switched to push to a different ES cluster, I did not run into the issue at all.
One suggestion I would like to make though for esbulk, is to have a mode where the concurrency is slowly raised over time, and adapts to the cluster on the other side, because it is almost too easy to overwhelm a cluster on AWS (specially since you can't dedicate ingest nodes yet). Other than that, the tool works like a charm and fast! (roughly allowing to index roughly about 1M docs/min in my use case).
Thanks again!
Hi,
I am running into the following error when attempting to index using the following command:
The data is open street map like data formatted in jsonl, and I run into the error after over 5M documents ingested, and I don't see any particular errors on the server side (OpenDistro 1.12 on AWS)
Any ideas on how to actually see the HTTP request in question that generates this error? I am assuming it is particular document, but no particular line number is provided in the error message.
Thank you in advance!