crawlab-team / crawlab

Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台,支持任何语言和框架
https://www.crawlab.cn
BSD 3-Clause "New" or "Revised" License
11.37k stars 1.79k forks source link

使用elastic数据源,存储数据失败 #1427

Open aibots-team opened 11 months ago

aibots-team commented 11 months ago

Describe the bug 使用elastic数据源,存储数据失败

2023-12-05 20:57:18 "version" : { 2023-12-05 20:57:18 "number" : "8.10.2", 2023-12-05 20:57:18 "build_flavor" : "default", 2023-12-05 20:57:18 "build_type" : "docker", 2023-12-05 20:57:18 "build_hash" : "6d20dd8ce62365be9b1aca96427de4622e970e9e", 2023-12-05 20:57:18 "build_date" : "2023-09-19T08:16:24.564900370Z", 2023-12-05 20:57:18 "build_snapshot" : false, 2023-12-05 20:57:18 "lucene_version" : "9.7.0", 2023-12-05 20:57:18 "minimum_wire_compatibility_version" : "7.17.0", 2023-12-05 20:57:18 "minimum_index_compatibility_version" : "7.0.0" 2023-12-05 20:57:18 }, 2023-12-05 20:57:18 "tagline" : "You Know, for Search"

2023-12-05 20:55:46 /usr/local/go/src/runtime/asm_amd64.s:1571 runtime.goexit() 2023-12-05 20:55:46 [ElasticsearchService] [400 Bad Request] error inserting record: {"_col":"results_Quotes","_tid":"656f1dcaa46a17eb1b2af5c4","author":["Dr. Seuss"],"tags":[["inspirational"]],"text":["“A person's a person, no matter how small.”"]} 2023-12-05 20:55:46 /go/pkg/mod/github.com/crawlab-team/go-trace@v0.1.1/trace.go:6 github.com/crawlab-team/go-trace.PrintError() 2023-12-05 20:55:46 /go/pkg/mod/github.com/crawlab-team/crawlab-core@v0.6.3-0.20231031044528-37e6d73eb203/ds/es.go:82 github.com/crawlab-team/crawlab-core/ds.(*ElasticsearchService).Insert.func1()

测试爬虫代码 656ec29d17dc482c2e4e5456.zip

aibots-team commented 11 months ago

使用docker部署的0.6.3版本