search

halo-dev / halo

强大易用的开源建站工具。
https://www.halo.run
GNU General Public License v3.0
33.24k stars 9.59k forks source link

2.13 版本搜索插件检索500，重置索引后，短暂恢复后再次错误，且重新刷新索引会出错 #5674

Closed lan-yonghui closed 4 months ago

lan-yonghui commented 5 months ago

系统信息

使用的哪种方式运行？

Docker Compose

发生了什么？

搜索插件 检索索引问题，刷新索引后同样不能检索。升级到2.14版本后没有出现检索错误的问题，猜测和内存有关，升级后内存得到了释放。。后续会再观测下是否会再次出现！！

相关日志输出

2024-04-09T12:03:18.504+08:00 ERROR 7 --- [run.halo.app.search.post.PostEventReconciler-t-1] r.h.a.e.controller.DefaultController     : Reconciler in run.halo.app.search.post.PostEventReconciler-worker-1 aborted with an error, re-enqueuing...

reactor.core.Exceptions$ReactiveException: java.io.IOException: Map failed: MemorySegmentIndexInput(path="/root/.halo2/indices/posts/_3.si") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 342 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
    at reactor.core.Exceptions$ReactiveException.fillInStackTrace(Exceptions.java:736) ~[reactor-core-3.6.3.jar:3.6.3]
    at java.base/java.lang.Throwable.<init>(Unknown Source) ~[na:na]
    at java.base/java.lang.Throwable.<init>(Unknown Source) ~[na:na]
    at java.base/java.lang.Exception.<init>(Unknown Source) ~[na:na]
    at java.base/java.lang.RuntimeException.<init>(Unknown Source) ~[na:na]
    at reactor.core.Exceptions$ReactiveException.<init>(Exceptions.java:726) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.Exceptions.propagate(Exceptions.java:410) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:101) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Mono.block(Mono.java:1779) ~[reactor-core-3.6.3.jar:3.6.3]
    at run.halo.app.search.post.PostEventReconciler.addPostDoc(PostEventReconciler.java:101) ~[classes/:2.13.1]
    at run.halo.app.search.post.PostEventReconciler.reconcile(PostEventReconciler.java:56) ~[classes/:2.13.1]
    at run.halo.app.search.post.PostEventReconciler.reconcile(PostEventReconciler.java:29) ~[classes/:2.13.1]
    at run.halo.app.extension.controller.DefaultController$Worker.run(DefaultController.java:163) ~[api-2.13.1.jar:na]
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:na]
    at java.base/java.util.concurrent.FutureTask.run(Unknown Source) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) ~[na:na]
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) ~[na:na]
    at java.base/java.lang.Thread.run(Unknown Source) ~[na:na]
    Suppressed: java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:103) ~[reactor-core-3.6.3.jar:3.6.3]
        ... 10 common frames omitted
Caused by: java.io.IOException: Map failed: MemorySegmentIndexInput(path="/root/.halo2/indices/posts/_3.si") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 342 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
    at java.base/sun.nio.ch.FileChannelImpl.mapInternal(Unknown Source) ~[na:na]
    at java.base/sun.nio.ch.FileChannelImpl.map(Unknown Source) ~[na:na]
    at org.apache.lucene.store.MemorySegmentIndexInputProvider.map(MemorySegmentIndexInputProvider.java:108) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.store.MemorySegmentIndexInputProvider.openInput(MemorySegmentIndexInputProvider.java:57) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:323) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.codecs.lucene99.Lucene99SegmentInfoFormat.read(Lucene99SegmentInfoFormat.java:94) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.index.SegmentInfos.parseSegmentInfos(SegmentInfos.java:411) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:368) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:304) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:293) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:1086) ~[lucene-core-9.9.1.jar:9.9.1 eee32cbf5e072a8c9d459c349549094230038308 - 2023-12-13 11:03:02]
    at run.halo.app.search.post.LucenePostSearchService.addDocuments(LucenePostSearchService.java:122) ~[classes/:2.13.1]
    at run.halo.app.search.post.PostEventReconciler.lambda$addPostDoc$0(PostEventReconciler.java:94) ~[classes/:2.13.1]
    at reactor.core.publisher.FluxPeek$PeekSubscriber.onNext(FluxPeek.java:185) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:129) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Operators$ScalarSubscription.request(Operators.java:2571) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.set(Operators.java:2367) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onSubscribe(Operators.java:2241) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.MonoJust.subscribe(MonoJust.java:55) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Mono.subscribe(Mono.java:4563) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onComplete(FluxSwitchIfEmpty.java:82) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxHandle$HandleSubscriber.onComplete(FluxHandle.java:223) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onComplete(FluxMap.java:275) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxFilter$FilterConditionalSubscriber.onComplete(FluxFilter.java:300) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxDefaultIfEmpty$DefaultIfEmptySubscriber.onComplete(FluxDefaultIfEmpty.java:139) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onComplete(FluxMapFuseable.java:152) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.MonoFlatMap$FlatMapMain.secondComplete(MonoFlatMap.java:246) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.MonoFlatMap$FlatMapInner.onNext(MonoFlatMap.java:305) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:74) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:122) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.MonoUsingWhen$MonoUsingWhenSubscriber.deferredComplete(MonoUsingWhen.java:268) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.FluxUsingWhen$CommitInner.onComplete(FluxUsingWhen.java:532) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.Operators.complete(Operators.java:137) ~[reactor-core-3.6.3.jar:3.6.3]
    at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:46) ~[reactor-core-3.6.3.jar:3.6.3]

附加信息

halo-log-2024-04-09 12_03.log

JohnNiang commented 5 months ago

/triage duplicate

Duplicate of https://github.com/halo-dev/halo/issues/5621.

建议根据以下关键日志提示的操作尝试修复一下：

Caused by: java.io.IOException: Map failed: MemorySegmentIndexInput(path="/root/.halo2/indices/posts/_3.si") [this may be caused by lack of enough unfragmented virtual address space or too restrictive virtual memory limits enforced by the operating system, preventing us to map a chunk of 342 bytes. Please review 'ulimit -v', 'ulimit -m' (both should return 'unlimited'), and 'sysctl vm.max_map_count'. More information: https://blog.thetaphi.de/2012/07/use-lucenes-mmapdirectory-on-64bit.html]
guqing commented 5 months ago

/triage duplicate

Duplicate of #5621.

建议根据以下关键日志提示的操作尝试修复一下：

和 https://github.com/halo-dev/halo/issues/5621 貌似不是一个错误吧，使用 docker compose 部署的而且删除 indices 目录后重建过几分钟又会出现此问题

lan-yonghui commented 4 months ago

4.23 出现了同样的问题，同样是重置后会刷新索引失败，现改用 meilisearch 插件来进行索引，看后续检索情况！！

LiberatorHg commented 4 months ago

一样的问题，用meilisearch代替一直转圈圈，也不好使

lan-yonghui commented 4 months ago

一样的问题，用meilisearch代替一直转圈圈，也不好使

meilisearch 目前没问题，使用 meilisearch 需要重启后在控制台重新刷新索引！！

LiberatorHg commented 4 months ago

一样的问题，用meilisearch代替一直转圈圈，也不好使

meilisearch 目前没问题，使用 meilisearch 需要重启后在控制台重新刷新索引！！

我找到了几个月前备份的数据，然后把indices目录替换成之前的备份，暂时是好的，