Hi,
Thanks for this wonderful project!
I successfully used it for smaller sites such as meta.chemistry before trying to add Stackoverflow.
I get this error at the following error in the linking phase (in localhost/sites I see linking staging_indices against stackoverflow). I was hoping if I added the site again, it would pick up from the error but turns out it restarts the whole process again. Is there any way to continue after this error?
11:41:52.037 [Thread-9] DEBUG org.tools4j.stacked.index.SeZipFileParser - Parsing /media/p/Seagate Backup Plus Drive/stackexchange/dumps/stackoverflow.com-Votes.7z
Exception in thread "Thread-9" java.lang.OutOfMemoryError: GC overhead limit exceeded
at org.tools4j.stacked.index.UnscoredCollector.search(DocCollector.kt:51)
at org.tools4j.stacked.index.DocIdIndex.searchByQueryForDocs(DocIdIndex.kt:121)
at org.tools4j.stacked.index.DocIdIndex.searchByQuery(DocIdIndex.kt:117)
at org.tools4j.stacked.index.DocIdIndex.searchByTerm(DocIdIndex.kt:80)
at org.tools4j.stacked.index.DocIdIndex.searchByTerm(DocIdIndex.kt:84)
at org.tools4j.stacked.index.QuestionIndexer$index$1.invokeSuspend(QuestionIndexer.kt:21)
at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:238)
at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.kt:116)
at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:80)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
at org.tools4j.stacked.index.QuestionIndexer.index(QuestionIndexer.kt:18)
at org.tools4j.stacked.index.SeDirParserListener.onFinishParseSite(SeDirParserListener.kt:31)
at org.tools4j.stacked.index.SeDirParser.parseSeSite(SeDirParser.kt:60)
at org.tools4j.stacked.index.SeDirParser.parse(SeDirParser.kt:35)
at org.tools4j.stacked.web.Server$Companion$mainServer$5$9$1.run(Server.kt:182)
at java.lang.Thread.run(Thread.java:748)
11:45:43.180 [nettyCallPool-4-2] ERROR org.tools4j.stacked.index.Utils - GC overhead limit exceeded
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.Double.valueOf(Double.java:519)
at io.ktor.features.Compression$comparator$2.invoke(Compression.kt:54)
at kotlin.comparisons.ComparisonsKt__ComparisonsKt.compareValuesByImpl$ComparisonsKt__ComparisonsKt(Comparisons.kt:25)
at kotlin.comparisons.ComparisonsKt__ComparisonsKt.access$compareValuesByImpl(Comparisons.kt:1)
at kotlin.comparisons.ComparisonsKt__ComparisonsKt$compareBy$1.compare(Comparisons.kt:91)
at java.util.Collections$ReverseComparator2.compare(Collections.java:5232)
at java.util.TimSort.countRunAndMakeAscending(TimSort.java:355)
at java.util.TimSort.sort(TimSort.java:220)
at java.util.Arrays.sort(Arrays.java:1438)
at kotlin.collections.ArraysKt___ArraysJvmKt.sortWith(_ArraysJvm.kt:1862)
at kotlin.collections.CollectionsKt___CollectionsKt.sortedWith(_Collections.kt:947)
at io.ktor.features.Compression.interceptor(Compression.kt:75)
at io.ktor.features.Compression$Feature$install$1.invokeSuspend(Compression.kt:174)
at io.ktor.features.Compression$Feature$install$1.invoke(Compression.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
at io.ktor.util.pipeline.SuspendFunctionGun.proceedWith(PipelineContext.kt:147)
at io.ktor.features.ContentNegotiation$Feature$install$2.invokeSuspend(ContentNegotiation.kt:90)
at io.ktor.features.ContentNegotiation$Feature$install$2.invoke(ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
at org.tools4j.stacked.web.Server$Companion$mainServer$5$10.invokeSuspend(Server.kt:238)
at org.tools4j.stacked.web.Server$Companion$mainServer$5$10.invoke(Server.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(PipelineContext.kt:278)
at io.ktor.util.pipeline.SuspendFunctionGun.access$loop(PipelineContext.kt:63)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(PipelineContext.kt:137)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(PipelineContext.kt:157)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:23)
Hi, Thanks for this wonderful project! I successfully used it for smaller sites such as
meta.chemistry
before trying to add Stackoverflow. I get this error at the following error in the linking phase (inlocalhost/sites
I seelinking staging_indices
against stackoverflow). I was hoping if I added the site again, it would pick up from the error but turns out it restarts the whole process again. Is there any way to continue after this error?