Block remapping log spam fixed by deleting local map - related to selected GC somehow

[douira]

This is a very mysterious issue I've experienced with surveyor. If using ZGC or Shen GC it hangs when joining a server, at "joining server" which then ends with a crash or an error message. Updating to 0.6.24 did not fix this. I suspect the same issue happens with G1GC but it's somehow able to handle it without timing out.

However, I removed (but not deleted) my local map cache and then it worked. Prior to moving the map cache away, it writes hundreds of megabytes of "remapping" log messages. Before updating to latest 1.20.1 AA this logging didn't happen but it might still be remapping stuff which might have been the cause of not being able to join servers.

I've trimmed this log file because it was 200MB https://mclo.gs/FywGkgX

I also have a JFR profile available if you need it.

Map data: -2880368472433132268 old.zip

Screenshot of the profile:

Stack of the leftmost leaf in this flamegraph:

net.minecraft.class_3513.method_15223(int)
net.minecraft.class_3513.method_15230(Object, int)
net.minecraft.class_3513.method_15225(Object)
folk.sisby.surveyor.terrain.RegionSummary.lambda$readNbt$3(RegionSummary, class_2378, class_2248)
folk.sisby.surveyor.terrain.RegionSummary$$Lambda+0x00000007248a0f50.1463091743.accept(Object)
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(Object)
java.util.stream.ReferencePipeline$3$1.accept(Object)
java.util.Iterator.forEachRemaining(Consumer)
java.util.Spliterators$IteratorSpliterator.forEachRemaining(Consumer)
java.util.stream.AbstractPipeline.copyInto(Sink, Spliterator)
java.util.stream.AbstractPipeline.wrapAndCopyInto(Sink, Spliterator)
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(PipelineHelper, Spliterator)
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(PipelineHelper, Spliterator)
java.util.stream.AbstractPipeline.evaluate(TerminalOp)
java.util.stream.ReferencePipeline.forEach(Consumer)
folk.sisby.surveyor.terrain.RegionSummary.readNbt(class_2487, class_5455)
folk.sisby.surveyor.terrain.WorldTerrainSummary.lambda$load$8(Map, class_1937, class_1923, class_2487)
folk.sisby.surveyor.terrain.WorldTerrainSummary$$Lambda+0x000000072489f360.2106052811.accept(Object, Object)
java.util.HashMap.forEach(BiConsumer)
folk.sisby.surveyor.terrain.WorldTerrainSummary.load(class_1937, File)
folk.sisby.surveyor.WorldSummary.load(class_1937, File, boolean)
net.minecraft.class_638.surveyor$getSummary()
folk.sisby.surveyor.WorldSummary.of(class_1937)
folk.sisby.surveyor.client.SurveyorClient$ClientExploration.onLoad()
folk.sisby.surveyor.client.SurveyorClient$$Lambda+0x00000007248861f8.865250577.run()
net.minecraft.class_1255.method_18859(Runnable)
net.minecraft.class_4093.method_18859(Runnable)
net.minecraft.class_1255.method_16075()
net.minecraft.class_1255.method_5383()
net.minecraft.class_310.method_1523(boolean)
net.minecraft.class_310.method_1514()
net.minecraft.client.main.Main.main(String[])
java.lang.invoke.LambdaForm$DMH+0x0000000723004400.679890578.invokeStaticInit(Object, Object)
java.lang.invoke.LambdaForm$MH+0x0000000723004c00.1792393294.invokeExact_MT(Object, Object, Object)
net.fabricmc.loader.impl.game.minecraft.MinecraftGameProvider.launch(ClassLoader)
net.fabricmc.loader.impl.launch.knot.Knot.launch(String[], EnvType)
net.fabricmc.loader.impl.launch.knot.KnotClient.main(String[])
java.lang.invoke.LambdaForm$DMH+0x0000000723004400.679890578.invokeStaticInit(Object, Object)
java.lang.invoke.LambdaForm$MH+0x0000000723004c00.1792393294.invokeExact_MT(Object, Object, Object)
org.prismlauncher.launcher.impl.StandardLauncher.launch()
org.prismlauncher.EntryPoint.listen()
org.prismlauncher.EntryPoint.main(String[])

[sisby-folk]

Oh that's really weird. We've been trying to figure out what's holding up load performance with very large save files? (#23) But we find it very very difficult to profile as we're less experienced there. If this is a memory issue on load causing a crash we might be able to solve two problems with one. I'll see if i can get this map data to load and cause the same issue.

Could you identify the hottest parts of the load process? your screenshot has expired for some reason.

[sisby-folk]

I'm just going to provide a sidenote here - surveyor still loads map data by loading the entirety of the current world's map data from the disk, it doesn't have an implementation like TACS for streaming terrain regions in/out. There is an upper limit for how much map data can be loaded at once somewhere that we can only solve with a streaming solution eventually, we just don't have the skills for it at the moment.