Infinite recursion in stream generation, watchdog timeout

[Vagor96]

When playing SevTech: Ages 3.1.1 the game stalls reproducible at a specific location (animals stop moving, no item drops). The issue seems to be related with this mod according to the attached server crashlog.

Steps to reproduce:

• Install SevTech: Ages 3.1.1 via curseforge
• Generate world with seed 3822593286363813618
• Head westwards until you reach a plain
• Head north until the game stalls (near x:80 z:-250)

crash-2019-04-22_15.29.26-server.txt

[Vagor96]

Still present in Streams 0.4.5 with Farseek 2.3.2.

[delvr]

Thanks for the update. Do you know when this behavior started, for example with a version bump of the modpack or one of its component mods?

[Vagor96]

I just did some tests with SevTech 3.0.8, 3.0.7 and 3.0.6 and the game always stalls, which is strange since i used a seed that was recommended for 3.0.6. But in contrast to 3.1.1 no stall report is created and only a minor problem with primal core is logged, that seems to be fixed since then:

PrimalCore loaded a new chunk (-10, 0 Dimension: 0) during chunk population, causing cascading worldgen lag

Unfortunately, I am not familiar with debugging Minecraft mods at all, which is why I can not provide more information. I do not even know if Streams really is the real problem.

[the-vindex]

We have a bit similar issue, streams seem to stall server until watchdog timeouts after 600 seconds and kills the server. It's clear from stall log, that it's recursively generates stream. We have lost cities as well, though

http://sprunge.us/3LtgZg

Versions: mods/Streams-1.12-0.4.4.jar (reproducible with 0.4.6) mods/Farseek-1.12-2.3.1.jar (reproducible with 2.3.2)

[delvr]

It's expected that Streams will cause occasional lag spikes when it generates a river in a newly-explored area, more so if other worldgen mods are present. However it should never become infinitely-recursive unless there's a buggy interaction with another mod. I suggest increasing or disabling the server tick timeout as described here; if this causes a different crash such as a StackOverflowError please let me know.

p.s. @the-vindex could you clarify your "lost cities" comment, was there some world corruption involved?

[Vagor96]

It doesn't generate a crash. It just infinitely stalls the server. I aborted the test after 20 minutes.

stall-2019-05-06_14.23.49.txt stall-2019-05-06_14.24.20-1.txt stall-2019-05-06_14.25.21-2.txt stall-2019-05-06_14.27.21-3.txt stall-2019-05-06_14.31.22-4.txt stall-2019-05-06_14.39.22-5.txt

[delvr]

Thanks @Vagor96, that does seem like infinite recursion in the presence of some other mods. What kind of world generator was this running in?

[Vagor96]

The world generates with the preset "quark_realistic" from Quark. Dimensional Control also seems to have something to do with world generation. Biomes are at least added by AbyssalCraft, Integrated Dynamics and Traverse. I tried it again with the most recent versions of this mods, but the error is still present.

But like I said, I do not have much idea of minecraft modding. But I know that the error is reproducible with my description above. Full modlist: Modlist_SevTech_3.1.1.pdf

[the-vindex]

We crash on the same, infinite loop in generation of world. Our mod list http://sprunge.us/c6voFF

[Vagor96]

The buggy interaction seems to be with Traverse (github, discord). If i remove the mod i cannot reproduce the crash anymore. @the-vindex Can you confirm this?

For completeness sake the merged (common lines) modlist: merged.txt

[Vagor96]

I did a few more tests and was able to exclude most of the mods in the merged list as the cause. Interestingly enough, the error does not occur to me if at least one of the following mods is removed:

• Integrated Dynamics (depending on CyclopsCore)
• Quark (depending on AutoRegLib)
• Traverse
• Streams

That it does not matter which of these mods I remove, is probably only because it changes the result of the given seed. But suppose @the-vindex and I have really the same problem and it is a compatibility issue, then only these mods come into question. In particular Traverse, because we share only the underlying libraries in the other two mods.

I hope that this information will suffice as an approach to find the issue as soon as possible, since I can barely use my savegame until then.

[delvr]

Many thanks @Vagor96 for your help investigating this issue. I'll take another look soon and hopefully find the root cause.

[Vagor96]

Any news on this?

[dkambersky]

This issue is causing somewhat frequent crashes that are hard to recover from on our server - when a player explores into an area that causes that bug on generation, they crash the server each time they log in subsequently, before an op manages to TP them out while they're logging in before the chunk gen kicks in.

We're running SevTech 3.1.2 for context.

Anything we could do to help with triage? Would extra crash logs help?

[the-vindex]

We ended up removing streams from the last instance. We detected the problem only due to running 10k x 10k pregen. Removing it from the existing world also worked, but left ugly dry-beds.

[demfels]

I have encountered this same problem I believe. When the player enters the 'bugged' area, it triggers a never-ending generation of streams. As you can see by this server log entry which is insane... It appears it is generating rivers out to the 470000x coordinate and that was only after 30 minutes or so.

Any suggestions on how to stop this from generating? I'd just like to preserve existing streams and stop any new ones from generating... or something? If you remove the mod and add it back would it stop this without having existing streams all end up being dried up?

9:41:40 PM streams: River at tp 470942 71 279
9:41:46 PM streams: River at tp 472087 68 342
9:41:47 PM streams: River at tp 472808 63 315
9:41:54 PM streams: River at tp 474647 64 324
9:41:57 PM streams: River at tp 475624 67 288
9:42:00 PM streams: River at tp 476255 68 488
9:42:00 PM streams: River at tp 476538 63 488
9:42:07 PM streams: River at tp 478440 63 405
9:42:11 PM streams: River at tp 479902 69 279
9:42:22 PM streams: River at tp 483560 64 378
9:42:25 PM streams: River at tp 484438 63 488
9:42:26 PM streams: River at tp 484631 65 459

I'd really appreciate a work around for this, even if it means no further rivers are generated. Right now, the best workaround I can come up with is to remove the mod and leave giant ravines of dried-up rivers all over the player map. Not a great option...

[Vagor96]

If we still follow the compatibility problem theory and assuming that all the tickets being referenced have the same cause, then it probably has something to do with AutoRegLib. This mod is the only common denominator. Is there anything that we can do to help you fixing this issue? @delvr

[Vagor96]

delvr/Farseek#51 doesn't back up this theory though. But it has once again traverse installed.

[delvr]

Thanks Vagor96 for following up on this. Sorry for the very long delay on my end. It's been a really difficult crash to reproduce, especially since reports tend to involve so many different mods my machine can barely run them. Anyway, I've just released Farseek 2.4 with an optimization to Farseek/Streams chunk generation that may prevent these infinite-generation issues. Please give it a try and let me know if it improves anything. (Or if it breaks with a different error). Thanks!

[Vagor96]

Thank you for the update! Unfortunately, it still does not work. stall-2019-11-27_22.11.06.txt

[delvr]

Ah, that's disappointing. I'll have to try something else.

Looking at the stack traces again, one line surprises me a bit:

at net.minecraft.world.gen.ChunkProviderServer.redirect$generateChunk$zzg000(ChunkProviderServer.java:523)

This .redirect extension is generally associated with Sponge, which SevTech apprently doesn't use. Is it possible one of the mods in use (whether inside the modpack or outside) bundles some Sponge mixins?

[Vagor96]

I'm sorry but I have no idea. But i asked in the SevTech repo if anyone could help. If you tell me what to do, I can also try to do more tests as soon as I have time.

[sam-kirby]

This strange method signature is indeed the result of a mixin, though not one written by Sponge. It does nothing but call the normal generator (which is the line above in the stack trace, indicating it got no further than this), then initialise an integer biome array. It is part of JEID, source for this version here: https://github.com/DimensionalDevelopment/JustEnoughIDs/blob/e384b99cc18e136e999a89d55e5eedcbb5445168/src/main/java/org/dimdev/jeid/mixin/core/MixinChunkProviderServer.java#L21-L32

I'm pretty sure I'll be able to replicate this issue without that line appearing in the stack trace.

[sam-kirby]

stall-2019-12-04_17.44.52.txt

[ProsperCraft]

Here is a fresh report- I had it while generating out to 10k on my TownCraft modpack - No sponge on my server, actually using Mohist-1150e37-server.

Stall report- https://gist.github.com/ProsperCraft/bf1e265f31ca6fe9f0ff57af07187091

more stall reports- stall-2019-12-15_05.57.17-6.txt stall-2019-12-15_06.29.18-7.txt stall-2019-12-15_06.54.15.txt stall-2019-12-15_06.54.46-1.txt stall-2019-12-15_06.55.46-2.txt stall-2019-12-15_06.57.47-3.txt stall-2019-12-15_07.01.47-4.txt stall-2019-12-15_05.25.42.txt stall-2019-12-15_05.26.13-1.txt stall-2019-12-15_05.27.14-2.txt stall-2019-12-15_05.29.15-3.txt stall-2019-12-15_05.33.16-4.txt stall-2019-12-15_05.41.17-5.txt

I got to 132,401 chunks generated out of 160,801

The server doesn't crash, it just quits responding and doesn't kick the clients.

Here is my log- I will try without Vanillafix as it seems to be the only thing mentioning mixins.

latest.log

Farseek-1.12-2.4 Streams-1.12-0.4.7

[ProsperCraft]

Disabling foamfix did not help, it still instalocks when chunk pregenerator starts back up on reboot-

latest.log stall-2019-12-15_07.25.26.txt stall-2019-12-15_07.25.56-1.txt

[ProsperCraft]

@Vagor96 perhaps remove (SevTech: Ages) from the topic as it happens in other packs too?

[Lairdjoker]

Hi, been having similar issues as above. Server crashed when exploring. Has any progress been made determining the issue or a hotfix?

Farseek-1.12-2.4.jar Streams-1.12-0.4.7.jar

crash-2019-12-15_17.01.53-server.txt

Thanks.

[delvr]

I haven't found the root cause yet, but I do have a shiny new computer that can run SevTech reliably, so I can reproduce the bug more easily now. I hope to have news soon!

[ProsperCraft]

Here is another stall report from the server locking up hard when a player discovered the problem area.

https://gist.github.com/ProsperCraft/ab696cd19d5810a7a8cfcecb2a05af81

[delvr]

Good news, I think I finally fixed it! 🎉 It works for the problematic area in SevTech at least, we'll have to see for the other bug reports.

Thanks everyone for your patience. I expect to upload the new version tonight.

[delvr]

Streams 0.4.8 released! This should fix the original SevTech timeout, everyone please let me know if it also fixes your own crash.

[dkambersky]

Fwiw there are people reporting that 0.4.8 has fixed this issue for them - e.g. https://github.com/DarkPacks/SevTech-Ages/issues/4041

[ProsperCraft]

Yes, this is fixed for us on the TownCraft modpack in 0.4.8 thank you!