full CPU usage/hanging task - LightCleaner - 1.12.2 Mohist - Cubic Chunks

[Agrrox]

Prerequisites:

BkCommonLib version: BKCommonLib v1.16.5-v3 #1043 official release

LightCleaner version: 1.16.5-v1

Server version: Mohist 1.12.2 #223 (https://ci.codemc.io/job/MohistMC/job/Mohist-1.12.2/changes)

terraplusplus 1.1.0 #593 (https://jenkins.daporkchop.net/job/BuildTheEarth/job/terraplusplus/job/master/97/)

Cubicchunks version: cc 1188 #119 (https://jenkins.daporkchop.net/job/OpenCubicChunks/job/CubicChunks/122/)

CubicWorldGen version: cwg 145 #52 (https://jenkins.daporkchop.net/job/OpenCubicChunks/job/CubicWorldGen/)

---

Problem or bug:

After using cleanlight <radius>, while radius is for example 1, the operation hangs and the server starts to utilize full CPU usage until I abort the operation with cleanlight abort which sometimes takes up to almost two minutes.

Expected behavior:

The cleaning of such a small radius should be instant.

Steps to reproduce:

1. set up Mohist server 223 or later.

2. install cubicchunks, cubicworldgen and terraplusplus into mods folder (either versions mentioned or latest)

3. install Bkcommonlib and Lighcleaner plugins

4. change mohist settings in /mohist-config/entity.yml to skip-activation-range: true (in new versions this option was merged into /mohist-config/mohist.yml)

5. use planet earth world with Build The Earth preset: TerraPreGenerated.tar.gz

6. use these values in server.properties:

spawn-protection=0
query.port=25457
server-name=BTE world
generator-settings=
force-gamemode=false
allow-nether=false
gamemode=1
broadcast-console-to-ops=true
enable-query=true
player-idle-timeout=0
difficulty=0
spawn-monsters=false
op-permission-level=4
vertical-view-distance=-1
pvp=false
snooper-enabled=false
level-type=EarthCubic
hardcore=false
enable-command-block=true
max-players=100
network-compression-threshold=256
resource-pack-sha1=
max-world-size=29999984
server-port=25565
debug=false
server-ip=0.0.0.0
spawn-npcs=true
allow-flight=false
level-name=TerraPreGenerated
view-distance=10
server-id=unnamed
resource-pack=
spawn-animals=false
white-list=false
generate-structures=true
online-mode=true
max-build-height=10000
level-seed=
use-native-transport=true
prevent-proxy-connections=false
enable-rcon=false
motd=Planet Earth world with default BTE preset

server.properties:

---

#### Other relevant details:

mod list

Name : Minecraft
ModID : minecraft
Version : 1.12.2
---------
Name : Minecraft Coder Pack
ModID : mcp
Version : 9.42
---------
Name : Mohist
ModID : mohist
Version : 1.12.2-233
---------
Name : Forge Mod Loader
ModID : FML
Version : 8.0.99.99
---------
Name : Minecraft Forge
ModID : forge
Version : 14.23.5.2854
---------
Name : Cubic Chunks Coremod
ModID : cubicchunkscore
Version : 1.12.2-0.0.1185.0-SNAPSHOT
---------
Name : Aroma1997Core
ModID : aroma1997core
Version : 2.0.0.2.b167
---------
Name : AromaBackup
ModID : aromabackup
Version : 3.0.0.0.b135
---------
Name : AromaBackuprecovery
ModID : aromabackuprecovery
Version : 3.0.0.0.b135
---------
Name : Banner mod
ModID : bannermod
Version : 1.1.0
---------
Name : Cubic Chunks Pregenerator
ModID : ccpregen
Version : 0.0.21
---------
Name : CubicChunks
ModID : cubicchunks
Version : 1.12.2-0.0.1185.0-SNAPSHOT
---------
Name : CubicGen
ModID : cubicgen
Version : 1.12.2-0.0.145.0-SNAPSHOT
---------
Name : RealCoords
ModID : realcoords
Version : 0.0.6
---------
Name : TerraPlusPlus
ModID : terraplusplus
Version : 1.1.0.593-1.12.2
---------
Name : Terramap
ModID : terramap
Version : 1.0.0-beta7.8_1.12.2
---------

plugin list

``` Name : CoreProtect Version : 2.14.4 --------- Name : LuckPerms Version : 5.3.0 --------- Name : WorldEdit Version : 6.1.11-SNAPSHOT-BukkitCcCompat1;9629fff --------- Name : Sledgehammer Version : 0.4.2 --------- Name : BannerText Version : 1.12.2-1 --------- Name : PlaceholderAPI Version : 2.10.9 --------- Name : TabTPS Version : 1.3.8 --------- Name : WorldGuard Version : 6.2.2-SNAPSHOT;8eeab68 --------- Name : ProtocolLib Version : 4.5.1 --------- Name : DebugStick Version : 1.1 --------- Name : SchematicBrush Version : 0.5.2-20 --------- Name : BungeeGuard Version : 1.0-SNAPSHOT --------- Name : VoxelSniper Version : 5.171.0-CC_COMPAT-SNAPSHOT --------- Name : BetterBrushes Version : 1.0.0.140406 --------- Name : Vault Version : 1.6.7-b${env.TRAVIS_BUILD_NUMBER} --------- Name : RelativeCoordCalculator Version : 0.5 --------- Name : F3NPerm Version : 2.5 --------- Name : VentureChat Version : 2.22.8 --------- Name : Skript Version : 2.5.3 --------- Name : LastLoginAPI Version : 1.5.1 --------- Name : Essentials Version : 2.18.2.0 --------- Name : BKCommonLib Version : 1.16.5-v3 --------- Name : AnnouncerPlus Version : 1.2.5 --------- Name : AdvancedBan Version : 2.3.0 --------- Name : LightCleaner Version : 1.16.5-v1 --------- Name : ExtraContexts Version : 2.0-SNAPSHOT --------- Name : SkQuery Version : 4.1.5 --------- Name : HeadDatabase Version : 4.14.0 --------- Name : HolographicDisplays Version : 2.4.8 --------- Name : CommandSigns Version : 1.6.7 --------- Name : BannerMaker Version : 1.9.0 --------- Name : DeluxeMenus Version : 1.13.3-Release --------- Name : ItemJoin Version : 5.2.2-RELEASE-b734 --------- Name : EssentialsSpawn Version : 2.18.2.0 --------- Name : WorldGuardExtraFlags Version : 4.1.4 --------- Name : HolographicExtension Version : 1.10.9 --------- Name : TouchscreenHolograms Version : 1.4.2 --------- ```

timings report capturing the whole hanging task:

https://timings.aikar.co/?id=d89b7ed8a7c84db1bb0716e6dae0ddae ``` [03:31:45 INFO]: Enabled Timings & Reset [03:31:45 INFO]: Timings Reset [03:32:14 INFO]: Agrro issued server command: /cleanlight 1 [03:33:17 INFO]: Agrro issued server command: /cleanlight abort >timings paste [03:34:50 INFO]: Preparing Timings Report... [03:34:50 INFO]: View Timings Report: https://timings.aikar.co/?id=d89b7ed8a7c84db1bb0716e6dae0ddae ```

[Agrrox]

Seems like the world was loaded on a most recent cubic chunks version. Starting with CC 1188+, there are some changes to storage format api. Maybe this could be the reason why it's not working? https://github.com/OpenCubicChunks/CubicChunks/pull/646

[Agrrox]

Seems like the world was loaded on a most recent cubic chunks version. Starting with CC 1188+, there are some changes to storage format api. Maybe this could be the reason and could help to solve the issue? OpenCubicChunks/CubicChunks#646

more info about this from one of CC devs: RegionCubeIO#getSave() (which returns SaveCubeColumns) has been replaced with AsyncBatchingCubeIO#getStorage() (which returns ICubicStorage)

[DaMatrix]

dev of the new CC storage format API here: it looks like your current code is actually listing the files in the region directory. with the introduction of ICubicStorage, regions themselves are no longer guaranteed to exist in the first place. you should use ICubicStorage#forEachColumn(Consumer<ChunkPos>)/ICubicStorage#forEachCube(Consumer<CubePos>) to enumerate all the cubes/columns in the world.

[bergerkiller]

@DaMatrix thanks, when I get to it after this 1.17 stuff cools down I'll look into changing that. Those methods enumerate all of them right, not just the ones that are currently loaded?

[DaMatrix]

yup, it gets them all.

the reason enumerating regions is no longer sufficient is that with this custom storage format API in place, the storage format is no longer guaranteed to be region-based at all (for instance, i made one that uses rocksdb)

[bergerkiller]

I've added support and this appears to work fully now. https://github.com/bergerhealer/BKCommonLib/commit/9bc3f44211e0093ecd88bdcaad379e0f510abc72 https://github.com/bergerhealer/BKCommonLib/commit/7a674260d786bf41cdcb445b943833415f2671ce

I did find the api a little strange to use, like having to go deep into the core and cast ICubeIO to AsyncBatchingCubeIO to obtain the ICubicStorage instance. Is there a better way? A file search didn't reveal much.

[DaMatrix]

we have plans to rework the other code surrounding ICubicStorage, so for now that is unfortunately the only way to get an instance, yes.

also, i feel the need to mention that i really like your template system :D

[Agrrox]

Thank you! the problem was resolved.