Warn about world bounds

[schrum2]

Whenever the user tries to initialize the world in a way that will generate shapes outside of the bounds of the world, the program should give a clear error message and exit.

Also, as an added safety check, we can put code in the spawnBlocks method that verifies the shape will be in the world bounds before sending to the Python code, since that seems to make the server hang forever

[schrum2]

A 7x7x7 archive fits in the world when forced to be linear. A 10x10x10 does not.

[MuellMark]

A Minecraft world is 60 million blocks x 60 million blocks large (30 million positive coordinates in the X and Z coordinates. However, it isn't as tall. While you can teleport up to 4096 blocks up, you can only place up to 320 blocks in the air. https://www.lifewire.com/how-big-is-a-minecraft-world-5212822#:~:text=One%20block%20in%20Minecraft%20is,ll%20eventually%20reach%20impassible%20lava.

[schrum2]

Actually, I've empirically figured out that 255 is the maximum boundary which makes sense ... this is the maximum 8-bit number. I guess they didn't expand the boundary until later versions.

[schrum2]

It seems that the EvoCraft server gets overwhelmed if a single spawnBlocks command consists of too many blocks. We're not sure what the limit is. @MuellMark will try to figure it out empirically. Once we know that limit, we can break any large spawnBlocks requests into a sequence of smaller requests.

[MuellMark]

Trying to place blocks that are all the same in a line over and over again has no problem placing over 9000 blocks, I'm going to run different tests to see if that crashes the server

[MuellMark]

I just tried doing a test where it switched out the block type on every other iteration and it still had no problems getting to over 5000 blocks generated at a time. I have one more test, generating with random blocks, but after that I'm going to look into what is being sent to the python code

[schrum2]

Seems the problem might be with reading a large volume rather than spawning a large volume, but need to confirm.

[MuellMark]

I am currently running a test on how many blocks can be read in before the system crashes. its already at a 500x500x1 space. The program stops at String response = commRecv(); which is on line 624 in MinecraftClient (at least I believe)

[MuellMark]

This is the error message I got when trying to print run with ranges bigger than 9x9x10

[MuellMark]

From testing with WidthHeightDepth and no parallelism, I have found out that generating:

• 10x10x10 crashes the server
• 9x9x9 kicks the player off of the server, but keeps the server running. This forces the user to rejoin to see everything created
• 8x8x8 runs perfectly fine, not crashing
• 9x9x10 (in any order, two 9's and a 10) also kicks the player off of the server, but keeps the server running.
• 9x10x10 (in any order, one 9 and two 10's) crashes the server

[MuellMark]

Testing the bounds with fillCube, I was able to generate a diamond block structure 102x250x102 in size. This has a size of 2,601,000 blocks. However, when trying to run the lonerShapeTask with 10x10x10, it crashes the server, despite only 2744 blocks being generated with fillCube (and around 2500 for a 9x10x10, which also crashes the server)

[schrum2]

All of the work here seems to indicate that individual large requests do not cause problems, so the hypothesis now is that the sever is instead being overwhelmed by the large number of back-to-back requests, which perhaps also interacts with the size (lots of small requests are fine, but lots of big ones are not).

The only way I can think of to fix this would be adding artificial pauses to slow down the code, but I don't want to do that if we can avoid it. That would add complication and slow down. Since the code mostly works as long as we work with reasonably sized shapes/archives/etc, this issue will go on hold.

[schrum2]

Still not sure what the problem is here, but if you find time between working on everything else, it would still be good to solve this. One idea worth considering is to add artificial pauses between calls to the Python code so that the Java server does not get overwhelmed. I don't know if we want to actually include anything like this in the code, but try it to see if it allows code to run that would normally crash the server.

[MuellMark]

We have found that the issue has been that the fillCube command in clearSpaceForShapes, which is in MinecraftClient (at about line 680) is trying to clear way to big of a space. The current plan now is to break this fillCube command up so that it clears shapes in a loop, rather than all at once. In theory, it should be able to handle 2,601,000 blocks at once, as calculated earlier, but I'll make that area smaller, especially at first

[MuellMark]

This should be working, however, while testing I accidentally deleted the entire ground. 4x4x4 redstone vs piston and 10x10x10 widthDepthHeight have both worked however, which is exciting as they hadn't worked before. I'm going to double check these after fixing my world to make sure everything is working the way the should before closing

[MuellMark]

I've cleaned up the code and it seems to be running, however, there are instances where it is very slow. For example, initialization for a 10x10x10 widthHeightDepth archive takes 3 minutes to clear and put the first shapes in the archive. I have done some more tests and I have concluded that clearing fewer, larger areas is faster than clearing many smaller areas. This, however, comes with some draw backs. While we have fixed the issue of crashing the server, I think there is still a problem. It is still possible to overload the server, stopping the process that is being run, but not outright crashing the server. This is the error message that has been popping up

I'm investigating this now

[MuellMark]

I've been testing this on the LonerShapeTask with the redstone vs piston characterization on the 4x4x4 size. The code will run, initialize the archive by placing all of the fences and shapes generated at initialization (all of which took my machine like 10 minutes), and then it stops. Nothing is placed in the world, no new messages show up from the server window, and nothing new comes up in the eclipse console, however, it is still running, and I know this because the little red square to stop it is still there. Below I'll copy and paste all of the things I ran it with. As I am writing this, I got it to work for 3x4x3 shapes, which is smaller, but still bigger than what we could initially do. There are just a few more kinks to work out before this is all running smoothly.

public static void main(String[] args) { int seed = 12; try { MMNEAT.main(new String[] { "runNumber:" + seed, "randomSeed:" + seed, "trials:1", "mu:100", "maxGens:100000", "base:minecraft", "log:Minecraft-MAPElitesWHDSimple", "saveTo:MAPElitesWHDSimple", "minecraftContainsWholeMAPElitesArchive:true","forceLinearArchiveLayoutInMinecraft:false", "launchMinecraftServerFromJava:false", "displayDiagonally:true", "io:true", "netio:true", "interactWithMapElitesInWorld:true", //"io:false", "netio:false", "mating:true", "fs:false", //"startX:-10", "startY:5", "startZ:10", "minecraftBlockSet:edu.southwestern.tasks.evocraft.blocks.MachineBlockSet", //"minecraftTypeCountFitness:true", "minecraftDiversityBlockFitness:true", //"minecraftTypeTargetFitness:true", //"minecraftDesiredBlockCount:40", //"minecraftOccupiedCountFitness:true", //"minecraftEvolveOrientation:true", //"minecraftRedirectConfinedSnakes:true", //"minecraftStopConfinedSnakes:true", "mapElitesBinLabels:edu.southwestern.tasks.evocraft.characterizations.MinecraftMAPElitesRedstoneVSPistonBinLabels", "ea:edu.southwestern.evolution.mapelites.MAPElites", "experiment:edu.southwestern.experiment.evolution.SteadyStateExperiment", "steadyStateIndividualsPerGeneration:100", //"extraSpaceBetweenMinecraftShapes:0", "spaceBetweenMinecraftShapes:5","parallelMAPElitesInitialize:false", "minecraftXRange:4","minecraftYRange:4","minecraftZRange:4", "minecraftShapeGenerator:edu.southwestern.tasks.evocraft.shapegeneration.ThreeDimensionalVolumeGenerator", "task:edu.southwestern.tasks.evocraft.MinecraftLonerShapeTask", "allowMultipleFunctions:true", "ftype:0", "watch:false", "netChangeActivationRate:0.3", "cleanFrequency:-1", "recurrency:false", "saveAllChampions:true", "cleanOldNetworks:false", "includeFullSigmoidFunction:true", "includeFullGaussFunction:true", "includeCosineFunction:true", "includeGaussFunction:false", "includeIdFunction:true", "includeTriangleWaveFunction:false", "includeSquareWaveFunction:false", "includeFullSawtoothFunction:false", "includeSigmoidFunction:false", "includeAbsValFunction:false", "includeSawtoothFunction:false"}); } catch (FileNotFoundException | NoSuchMethodException e) { e.printStackTrace(); } }

[MuellMark]

I tried running the 4x4x4 again and it worked, but I still need to investigate why it isn't generating reliably

[schrum2]

Verify that this is a consistent problem, and that there wasn't simply something weird with your machine during that one test.

If needed, we can impose a restriction on the frequency of server commands

[schrum2]

I ran the code you committed, and it crashed on the readCube command, indicating an overwhelmed server, so definitely need to tweak the magnitude or frequency of commands being made.

[schrum2]

Once you get to a point where nothing else seems to work, try solving the problem with a BlockingQueue. Be sure to start with a clean slate where all other code is cully committed, since there is a chance all of this will be discarded if it doesn't work.

Create the BlockingQueue in the if statement of the getMinecraftClient() method in MinecraftClient, and fill it with numbers 1 up to some command line parameter (start it at 10 first, but I'm hoping we can tolerate a much larger number). Then, modify each of the three basic synchronized methods spawnBlocks, fillCube, and readCube (only the bottom-level version of each of these) so that they take an element from the queue at the start, and return it afterward. I hope that will slow things down.

[MuellMark]

Through testing, I have found that having a sleep timer of 200 milliseconds in between each clear is the absolute smallest it can be before it starts crashing, at least for an archive of 4x4x4 shapes with the RedstoneVSPiston binning scheme. With smaller shapes, I've had some success with 100 ms, but wit bigger shapes (which is what we're trying to make), this has made it crash. I am currently testing ways to make sure we clear the smallest amount of space possible, rather than trying to cut down on the sleep time

[MuellMark]

We've added in a warning when the server gets overwhelmed after the code crashes that indicates that the sleep timer my not be large enough. Through my testing, I've found that a sleep timer of 200 ms is reliable, and has not crashed the server, but anything smaller is not. If we choose to make larger shapes, we may need to increase the sleep timer, but this is also suggested if the server gets overwhelmed. There are other issues that may come up in the future that are related to this issue, as the server crashes happen due to complex issues that are difficult to track.

[schrum2]

Since the initial clear takes so long now, print a message at the start saying it will take a while, and then print periodic messages during the clearing reporting progress. Also, add a command line parameter that gives the option to skip the initial clear

[MuellMark]

I added in several print statements to track what the code is doing when to avoid confusion from the user. Also, I added a command line parameter that skips the initial clear, however, there is also a second, smaller clear for the initial shapes that are generated. When generating large shapes, fence placement can also take some time, so I added a print statement there.