This is an experiment to rewrite some of the core classes to fix a number of long-standing issues. This is also a chance to reevaluate the implementation having learnt quite a lot since I started writing this. Here are the issues that I plan to tackle

256 - chunk storage

257 - chunk storage

Changes in Amulet Core 2.0

Module structure changes

The amulet.api module is going to get removed and its contents moved into the root. amulet.api.block.Block > amulet.block.Block amulet.api.entity.Entity > amulet.entity.Entity

The level classes are getting rewritten and moved around amulet.level.formats.leveldb_world > amulet.level.bedrock.BedrockLevel amulet.level.formats.anvil_world > amulet.level.java.JavaLevel

amulet.libs.leveldb will be removed. This was already depreciated.

Level construction changes

The level classes are now constructed through class methods. #258 The default constructor now just defines minimal data.

Level class changes

The level class is rather cluttered. I want to compartmentalise methods to declutter it. This was prototyped in #261 and modified a bit since. level.get_dimension() returns a dimension object within which all data from a dimension can be acquired. dimension.get_chunk_handle returns a class containing methods to get, set, delete and do other things with a chunk.

Chunk class changes

In 1.0 we have one chunk class shared by all levels. The idea for this was that we would only need to write one operation which would work for all world types. I still like this idea but it has been a little limiting because not all data fits into the same format. 2.0 breaks the chunk class up into components (BlockComponent, EntityComponent, ...) and each level defines its own chunk class (or multiple) which is built from the common components and custom components. Code then needs to check if the chunk is an instance of the required component before making the changes.

Universal format

The universal format is a custom format I created that is designed to be a superset of all the formats. This format is subject to change so that we can support new data as it gets added. Currently only blocks are implemented but the plan is to extend this to items and entities. In 1.0 the blocks in the chunk are stored in this format and the idea is that plugin writers should use the translator to convert their desired format to the universal format however I have seen multiple cases of hard coding in the universal format. The universal format also obscures what the real data. After quite a bit of thought I have decided to switch to storing the chunk data in its native format. This means that a block in the chunk will now look like Block("minecraft", "stone") instead of Block("universal_minecraft", "stone")

The universal format will become a private implementation detail used to translate between different formats.

Data versioning

With the change to native data we need to know which platform and version the data is defined in. Java chunks only support data defined in one format but Bedrock blocks can be defined in any historical format. Any class that takes data which may change between platform or version must have a platform and version identifier. This includes Chunk, Block, Entity, ...

Data storage changes

Previously when we loaded data (eg chunks) there was one instance that was stored in a dictionary. Every “get” call returned the same object. We could only clear the dictionary when the changes were saved or the container was explicitly cleared by the app. This is fine in a single threaded environment where the app manages the state and calls an operation.

I would like to support multi-threading and multiple plugins that don’t have to be aware of each other. The new implementation will return a deep copy of the data for every “get” call. #260 The caller owns the data and once it loses the reference it will automatically get deleted. To modify the data the caller needs to change the data and call the equivalent “set” method.

Thread locking

Usually when writing a library it is down to the user of the library to manage synchronisation but in this case we may have multiple plugins each running their own threads that are not aware of each other. To solve this we have added locks associated with each piece of data which the caller must acquire if they wish to modify the data. #262

The code would look something like this.

level: BedrockLevel
with level.edit_parallel():  # If another thread has acquired the world for unique editing then this will block until it is finished
    dimension = level.get_dimension("minecraft:overworld")
    chunk_handle = dimension.get_chunk_handle(0, 0)

    with chunk_handle.edit() as chunk:  # If another thread has locked this chunk, this will block until it is finished.
        # edit the chunk
        # If the edit context manager exits without erroring it will automatically set the chunk

    # The above is equivalent to the following
    with chunk_handle.lock():
        chunk = chunk_handle.get()
        # edit the chunk
        chunk_handle.set(chunk)

    # Once level.edit_parallel() exits it will automatically create an undo point

History changes

The history system should be modified to support disabling of the undo point #255 I can't remember if I have done this or not.

Signals

Currently to tell if a piece of data has changed we need to periodically manually check. The chunk generator thread is constantly checking if the chunks have changed in order to know if it needs to rebuild them.

To improve this I have added a signal system that anything can subscribe to to get notified. #268 The chunk generator just needs to subscribe to the changed signal of all the chunks it is drawing and it will get automatically notified when they get changed.

Raw data editing

I would like to support editing/viewing the raw data as well as in the higher level representation. #259 To do so the code would have to uniquely lock the level so that no other threads can edit the level in the higher level representation. This would allow raw viewing/editing of the leveldb database and the Java region data.

Amulet-Team / Amulet-Core

Prototype V2 #270