midnightfreddie / McpeTool

get/put/delete cli and http API for Minecraft Bedrock Edition world directories
https://www.onlinetoolplanet.com/mcpe-bedrock-tool
MIT License
38 stars 6 forks source link

How do you translate in game coordinates to the dbkeys? #20

Open erichexter opened 6 years ago

erichexter commented 6 years ago

Sorry for asking a dumb question, but do you have a sample, if I know two coordinates that create a cube, how do I find the keys which contain those coordinates?

midnightfreddie commented 6 years ago

Hi, good question! And I had an answer mostly typed up, but my resulting example keys don't exist, so I need to figure out where I mathed wrong. It should be that chunks are 16x16 XxZ, and subchunks are 16 high. So (X % 16), (Z % 16), 2f, (Y % 16) (X / 16), (Z / 16), 2f, (Y / 16) with little endian byte order should be the key for a given coordinate's block data subchunk for overworld blocks. So 0,0,0 would be in key 00000000000000002f00 and 16,16,16 would be in 00000001000000012f01 01000000010000002f01, and -1, -1, 32 would be in FFFFFFFFFFFFFFFF2f02 .

I'll go figure out where I went wrong and come up with some working examples and code.

ref: https://minecraft.gamepedia.com/Bedrock_Edition_level_format

Edit: D'oh! I modulus'ed when I should have integer-divided. The modulus is how you figure out the in-subchunk offest. I'll work on some example code.

Edit 2: Oh, also, the representation is little endian, so the least significant bytes go first! Two hex digits is one byte.

midnightfreddie commented 6 years ago

Okay, I figured out my math issues. All division below is of course integer division. The remainder/modulus will be used to find the byte offset within the subchunk data. Also, X, Z, and dimension are 32-bit signed integers in little endian byte order. The true key is a byte array, but we represent it with a hex string for convenience. In the examples below, I've bolded the chunk Z coordinate for clarity.

Each chunk is 16x16x256 (X,Z,Y), and the subchunk block data keys are 16 high. So for x, z, y coordinates of 413, 54, 105:

So all keys beginning with 1900000003000000 are about this coordinate's chunk. (In the overworld; other dimensions add a 32-bit dimension ID, so the same coordinates in the Nether I think have keys that start with 1900000003000000FFFFFFFF and 190000000300000001000000 for the End.)

The tags and subchunk indexes are 8-bit values. (Unsigned? Not sure it matters as there are no negative Y chunk coordinates and no tags <0 or > 127.)

47 ([0x2F]) is the subchunk prefix tag, so all keys beginning with 19000000030000002f are the Y subchunks for this coordinate.

So, the subchunk key for X=413, Z=54, Y=105 is 19000000030000002f06

ref: https://minecraft.gamepedia.com/Bedrock_Edition_level_format

midnightfreddie commented 6 years ago

I saw you have some C# repos, so I made a C# example (embedded in PowerShell).

I then saw you have plenty of JavaScript, too, and I've been meaning to do this in JavaScript, anyway. I may even have an equivalent function deep in a dev branch somewhere already. If I find or make one I'll edit a link in here.

erichexter commented 6 years ago

I was thinking I can grab the coordinates out of this plugin and make a call to your api to implement the schematic export. https://github.com/The-HeX/minecraft-worldedit-bedrock

midnightfreddie commented 6 years ago

Cool, yeah! That's the kind of thing I hoped this could be used for. Do note that the world can't be open by Minecraft (or any other tool) when using McpeTool because leveldb only allows a single process access at a time. If you want/need to pass coordinates from in-game, I've long had the idea that a sign could be placed in-world, and McpeTool could find a sign with particular text and take action based on its coordinates.

What's funny is that I really haven't been much into modding Minecraft, so I'm not that familiar with the other tools. When I started this it was pre-Bedrock, and there weren't really any tools that could do anything with the MCPE worlds.

The schematic imports/exports look like one of the things I had in mind: copy/paste bits of worlds around, even between worlds. Are there still mismatches in block types and properties between Bedrock and Anvil? Do the Bedrock schematic imports handle this already?

erichexter commented 6 years ago

The single process access to the db, is a bit of a problem. I can see using something like signs or tagged structure blocks to mark regions for export.

As far as the import/export. I wrote the import and I wrote a translator to translate between java and bedrock schematics. Its not perfect, A few of the directional items still face the wrong direction, but that will just take a little time to work through the mappings of the data value(s). I have been able to get MCEdit Unified to load a bedrock world and export a schematic. I have been unsuccessful importing a schematic then saving a bedrock world in that tool, it always crashes on the save( currently running it from source master branch).

Olen commented 4 years ago

I know this is a really old thread, but jut want to thank you for your work. My kid started playing a while ago, and after building a "perfect" world in single player mode on his tablet, we set up a private server so more of us could play toghether, even while not on the same network, and he really, really, REALLY wanted to keep on building in the same world on that server, This tool allowed me to ensure that I could copy the world from android to the server, while still allowing him to keep all his assets, including ownership of tamed animals. I had to fetch the entities from various coordinates and modify the owner-tag, as the UniqueID of his user apparently changed when moving from single player to a player on the server.

In the process I used your PowerShell code to create python tool to convert coordinates to keys, and while doing that I discovered a small issue with your code. If the X or Z is negative, but small (< 16) your code will convert it to e.g. x = -13 -> ChunkX = 0 -> key 00000000. But while trying to get data fram the LevelDB I discoverd that this really should be key FFFFFFFF. So it makes me wonder if your convesion of all negative values are off by one, or if this ONLY happens for values close to 0? Eg. should your "-413" really be E6FFFFFF, or have you verified that E7FFFFFF is actually correct?

I currently have a simple "if x < 0; x = x - 16" (and the same for z) but have not tested it for larger negative values of x or z to validate the theory.

midnightfreddie commented 4 years ago

Thanks! I'm glad it was helpful. At least one other person used this for keeping a player whole when moving from server to local or vice-versa.

I went through a phase of doing two's compliment wrong, and that's probably what happened in this code. I'll take a look at it.

midnightfreddie commented 4 years ago

Oh, it's not a two's compliment problem as I'm relying on the native signed dword int to handle that.

Okay, I just confused myself a lot working through this, but I mostly see the problem. My C# isn't doing integer division like I expected; it's discarding the fraction which only works for positive results. Oddly enough, Python integer division is working as math expects, so I'm not sure why your code is having trouble...or maybe you just noticed it's off from mine...mine's wrong, and not just by 1...more like 1 and 1/16.

Edit: Have to pull something like this in C# instead: int ChunkX = (int)Math.Floor(x / 16.0);. I'll update the example.

Olen commented 4 years ago

Hi, and thanks for looking at it so long after. I have so far managed to fix what I needed to fix, but it is still good to make it right.

I discovered the issue when I created python code. Python also discards fractions, so e.g.

>>> int(-0.1)
0

Or

>>> int(-13/16)
0

So my code worked like yours. But then I could not find the items I was expecting after calculating the key for negative coordinates, but I found them buy manually searching through all the neighbor-coordinates. I was just not sure if this was JUST related to negative values in the first blockchunk or not. (E.g. if fffffffff goes from 0-(-15) or from (-1 to -16). But I think I have resolved all my issues now.

midnightfreddie commented 4 years ago

Huh, weird. I just pulled up python on my Ubuntu-on-Windows and got

Python 2.7.17 (default, Nov  7 2019, 10:07:09)
[GCC 7.4.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> int(-13/16)
-1

But I just noticed that's python2...here's python3 on the same system:

Python 3.6.9 (default, Nov  7 2019, 10:44:02)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> int(-13/16)
0

Ok, this seems to work the same (and correctly for our purposes) in both:

>>> import math
>>> int(math.floor(-13/16.0))
-1

Also , I think modulo is working properly. 0xFFFFFFFF offset 0 should be -16 and offset 15 should be -1:

>>> -16 % 16
0
>>> -1 % 16
15

But after this, we'll have to check and be sure that's right for the game coordinates within the chunks.

Olen commented 4 years ago

Yeah. I am using python3, so that explains the diff. Added math.floor now, and it works as expected, and i do believe that this will give the correct coordinates after trying to fetch entities from different locations in the game.

midnightfreddie commented 4 years ago

Awesome. I'm pretty confident in the chunk key being right using math.floor (in any language), but I have a lot to figure out about block encoding as Bedrock changed that quite radically in 1.2.13.

danhale-git commented 3 years ago

Here it is in Go:

const (
    chunkSize = 16
)

// SubChunkKey builds the levelDB key for the sub chunk at the given x/y/z coordinates.
//
// https://minecraft.fandom.com/wiki/Bedrock_Edition_level_format#NBT_Structure
func SubChunkKey(x, z, dimension int32, y int) ([]byte, error) {
    x = int32(math.Floor(float64(x) / chunkSize))
    z = int32(math.Floor(float64(z) / chunkSize))
    y = int(math.Floor(float64(y) / chunkSize))

    key := make([]byte, 0)

    key = append(key, littleEndianBytes(x)...)
    key = append(key, littleEndianBytes(z)...)

    if dimension != 0 {
        key = append(key, littleEndianBytes(dimension)...)
    }

    key = append(key, []byte{47}...) // 47 is the SubChunkPrefix key type tag
    key = append(key, byte(y))

    return key, nil
}

func littleEndianBytes(i int32) []byte {
    b := make([]byte, 4)
    binary.LittleEndian.PutUint32(b, uint32(i))
    return b
}

// Tests

func TestSubChunkKey(t *testing.T) {
    testSubChunkKey(0, 0, 0, "00000000000000002F00", t)
    testSubChunkKey(16, 16, 16, "01000000010000002F01", t)
    testSubChunkKey(-1, 32, -1, "FFFFFFFFFFFFFFFF2F02", t)
}

func testSubChunkKey(x, y, z int, want string, t *testing.T) {
    b, err := SubChunkKey(int32(x), int32(z), 0, y)
    if err != nil {
        t.Errorf("unexpected error returned: %s", err)
    }

    got := strings.ToUpper(hex.EncodeToString(b))

    if want != got {
        t.Errorf("unexpected key '%s': expected '%s'", got, want)
    }
}