Block and Item database.

[broese]

Moved from the main thread - I think it should be a separate issue.

Well, there is an idea for a bit of work - if it suits you. We will need a new module to load the block/item information from JSON data. This data can be exported from the server's jar file of the appropriate version using java -cp server.jar net.minecraft.data.Main --all That's a new function that was added with 1.13 release.

In generated/reports/ you then can find files blocks.json and items.json. We need to parse these to get IDs used in this particular protocol version. If you'd like, you could write a parser for these things using the json-c library that MCB already uses to parse the user profiles. Then some routines to lookup data by IDs and vv., so sort of a database that I could access to map IDs

Also some code to load/delete the databases. Since blocks database is huge, it also would make sense to maybe keep it in memory and only delete if a new protocol version was requested - so that the file does not need to be reparsed every time the user reconnects. My idea : define a "database_t" object that I can get from the load function and if that database was already once loaded, I just get the same copy again, without parsing the file again. Let's assume the json files are in database/ subdirectory and are identifable by their protocol ID, e.g. database/blocks_404.json and database/items_404.json

const database_t *load_database(int protocol_id);
unload_all_databases();

Then another issue: I'm not sure if this is legal to distribute JSON files from Mojang's data, and it would be too much to ask an average user to do it themselves. Maybe a better way would be an external tool to parse json and dump a database file in out own format - maybe even purely binary. This binary file is then distributed together with MCB - and is loaded quicker.

The data format for items could be pretty simple:

struct item_t {
  const char *name;
  int id;
};

/* example
"minecraft:obsidian": {
  "protocol_id": 139
}, => { "obsidian", 139 }  i.e. strip "minecraft:" part
*/

// lookup
int get_item_id(database_t *db, const char *name);
const char *get_item_name(database_t *db, int item_id);
get_item_id(db, "obsidian") => 139
get_item_name(db, 139) => "obsidian"

regarding the blocks, I need think of good way to organize them cont…

Originally posted by @broese in https://github.com/broese/mcbuild/issues/20#issuecomment-450432184

[broese]

I've started working on the build function. What I need is another function to map the material name to the block ID. At the moment we only have the name to item ID mapping and those IDs are different.

int get_block_id(database_t db, const char name) get_block_id(db, "cobblestone") => 14 get_block_id(db, "nether_brick_stairs") => 4540 // north,bottom,straight,false marked as default

I don't know yet how we should handle the blocks with different variants. For now I propose fetching the default ID, and if none was marked as default - the first one in the list (not sure if it's true for any block, they all have at least one default). This should work well at least for the simple blocks with just one state.

I would also like to check for that and for now prohibit using non-simple blocks as material. I need a function to check for the number of states:

int get_number_of_states(database_t *db, int block_id) get_number_of_states(db,5) => 1 // polished_diorite get_number_of_states(db,8) => 2 // grass_block

[harbaughfan]

Sounds great! Each block name does have a default block id so I'll return that in the first function.
I'm on it!

[harbaughfan]

get_block_id() and get_number_of_states() are now complete & in a pull request.

[broese]

1. "db" variable stays constant during the entire gameplay. Maybe throwing around a point was a bad idea after all. Could it be possible that db is simply set by load_database() internally in the database module - even as static variable, so it's not exposed to other modules at all. Then all the database functions are simply called without the database_t *db argument and within module they will have the access to it. This will simplify things IMO.
2. Maybe you could review the naming scheme for database functions. In other modules I usually start the function name with (abbreviated) module name, so I called the functions db_blk_is_empty() and db_blk_is_noscan().
3. I noticed that your module is using CR-LF newlines. Could you use LF and fix it in the module please, so it's consistent throughout the project? But please make a separate commit for that, otherwise other changed lines may become unrecognizable in the diffs.
4. I will slowly start moving/deleting stuff from mcp_ids.* to mcp_database so we can get rid of it, and also modify some stuff in mcp_types, so the code becomes more consistent about block types.

[harbaughfan]

All this sounds great! Thanks for this list, simple things like cr-lf I don't even think about, I'll just learn how to do it.
I created a good system to add any table-lookup (I_ITEM, I_NSTACK, I_ARMOR etc) that you want. I just have an excel spreadsheet with every item, then I can just add column for any property you want and it generates the C code that I can just copy paste into the module. So I'll upload the stacksizes & items & armor, then let me know what other properties to add! Should be tomorrow night. Then I will dive into the list above. Thank you!!

[harbaughfan]

Hi just an update on the pull request, I added the item flags array for stacksize and item-only (item without blockform). I have an excel spreadsheet that allows me to add any flag you want and convert to C-code.

db_item_is_itemonly() returns true if the item is an item only, I was just wondering does it clean things up for you if the truth statement is reversed? If so I could convert it to db_item_is_also_blk() which returns false if the item is an item-only. db_stacksize() is also added.

PS always feel free to change the names of my functions, I am choosing wordy function names to help with clarity in passing code from me to you.. but once you start working with them, if shorter names help always feel free to change & I can just reset my git clone to whatever you change to.

[harbaughfan]

Another update on the pull request, The active database is now tracked within the mcp_database.c module itself, (set by load_database) so external functions don't need to reference the database pointer.
So the pull request now satisfies #1 #2 and #3 in the todo list above.

[harbaughfan]

If you ever want to add a flag manually while you work, please do and I’ll backfill it into my excel spreadsheet so it’s preserved in the future. Also is it time for a block flags table? I’m at your service for anything you’d like to assign and I still have the bid_t conversion to do.

[broese]

I've started dissolving the mcp_ids module and moving to functions from the database module. THat fixed some of the inventory handling issues, now that the items can be properly looked up.

You could look at the function set_block_dots() in mcp_build. Here, depending on the block type (i.e. flags), I determine how to place them properly. On one hand, I would think we would need most of those, but on the other hand, this could be wasted effort if it turns out the placement has become more generic.

I think we would in any case need I_CONT - i.e. things that are some sort of containers and open a dialog when clicked.

[harbaughfan]

Added I_CONT to a pull request, and next time I'll do the rest of the flags in set_block_dots. If it turns out they're not needed no worries!

One thought I've been having, I feel like the item table is really item/block table, meaning we will not plan on a separate table for block flags yet, and the reason it works is because practically all the blocks also exist as items. If ever you start noticing it would be better to have two separate tables (one for blocks & one for items) we can just switch course.

Here is a list of every block without an associated item. Along with an example item that more or less represents it? Or append this list of blocks to the bottom of the items flags table?

block	associated item (example)
water	water_bucket
lava	lava_bucket
tall_seagrass	seagrass
piston_head	piston
moving_piston	piston
wall_torch	torch
fire	air
redstone_wire	redstone
wall_sign	sign
redstone_wall_torch	redstone_torch
nether_portal	air
attached_pumpkin_stem	pumpkin_seeds
attached_melon_stem	melon_seeds
pumpkin_stem	pumpkin_seeds
melon_stem	melon_seeds
end_portal	air
cocoa	cocoa_beans
tripwire	string
potted_oak_sapling	flower_pot
potted_spruce_sapling	flower_pot
potted_birch_sapling	flower_pot
potted_jungle_sapling	flower_pot
potted_acacia_sapling	flower_pot
potted_dark_oak_sapling	flower_pot
potted_fern	flower_pot
potted_dandelion	flower_pot
potted_poppy	flower_pot
potted_blue_orchid	flower_pot
potted_allium	flower_pot
potted_azure_bluet	flower_pot
potted_red_tulip	flower_pot
potted_orange_tulip	flower_pot
potted_white_tulip	flower_pot
potted_pink_tulip	flower_pot
potted_oxeye_daisy	flower_pot
potted_red_mushroom	flower_pot
potted_brown_mushroom	flower_pot
potted_dead_bush	flower_pot
potted_cactus	flower_pot
carrots	carrot
potatoes	potato
skeleton_wall_skull	skeleton_skull
wither_skeleton_wall_skull	wither_skeleton_skull
zombie_wall_head	zombie_head
player_wall_head	player_head
creeper_wall_head	creeper_head
dragon_wall_head	dragon_head
white_wall_banner	white_banner
orange_wall_banner	orange_banner
magenta_wall_banner	magenta_banner
light_blue_wall_banner	light_blue_banner
yellow_wall_banner	yellow_banner
lime_wall_banner	lime_banner
pink_wall_banner	pink_banner
gray_wall_banner	gray_banner
light_gray_wall_banner	light_gray_banner
cyan_wall_banner	cyan_banner
purple_wall_banner	purple_banner
blue_wall_banner	blue_banner
brown_wall_banner	brown_banner
green_wall_banner	green_banner
red_wall_banner	red_banner
black_wall_banner	black_banner
beetroots	beetroot
end_gateway	air
frosted_ice	ice
kelp_plant	kelp
dead_tube_coral_wall_fan	dead_tube_coral_fan
dead_brain_coral_wall_fan	dead_brain_coral_fan
dead_bubble_coral_wall_fan	dead_bubble_coral_fan
dead_fire_coral_wall_fan	dead_fire_coral_fan
dead_horn_coral_wall_fan	dead_horn_coral_fan
tube_coral_wall_fan	tube_coral_fan
brain_coral_wall_fan	brain_coral_fan
bubble_coral_wall_fan	bubble_coral_fan
fire_coral_wall_fan	fire_coral_fan
horn_coral_wall_fan	horn_coral_fan
void_air	air
cave_air	air
bubble_column	air

[broese]

The problem with the blocks mapping is that their IDs will probably shift with the time, while item IDs seem to stay in place. Alternatively, we could map string ID to the flags. It might be less efficient, but may become necessary if the item IDs become incompatible with the older versions in the future. Your table above might become useful once we get to finer stuff like planting etc. I have something similar in get_base_material()

[harbaughfan]

Sounds great. I made this table with every block-state property a block can have, along with relevance to mcbuild. Just something to inspect and maybe get ideas on what flags are/will be useful. https://gist.github.com/harbaughfan/a7b989613b24168aa4594afea289fe03

[harbaughfan]

Hi, I'm adding flags one-by-one today. Not sure if it's more proper to add the pull request right away then continue to add flags to the pull request, or wait until all flags are added before submitting a pull request. Does it make any difference?

Exciting progress after only 2 weeks! Bravo.

[harbaughfan]

Note for lever and buttons, they are placed identically, and there is upcoming grindstone which will be too. So I created I_FACE flag, which means the block has the face property giving which neighbor it is attached to. Torches banners signs and skulls are handled differently, where they have their own block name if they are on a wall, but are the same item in inventory. Can't figure out how to flag that.

[harbaughfan]

Note on I_RSDEV, I included observers which previously had its own handling due to different meta. But now it can be handled the same as the rest. They all have the same "facing" property of North, East, South, West, Down, and Up. Note hopper does not have an 'up' blockstate, it only has North, East, South, West, and Down.

[harbaughfan]

Note on I_CHEST and I_FURNACE, new for 1.13 is side-by-side single chests. So I felt it was better to keep the two flags separate.. noting ender_chest cannot double so is like a furnace. So I_CHEST is chests & trapped chests, while I_FURNACE is ender_chest and furnace. I_CHESTs have the property type: left / right / single which can be used in the future for placing chests accurately.

[harbaughfan]

Hi, I created db_get_rotated_block() which given any block id will return the id of the same block but rotated 90 180 or 270 degrees. Example block 1649 is oak_stairs facing north, and if you issue db_get_rotated_block(1649, 270) you get back 1689, which is the same oak_stairs except facing west. I was tinkering around and got slabs working, then stairs, but found stairs wouldnt rotate if the build placement was rotated. So then I saw the old way of rotating blocks with meta arithmetic, and understood we would need this function. I pushed db_get_rotated_block() to our pull request, but I am leaving all my other tinkering with set_dots on a side branch (called 'tangent') since it might be a completely different direction than you're going.

[broese]

That's great news! Meanwhile I've been trying to slowly break apart the mcp_ids module. I want to get rid of the legacy types and functions before I would proceed with the build functions update.

[harbaughfan]

That makes perfect sense as b.raw / .id .meta is no longer applicable. Question is the following function still useful? It's on my todo from a few weeks ago, but maybe things have changed and we have leaned toward another way to approach things.

//int get_matching_block_ids(database_t *db, const char *name, prop_t *match, int *ids); // places all ids matching a set of propeties for the block name into array ids (can be assumed to be long enough) and returns the number of ids

[broese]

I'm not quite sure at the moment. The intention here was to provide a replacement for the "metagroup" mappings. Mostly for the purposes of rotation. For example, if I place an oriented block with some base ID I expect for this orientation, but the server sends a SP_BlockChange me with a different ID that contains some other unrelated info (like "waterlogged"). How can I determine it's the correct placement, just irrelevant status? get_matching_block_ids() was meant to be very generic, and maybe it's an over-complicated way to deal with it.

Maybe instead we could have a function that determines a "default" value for a block ID, but not for all possible states like that in the original exported table, just for a subset of them matching a certain placement.

[harbaughfan]

OK thank you that helps a lot. It's just so complicated, if I am interpreting this right, a very simple version of what you want might be db_blocks_have_same_name_and_direction(server_block_id, expected_block_id) returning true if the server_block_id has the same name, direction, (axis, facing, half, hinge, type, rotation etc) as the block you tried to place. The more complicated version would be that you specify axis, facing, half, hinge etc. in the function call, and i return all block ids that match the criteria. I think I get it now... that can be a project for me this weekend.

[harbaughfan]

Hi, is it possible & good to pass a libhelper array from function to function? They would be nice to pass around since they keep track of the # of elements in the array & also manage memory.

[harbaughfan]

Hi, I've got a ton of time this weekend if there's anything I can do next. Refresh mapart colors, learn how entities work? If nothing particular, I can always work on improving code quality of mcp_database.c passing arrays properly, labeling parameters const if they're not changed, code clarity & efficiency, error handling, change the database file from ascii to binary, Setup a 19w server with new protocol and open wiki.vg and see what upcoming changes there are.