cannot get init_mapblock() to work

[ghost]

here is my rewrite of the map preloading script which:

1. uses your new libminetest rewrite
2. includes landcover
3. tries to process the source images in 16x16 footprints and splits the resulting elevation values into 16 m sections so that setting nodes can be done by block

in particular I am stumped on the loading of a recently initialized map block. could map.MapInterface.init_mapblock() return the blockID or the MapBlock itself? instead I am trying to load the block immediately after init_mapblock with the same blockpos, please see line 115:

https://gist.github.com/bobombolo/1e430eff993b7029b1030582d725f164#file-preload-py

2016-04-05 11:14:58,453 map:__init__                 :35   DEBUG    MapBlock obj
ect initiated at (176, 0, 0)
2016-04-05 11:14:58,453 map:load                     :480  DEBUG    Mapblock 176
 not found
Traceback (most recent call last):
  File "preload.py", line 120, in 
    map_block.set_node(node["pos"], cover[node["lclu"]])
AttributeError: 'NoneType' object has no attribute 'set_node'

anyway this script is close to working...

now that the script handles the nodes block by block, do you see any way to speed things up? maybe by calling implode() directly or somehow avoiding repeated calls to methods that convert positions and/or test to see if we are in the right mapblock (since we are for sure)?

[Lymkwi]

libminetest.errors.MapError: database is locked

This just means another program is working on your database.

I have some confusion regarding map vessels vs map interfaces, different ways of writing a position (Pos class, {x,y,z}, int, blockID, internal nodeID from 0-4096, etc)

Yes I have been confused as well for a while. Poses work a different way now, and I'll write documentation to detail the needed types for parameters. Basically, a position is relative to your container. The same Pos object can refer to two nodes depending on the container you'll use that function with. A Pos object (from libminetest.utils), which is now initialized like that :

pos = libminetest.utils.Pos(x, y, z)

That pos object is just a tuple of those 3 values, with some useful functions, like a convertion function to a unique integer in a container. To make this convertion, the functions needs a parameter which is called max_val and represents the maximum (unreachable) coordinate value.

AttributeError: 'dict' object has no attribute 'x'

That function needed a Pos object, so if you give it a dict, it won't find the appropriate fields.

AttributeError: 'NoneType' object has no attribute 'set_node'

I like that one; it means you tried to use a none object's set_node field, which doesn't exist : MapVessel.load gave you a None object because it didn't find your mapblock.

[ghost]

actually I have not solved the init_mapblock issue, the reason it started working is because I entered the map and explored around, initializing the area of interest, so that when I ran the script again set_node just worked because nothing needed to be initialized. can you look at my script and see why the init_mapblock doesn't work? I moved it from a gist to a repo:

https://github.com/bobombolo/preload/blob/master/preload.py#L146-L161

you can see i have commented out some alternate approaches (initializing mapblocks that I know are not initialized in an empty map.sqlite file, trying to use mapBlock.load() and map.Block.write() incorrectly...

also, is there a way to use MapBlock() to init and create a MapBlock instance instead of MapInterface.init_mapblock()?

[Lymkwi]

also, is there a way to use MapBlock() to init and create a MapBlock instance instead of MapInterface.init_mapblock()?

You could create a mapblock in your code, insert it in the MapInterface's cache, and its id in the object's stackcaches, then call check_cache() to check whether the cache is overflowing or not ; but that's exactly what init_mapblock() is doing for you. If you plan on using a MapVessel directly though, you can now initialize your own mapblocks and write them manually with MapVessel.write(blockID, binary_blob).

I'm pretty sure the sole reason why your script didn't work was the stupid mistake I fixed in #8. In your code, you could initialize your own mapblock, write all the nodes you need in it, then write it using the MapVessel object of your MapInterface. That would probably be the most efficient was of inserting those nodes. I'm going to rework your script right now, and see if that method works (it should). EDIT: Actually, since you import a schematic, there is no way you can bypass the MapInterface..

[ghost]

well I can do that schematic import as a separate step, I'm curious to see how you would do what you are discussing about manually updating mapblocks.

[Lymkwi]

It would look like that (without schematics, comments, etc):

for mapblocky in blocks:
    mapblockpos = libminetest.utils.Pos(mapx, mapblocky, mapz)
    mbobj = libminetest.map.MapBlock(abspos = mapblockpos.getAsInt(max_val=4096))

    for node in blocks[mapblocky]:
        npos = libminetest.utils.Pos(node["pos"].x % 16, node["pos"].y % 16, node["pos"].z % 16)
        mbobj.set_node(npos.getAsInt(), cover[node["lclu"]])
        t = t+1

    #then write the block
    map_interface.container.write(mapblockpos.getAsInt(max_val=4096), mbobj.implode())
    #reporting
    print("[BLOCK: {block}] [ELAPSED: {elapsed:.2f}s] [READ: {read:.4f}%] [WROTE: {wrote:.4f}%]".format(
        block=str(mapblockpos),
        elapsed=(time()-s),
        read=((p/tpix)*100),
        wrote=((t/tpix)*100)
    ))

[ghost]

hey that fixed another issue with missing nodes. https://github.com/bobombolo/preload/issues/6 thanks.

[ghost]

I think there is still an issue somewhere, and since I haven't changed the logic of the script, I am wondering if there is still an issue with the lib. if I use the original method of map_interface set_node() I get some missing nodes, and if I use your code there is a shift of 15 of the 16 rows in every block being shifted 16 spaces to the north, which I didn't realise because I used a test map of 32x32 nodes all on y=0 so it looked at a glance like it was correct but it is shifted to the north. it is much more obvious on a real terrain, creating strange hoops which made it obvious. see https://github.com/bobombolo/preload/issues/11

(before your code snippet was implemented the defect of holes looked like: https://github.com/bobombolo/preload/issues/6

[Lymkwi]

The only mathematical explanation I have is the modulo screwing up everything... Which wouldn't make sense.. But I haven't seen any issues on my own scripts when loading images or large schematics..