Spawning, destroying, and saving instance data in a sqlite database stream

I have managed to get VoxelLODTerrain to create a sqlite database file, it also generates the scheme within the database, but the database is empty of rows. I have called the save_modified_blocks() function with save_generator_output set to true, but still no rows are added to the database. I really need to access the instancing data.

I am not entirely sure how to communicate what it is I need help with but here goes...

In the game I am building, bees pollinate and cross pollenate instanced plants to produce new plant instances. I am struggling to determine the best means of populating the plants onto the world (currently using the voxel instancer to spawn in generic plant that replaces its own mesh and texture on _ready() with that of a random plant). I need to be able to destroy a plant, plant a seed to grow a new plant, and save some custom data for each plant instance to recall the type of plant it randomly chose on _ready().

Grass wont be interactable, but the rocks will eventually be. I understand that the instancing api is not complete( or atleast I believe i read so somewhere in the docs). In the docs it mentions possible writing my own instancing implementation, but I am still struggling to wrap my head around the SDF terrain in relation to voxel data. (its hard to conceptualize it as anything other than a 3d grid of block cells) Is there some example or resource where I can see an implementation of how to:

get the voxel data at a specific cell
determine if/where the surface cuts through that cell
instance a plant at the surface point within the cell
determine if/where there is a plant within the cell
destroy a plant in a given cell if it exists
save any instance data altered in game, along with some bit of data representing the plant,
recall and repopulate the worlds plants when reloading the map.

Below is a screen shot of some of the plants. in the image you can see a fully grown mushroom and a small mushroom next to small orange tulip. These two small plants were not placed by generation but by the player, and thus will be gone when I restart the map.

When a plant is fully pollinated it will produce a seed item (the icon with the 4 white shapes on the ground in the image below ). that seed has a random chance of "taking root" and spawning in a new plant instance. Plants spawned in this way also get lost when reloading the map

I think my questions are these:

how do I get the instance data to save and show up in the sqlite db
how can I query whether a given location/cell/block has an instance
how can i add or remove instances at runtime and save them to the sqlite db

NOTE: I wrote this answer assuming you are using VoxelInstancer, but I realize you might not be using that, since you didn't mention it. The TLDR is, you can't save custom arbitrary stuff to the SQLite file, it is specialized for voxel/chunk data. It also won't automatically happen with separate nodes. You may have to do your own thing currently.

I have called the save_modified_blocks() function with save_generator_output set to true

These two things are unrelated. save_generator_output only saves voxel data to the stream when it gets generated in its thread, even before it reaches the terrain. But I think that does not include instances. Also save_modified_blocks does what it says: it only saves changes. If you generate terrain and instances, nothing will save unless stuff has been edited at runtime.

get the voxel data at a specific cell

You can do that by obtaining a VoxelTool from VoxelLODTerrain using get_voxel_tool(), and then using functions of VoxelTool such as get/set_voxel (see Editing the terrain). However this is not related to instances created with VoxelInstancer.

determine if/where the surface cuts through that cell

VoxelInstancer already works exclusively on meshes, so it can only spawn instances on top of the meshes representing the surface. So in a way, knowing where the surface is to generate instances isn't really needed.

Knowing where the surface is can be done with regular collision raycasts. In the absence of colliders, doing that with voxels is a bit more involved: VoxelTool has raycast too, which does a raycast using the SDF instead, but it might yield slightly different results because it is not based on geometry, and might perform differently too. Instead it checks multiple voxels until it finds a change of sign in SDF data.

instance a plant at the surface point within the cell

It is currently not possible to create arbitrary instances with VoxelInstancer. It may be added in the future. They are always either generated procedurally, or deleted. If you want to do this, maybe you have to use regular nodes instead.

determine if/where there is a plant within the cell

how can I query whether a given location/cell/block has an instance

That might also be up to you depending on what you use. Instances from VoxelInstancer have no exposed function to detect which ones are in an area, the only way is for them to have a collider maybe. If plants are centered on a specific voxel (instead of the mesh surface), you can store data in voxels by using set/get_voxel_metadata in VoxelTool, but that functionality was initially intented for blocky voxels where each voxel directly corresponds to a model in the world (like Minecraft), so might not exactly match what you are using.

destroy a plant in a given cell if it exists

Instances of VoxelInstancer can be destroyed if terrain is excavated below them (though I havent tested this in a while, it might be broken). Also if they have a collider or are using a scene instance with a collider (VoxelInstanceLibrarySceneItem), you can detect one with a raycast and do hit.collider.queue_free() on it. But this has no relation with a "cell", since VoxelInstancer does not store its data in a voxel grid, they are just point clouds stored in chunks.

save any instance data altered in game, along with some bit of data representing the plant,

VoxelInstancer is currently only meant for large amounts of decoration such as rocks and foliage, it does not save anything more than a compressed position per instance, and only when some instances are deleted in a chunk (and marked persistent in VoxelInstanceLibraryItem*). There is no way to directly attach arbitrary data to every one of them. You would have to make your own system for this. You could start with unmodified generated instances storing no state, and once any gets modified, its instance gets deleted and replaced with a custom node with all the state it needs? But it's just an idea, I haven't worked on stuff like this so far so the current API might not be enough (in particular the multimesh one, which is supposed to be the most important due to its rendering performance).

recall and repopulate the worlds plants when reloading the map.

how do I get the instance data to save and show up in the sqlite db

VoxelInstancer will reload saved instance chunks if they are persistent and any were removed from a chunk. It does not save anything if a chunk hasn't had instances removed or if items are not persistent. But again, it only saves positions, not arbitrary data.

It also won't be present in the DB straight away, only a short time after save_modified_blocks is called (it's not instant) or when the chunk goes out of range. If you are referring to custom nodes unrelated to VoxelInstancer, there is no API to save custom stuff in the DB, it is specialized for voxel/chunk data.

These two small plants were not placed by generation but by the player, and thus will be gone when I restart the map.

It's up to you to save custom nodes/scenes instanced separate from terrain. There is no system automatically detecting your nodes or saving them, you have to do that yourself.

Thanks, that sort of illuminates some of the problem. I am using voxel instancer

You could start with unmodified generated instances storing no state, and once any gets modified, its instance gets deleted and replaced with a custom node with all the state it needs? But it's just an idea<

This is exactly what I am currently doing, using VoxelInstancer to populate the generic plant onto the world and then the script attached to said generic plan scene instance randomizes its data.

These two things are unrelated. save_generator_output only saves voxel data to the stream when it gets generated in its thread, even before it reaches the terrain. But I think that does not include instances. Also save_modified_blocks does what it says: it only saves changes. If you generate terrain and instances, nothing will save unless stuff has been edited at runtime.<

I only mentioned these two together as opposite ends of testing whether I could trigger data to be written to the database. I was assuming that with save_generator_output(), it would save generated chunk data into the database and recall it from the database to regenerate chunks as needed "like minecraft" instead of running the generator on the fly. My thinking was that this would at least allow me to see some data populated into the database to confirm that it is receiving any data at all. I used save_modified_blocks() to also with the same goal in mind. (Get ANY data at all to show up in the database) to know that it is in fact storing some. I did modify the terrain before calling it, but I have not been able to see any data saved to the database in any form via any method, only just the scheme.

VoxelInstancer already works exclusively on meshes, so it can only spawn instances on top of the meshes representing the surface. So in a way, knowing where the surface is to generate instances isn't really needed. <

I mainly wanted to know if there was a faster more efficient route than raycasting, that could be used to obtain a position where I might place a plant using my own custom instancing system, so that it is on the surface and not floating. My attempts to write a system that keeps up with the LOD system and player movement while casting rays to determine the position to place the plants after the terrain has been altered was very inefficient and expensive. This was my goal with the database as well, to query to determine where the surface is rather than casting to find it.

VoxelInstancer is currently only meant for large amounts of decoration such as rocks and foliage, it does not save anything more than a compressed position per instance, and only when some instances are deleted in a chunk (and marked persistent in VoxelInstanceLibraryItem*). There is no way to directly attach arbitrary data to every one of them. You would have to make your own system for this. <

I have a system for determining the data for the plants and even a way to save them to their own table within a sqlite db I am using to store the rest of the games data. The trouble is that world regenerates instances that I have destroyed, if I destroy them without modifying the terrain, and if I modify the terrain without altering the surface height, the world just puts the instance right back what it was.

VoxelInstancer will reload saved instance chunks if they are persistent and any were removed from a chunk. It does not save anything if a chunk hasn't had instances removed or if items are not persistent. But again, it only saves positions, not arbitrary data. <

The trouble is that world regenerates instances that I have destroyed, if I destroy them without modifying the terrain. If I modify the terrain without altering the surface height, the world just puts the instance right back where it was.

It also won't be present in the DB straight away, only a short time after save_modified_blocks is called (it's not instant) or when the chunk goes out of range. If you are referring to custom nodes unrelated to VoxelInstancer, there is no API to save custom stuff in the DB, it is specialized for voxel/chunk data. <

I have not managed to see any data make it into the database in anyway.

I am prepared to move forward with improving my own system for plant instancing I just want to make sure that I am not overlapping functionality that already exists, or regenerating data that already exists. My current algorithm is as follows.

instance generic plants using voxel instancer.
on _ready() of each plant instance randomize plant specs and visuals
if player plants seed, or seed takes root at random position, do a shape cast to determine if there are any other plants too close thus preventing its placement, and do a raycast to determine the surface position at which to place the plant
as the player moves occlude plants that are a certain distance away. repeat steps 1 and 2 for each newly generated chunk
as plants become occluded or just before modifying terrain, load them into a buffer to write to plant data storage
when terrain is modified determine the area where the modification has occured and query the custom plant data storage to find any plants that fall within that area and determine if they should be moved or removed.

does this sound right? am I overlooking anything else?

I mainly wanted to know if there was a faster more efficient route than raycasting

If you want to plant seeds at specific locations, there is no other way. The instance generator iterates every triangle of the surface mesh (not just around a specific position) and decides to put an instance using random and noise, it doesn't really places instances relative to a specific position, and does not even check existing instances (which is why it can sometimes place two instances overlapping each other, though it is usually rare and can be worked around by tweaking the random algorithm).

My attempts to write a system that keeps up with the LOD system and player movement while casting rays to determine the position to place the plants after the terrain has been altered was very inefficient and expensive

If you do this way too much and too often, it may be too expensive. Also Godot physics isn't the fastest engine out there. Maybe you need to do that only at specific times and at different frames (also shape cast sounds very expensive if done many times on arbitrary meshes such as the terrain). I already have this problem of keeping up with LOD sometimes, but it's still an area that needs research to figure out. So far I've been working around that by carefully tuning parameters and choosing meshes that can sink a bit in the ground without looking too bad.

This was my goal with the database as well, to query to determine where the surface is rather than casting to find it.

The database does not contain this information, only a 3D grid of voxels, which you would have to further process in order to get a hint, and yet it would not tell you the right height because if you get LOD0 from that data, and LOD3 is currently rendered, your instances would look sunk anyways. It would have been even more expensive than raycasting.

The trouble is that world regenerates instances that I have destroyed

That could be a bug, assuming your items have persistent set to true. If any instance gets removed, that counts as a modification and so all instances in that chunk (that are managed by VoxelInstancer) must be saved either when modified chunks are saved or when the chunk goes out of range. Maybe if you can reproduce this in a minimal test project I could look into it (though I won't be coding much in the next two weeks).

Zylann / godot_voxel

Spawning, destroying, and saving instance data in a sqlite database stream #507