Extracting multiple meshes from a single vtk file

[Alexxkrehmen]

Hi,

I have a big vtk file (6.3 GB) which takes ages to parse and I would like to automate the extraction of multiple meshes from it.

The file is containing voxels with ids of different cellular bodies. In total I have 53 items to extracts from this file. Some of them have neighbouring voxels, so when extracting everything in one row, they are glued together in the same mesh, and I have to manually seperate them and fill the gaps, which is quite tedious.

I've found a way to extract each mesh one by one, using this setup :

In the vtkThreshold node I use the int id - 0.5 and id + 0.5 values to isolate the item I want to extract (so here, 12.5 and 13.5 to get the item with id 13)

This gives the expected result for this item... But it takes several minutes. So, unless a better (and faster) solution is available, I would like to automate this process to extract all the 53 meshes.

I've tried to write this little script :

import bpy
import time

for value in range(1,53):
    print(value)
    bpy.data.node_groups["NodeTree"].nodes["vtkThreshold"].m_Value1 = value - 0.5
    bpy.data.node_groups["NodeTree"].nodes["vtkThreshold"].m_Value2 = value + 0.5

    bpy.data.node_groups["NodeTree"].nodes["VTK To Blender Mesh"].m_Name = "mesh"+str(value)

    bpy.ops.node.bvtk_node_update(node_path="bpy.data.node_groups['NodeTree'].nodes['VTK To Blender Mesh']")

But it is not working as expected : the bvtk_node_update() function seems to be asynchronous, and so the "for" loop is continuing to run while the first mesh is still not generated.

What should be done here ? Thanks for your help !

[tkeskita]

I'm not sure what's wrong there (modality issue?), but one way to trigger update is via frame change, maybe try bpy.context.scene.frame_set(value)?

You could theoretically parallelize mesh generation by running several Blenders from a script generating and exporting one mesh each, but you probably would run out of memory with that data size.

[Alexxkrehmen]

I've tried the frame change... but this isn't helping much.

The loop is still fully completed before the first node processing is started :/

About the parallelization, I'll try anyway to run multiple blenders, I have plenty of ram to spend ;) But would it be possible to upgrade the vtkThreshold node with a multi threaded code ?

[Alexxkrehmen]

Still unfortunate :

I've tried python threading to call each pass with a delay, but it crashes blender : the bvtk nodes execution seems unhappy with threaded environement

I've also tried with coroutine, but in this case, I'm facing the same original problem : the nodes updates isn't executed until the my script is ended.

[tkeskita]

I meant no threading but generate x python scripts and run them via commands like blender -b file.blend -P script01.py, y scripts at a time.

[Alexxkrehmen]

Ok I see... but this isn't really practical for me. I've found an alternative by creating a modal operator which allows to recall my script at different time intervals. I'm still having an issue with it, but I'll send the script as soon as I'll have found a fix.

[Alexxkrehmen]

Still stuck with the automate script :/

Here my code is waiting for the previous mesh to be finished before starting a new one. The first mesh is properly created, but the next one is not. The bvtk_node_update function is actually called, but something is not working after that because nothing is created and my timer is waiting forever for the new mesh to be created.

Here is my current script, do you have any idea why bvtk_node_update is only working on the first iteration ?

import bpy
import time

start_value = 1
max_value = 52
value = start_value

def process_nodes():
    global value , start_value , max_value

    if value != start_value:
        n = "mesh"+str(value-1)
        if bpy.data.objects.get(n) is None:
            print("object "+n+" not completed")
            return False

    #save file before doing anything, in case of crash
    #bpy.ops.wm.save_mainfile()

    # set the thresholds
    bpy.data.node_groups["NodeTree"].nodes["vtkThreshold"].m_Value1 = value - 0.5
    bpy.data.node_groups["NodeTree"].nodes["vtkThreshold"].m_Value2 = value + 0.5

    # set output name
    bpy.data.node_groups["NodeTree"].nodes["VTK To Blender Mesh"].m_Name = "mesh"+str(value)

    bpy.data.scenes["Scene"].frame_current = value

    bpy.ops.node.bvtk_node_update(node_path="bpy.data.node_groups['NodeTree'].nodes['VTK To Blender Mesh']")

    print("building object "+str(value))

    value = value + 1

class ModalTimerOperator(bpy.types.Operator):
    """Operator which runs its self from a timer"""
    bl_idname = "wm.modal_timer_operator"
    bl_label = "Modal Timer Operator"

    _timer = None

    def modal(self, context, event):
        if event.type in {'RIGHTMOUSE', 'ESC'}:
            self.cancel(context)
            print("Cancelled")
            return {'CANCELLED'}

        if event.type == 'TIMER':
            global value,max_value
            process_nodes()
            if value > max_value:
                return {'FINISHED'}

        return {'PASS_THROUGH'}

    def execute(self, context):
        global value , start_value , max_value
        print("start automate")
        wm = context.window_manager

        # repeat every 5 seconds
        self._timer = wm.event_timer_add(5, window=context.window)

        wm.modal_handler_add(self)
        return {'RUNNING_MODAL'}

    def cancel(self, context):
        wm = context.window_manager
        wm.event_timer_remove(self._timer)

def register():
    bpy.utils.register_class(ModalTimerOperator)

def unregister():
    bpy.utils.unregister_class(ModalTimerOperator)

if __name__ == "__main__":
    register()

    # test call
    bpy.ops.wm.modal_timer_operator()

[tkeskita]

No idea. What you have is similar to class BVTK_OT_AutoUpdateScan. What happens if you call bvtk_auto_update_scan instead of bvtk_node_update? This is just a blind test though. I haven't digged into this modal usage aspect..

[Alexxkrehmen]

Same issue :/ I've tried with this : bpy.ops.node.bvtk_auto_update_scan( node_name="VTK To Blender Mesh" , tree_name="NodeTree" )

it gives me the same result : the first mesh is properly extracted, and nothing in the second iteration.

To let you know what's happening, my logs are displaying this sequence :

start automate
building object 1
object mesh1 not completed
object mesh1 not completed
object mesh1 not completed
object mesh1 not completed
building object 2
object mesh2 not completed
object mesh2 not completed
object mesh2 not completed
object mesh2 not completed
object mesh2 not completed

With "building object 2" displayed, I'm sure that the function bvtk_node_update is at least called, but something is not working after that. And I get "object mesh2 not completed" looping forever. Also, the nodes are responsive as if the update action had never been called.

[Alexxkrehmen]

Hi,

I've found a workaround : by putting the update function call inside the check for mesh completion, I get it to work and it seems to be executed only once. I just had to add this line at the beginning of my function.

if value != start_value:
        n = "mesh"+str(value-1)
        if bpy.data.objects.get(n) is None:
            print("object "+n+" not completed")
            #call again the update function
            bpy.ops.node.bvtk_node_update(node_path="bpy.data.node_groups['NodeTree'].nodes['VTK To Blender Mesh']")
            return False

This is dirty... but it works... so...