[BUG]: Image Generation stopping each 42 iterations

[franferraz98]

Describe the bug I've set up a script that generates images on a loop. It loads some objects from .blend files, modifies them and renders one single image each time. Without any error logs or messages, the execution stops each 42 iterations. Nothing works wrong apparently, but if I try to set a number of iterations bigger than 42, it stops early.

General Information

1. Which BlenderProc version are you using? 2.2.0

2. On which operating system are you? Ubuntu 20.04

3. Have you checked the issue tracker to see if a similar issue has been opened? Yes, didn't find any.

4. Have you changed BlenderProc in any way besides the config file? If yes, are you sure that this change does not affect the problem you are having? No, I haven't changed it.

To Reproduce Steps to reproduce the behavior:

1. Provide the full command you used to run BlenderProc: blenderproc run main.py
2. Provide the full python file, you used:

It will most probably not be useful as I'm using some libraries, but:

import blenderproc as bproc
from idna import check_initial_combiner
from matplotlib.style import use
import bpy
import os
import random
import sys
import yaml

# SPECIAL IMPORTS FOR BLENDERPROC
# -------------------------------------------------------------------------------------------------

dir = os.path.dirname("src")
if not dir in sys.path:
    sys.path.append(dir)

import src

# this next part forces a reload in case you edit the source after you first start the blender 
# session
import imp
imp.reload(src)

# this is optional and allows you to call the functions without specifying the package name
from src import utils

# -------------------------------------------------------------------------------------------------

def main():

    # Access to the config file and get
    my_config = os.path.join('config_files', 'config.yml')
    config_file = open(my_config, 'r')
    config = yaml.safe_load(config_file)

    # Get params
    scene = config['scene_file']
    output = config['output_path']
    num_images = config['num_images']
    focus_names = config['focus']
    lattice_names = config['lattice']
    use_lattice = config['use_lattice']
    camera_name = config['camera']
    cam_width = config['cam_width']
    cam_height = config['cam_height']
    point_to_focus = config['point_to_focus']
    rotation = config['rotation']
    hdf5 = config['hdf5']
    coco = config['coco']
    clear_tmp = config['clear_tmp']
    clear_output = config['clear_output']
    jitter_x = config['jitter_x']
    jitter_y = config['jitter_y']
    jitter_z = config['jitter_z']

    base_distance = config['base_distance']
    min_distance = config['min_distance']
    max_distance = config['max_distance']

    data_split = config['data_split']
    train_percent = config['train_percent']
    val_percent = config['val_percent']
    test_percent = config['test_percent']

    # Clear tmp
    if clear_tmp:
        utils.clear_tmp()

    # Clear output
    if clear_output:
        utils.clear_output()

    # Initialize BlenderProc
    bproc.init()

    # load the objects into the scene
    loaded_objects = bproc.loader.load_blend(scene, obj_types= ['mesh', 'volume', 'lattice', 'empty', 'light', 'camera'])

    # load textures
    loaded_textures = bproc.loader.load_blend(scene, data_blocks='materials')

    # get camera
    camera = bproc.filter.one_by_attr(loaded_objects, "name", camera_name)

    # define the camera resolution
    bproc.camera.set_resolution(cam_width, cam_height)

    # get focus
    focuses = []
    for name in focus_names:
        focuses.append(bproc.filter.one_by_attr(loaded_objects, "name", name))

    lattices = []
    for lattice_name in lattice_names:
        lattices.append(bproc.filter.one_by_attr(loaded_objects, "name", lattice_name))

    # get coordinates
    position, euler_rotation = camera.get_location(), camera.get_rotation()
    matrix_world = bproc.math.build_transformation_mat(position, euler_rotation)

    # set camera
    if point_to_focus:
        poi = bproc.object.compute_poi([focuses[0]])
        angle = 0
        location = utils.rotate(position, angle, axis=(0, 0, 1))
        my_rotation_matrix = bproc.camera.rotation_from_forward_vec(poi - location)
        matrix_world = bproc.math.build_transformation_mat(location, my_rotation_matrix)
        bproc.camera.add_camera_pose(matrix_world)
    else:
        matrix_world = camera.get_local2world_mat()
        bproc.camera.add_camera_pose(matrix_world)

    # activate normal rendering
    bproc.renderer.enable_normals_output()
    bproc.renderer.enable_depth_output(activate_antialiasing=True)

    # get original mesh
    bpy.ops.object.select_all(action='DESELECT')
    if use_lattice: # Save lattices
        for name in lattice_names:
            bpy.data.objects[name].select_set(True)

    for name in focus_names:
        bpy.data.objects[name].select_set(True)
    bpy.ops.wm.save_as_mainfile(filepath=os.path.join('tmp', 'original.blend'))

    # Split data in train/val/test
    tran_thresh = int(train_percent*num_images) - 1
    val_thresh = int(train_percent*num_images + val_percent*num_images) - 1
    test_thresh = int(train_percent*num_images + val_percent*num_images + test_percent*num_images) - 1

    # Make some renders
    for iteration in range(num_images):

        print("ITERATION: ", iteration)

        # Spin around object
        if rotation and point_to_focus:
            poi = bproc.object.compute_poi([focuses[0]])
            angle = (iteration/num_images)*360
            location = utils.rotate(position, angle, axis=(0, 0, 1))
            my_rotation_matrix = bproc.camera.rotation_from_forward_vec(poi - location)
            matrix_world = bproc.math.build_transformation_mat(location, my_rotation_matrix)
            bproc.camera.add_camera_pose(matrix_world, frame=0)

        # restore original mesh
        loaded_retrieval = bproc.loader.load_blend(os.path.join('tmp', 'original.blend'), obj_types=["mesh", "lattice"])

        if use_lattice: # Get lattice
            # get originals
            l_originals = [] 
            originals = [] 
            for l_name, f_name in zip(lattice_names, focus_names):
                l_originals.append(bproc.filter.one_by_attr(loaded_retrieval, "name", l_name+".001"))
                originals.append(bproc.filter.one_by_attr(loaded_retrieval, "name", f_name+".001"))
            # Substitute mesh
            for lattice, original in zip(lattices, l_originals):
                mesh = utils.restore_lattice(lattice.blender_obj, original.blender_obj)
                lattice.blender_obj = mesh
            for focus, original in zip(focuses, originals):
                mesh = utils.restore_mesh(focus.blender_obj, original.blender_obj)
                focus.blender_obj = mesh
            # deselect all objects
            bpy.ops.object.select_all(action='DESELECT')
            # select the object
            for l_name, f_name in zip(lattice_names, focus_names):
                bpy.data.objects[l_name+".001"].select_set(True)
                bpy.data.objects[f_name+".001"].select_set(True)
                # delete all selected objects
                bpy.ops.object.delete()
        else:
            # get originals
            originals = [] 
            for name in focus_names:
                originals.append(bproc.filter.one_by_attr(loaded_retrieval, "name", name+".001"))
            # Substitute mesh
            for focus, original in zip(focuses, originals):
                mesh = utils.restore_mesh(focus.blender_obj, original.blender_obj)
                focus.blender_obj = mesh
            # deselect all objects
            bpy.ops.object.select_all(action='DESELECT')
            # select the object
            for name in focus_names:
                bpy.data.objects[name+".001"].select_set(True)
                # delete all selected objects
                bpy.ops.object.delete()

        flips = []
        flips.clear()
        if use_lattice:
            for lattice in lattices:
                # GENERATE OK/NOK
                flip = random.randint(0, 1)
                flips.append(flip)

                # JITTER LATTICE
                if flip:
                    mesh = utils.jitter_lattice_special(lattice.blender_obj, (jitter_x, jitter_y, jitter_z))
                    lattice.blender_obj = mesh
                    # Alter distance
                    mesh = utils.jitter_distance(lattice.blender_obj, min_distance, max_distance)
                    lattice.blender_obj = mesh
                else:
                    mesh = utils.jitter_lattice_special(lattice.blender_obj, (jitter_x*0.8, jitter_y*0.8, jitter_z*0.8))
                    lattice.blender_obj = mesh
                    # Alter distance
                    mesh = utils.jitter_distance(lattice.blender_obj, base_distance, base_distance)
                    lattice.blender_obj = mesh

        else:
            for focus in focuses:
                # GENERATE OK/NOK
                flip = random.randint(0, 1)
                flips.append(flip)

                # JITTER MESH
                if flip:
                    mesh = utils.jitter_mesh(focus.blender_obj, (jitter_x, jitter_y, jitter_z))
                    focus.blender_obj = mesh

        # Label objects for COCO
        for j, obj in enumerate(loaded_objects):
            if obj.get_attr("name") in focus_names:
                f_idx = focus_names.index(obj.get_attr("name"))
                if flips[f_idx]:    # NOK
                    obj.set_cp("category_id", j*2)
                else:   # OK
                    obj.set_cp("category_id", j*2+1)

        # render the whole pipeline
        data = bproc.renderer.render()

        # Get and edit seg data
        seg_data = bproc.renderer.render_segmap(map_by=["instance", "class", "name"])

        # give proper names
        for elem in seg_data["instance_attribute_maps"][0]:
            if elem['name'] in focus_names:
                f_idx = focus_names.index(elem['name'])
                if flips[f_idx]:    # NOK
                    elem['name'] = 'NOK'
                else:   # OK
                    elem['name'] = 'OK'

        # Write data to coco file
        if coco:
            if data_split:
                if iteration <= tran_thresh:
                    bproc.writer.write_coco_annotations(os.path.join(output, 'coco_data', 'train'),
                                        instance_segmaps=seg_data["instance_segmaps"],
                                        instance_attribute_maps=seg_data["instance_attribute_maps"],
                                        colors=data["colors"],
                                        color_file_format="JPEG")
                elif iteration > tran_thresh and iteration <= val_thresh and iteration <= test_thresh:
                    bproc.writer.write_coco_annotations(os.path.join(output, 'coco_data', 'validation'),
                                        instance_segmaps=seg_data["instance_segmaps"],
                                        instance_attribute_maps=seg_data["instance_attribute_maps"],
                                        colors=data["colors"],
                                        color_file_format="JPEG")
                elif iteration > tran_thresh and iteration > val_thresh and iteration <= test_thresh:
                    bproc.writer.write_coco_annotations(os.path.join(output, 'coco_data', 'test'),
                                        instance_segmaps=seg_data["instance_segmaps"],
                                        instance_attribute_maps=seg_data["instance_attribute_maps"],
                                        colors=data["colors"],
                                        color_file_format="JPEG")
            else:
                bproc.writer.write_coco_annotations(os.path.join(output, 'coco_data'),
                                        instance_segmaps=seg_data["instance_segmaps"],
                                        instance_attribute_maps=seg_data["instance_attribute_maps"],
                                        colors=data["colors"],
                                        color_file_format="JPEG")

        # write the data to a .hdf5 container
        if hdf5:
            bproc.writer.write_hdf5(output + "/hdf5", data)

if __name__ == '__main__':
    main()

3. Provide a link to all 3D models you used, if they are from one of the publicly available supported datasets, provide the name or path so that it is possible to reproduce the error.

Expected behavior The generation of 200 images, as stated in my config file.

Screenshots These are the last lines printed while executing.

Additional context None

[themasterlink]

Hey,

I can't say what exactly the error is by looking at your code, but what you do is highly dangerous and might lead to bugs.

BlenderProc is designed to be rerun multiple times to create a dataset, do not I repeat do not run it once to create all images.

So every loop of yours should be done in a separate BlenderProc run, this might make it a tiny bit slower (1-3%) depending on your object size, but avoids blender pitfalls like you experience currently. Blender is just not designed to render hundreds of images with ever changing scenes, that's why we recommend to rerun BlenderProc several times avoiding messing up the internal state machine blender is.

Best, Max

[franferraz98]

Hi!

Thanks for the quick response. So if I got you right, I should basically make an external (bash) script that invokes blenderproc run main.py serveral times instead of looping inside the main function?

In that case I'd be loading the whole scene each time I need to render an image. Isn't that super inefficient?

Thanks in any case, I'll try it later!

[themasterlink]

Hey,

Thanks for the quick response. So if I got you right, I should basically make an external (bash) script that invokes blenderproc run main.py serveral times instead of looping inside the main function?

Exactly, you can also use python for that ;) subprocess.run()

In that case I'd be loading the whole scene each time I need to render an image. Isn't that super inefficient?

I wouldn't do it for every frame, but the idea is that you load a scene render let's say 5-10 images in this one fixed scene. Restart BlenderProc and render another 5-10 frames and so on.

Best, Max

[franferraz98]

Okay, I'll try it.

[franferraz98]

Okay, it works. Thanks for your help, I'll close the issue now.

Still, I'm a bit confused about this. What could possibly change between generating 10 images or 50? Is this a Blender issue?

[cornerfarmer]

Hey @franferraz98,

good to know that it works now! We are also a bit confused why 50 iterations does not work. The only limitation should be the available resources (memory, disk space...).

Could you transform your code into a minimal example, which still produces the error? It would be a great help for us to be able to reproduce the behavior.

[franferraz98]

Sure, I'll do it when I get a bit of spare time. Today I'm a bit busy but I think I can have it in 2-3 days.

[MartinSmeyer]

@franferraz98 thanks, did you already find some time to look into it?

[franferraz98]

No sorry, I've been dealing with some other projects and haven't been able to tackle this. I think that next week I will find some time, I'll keep you updated.