[BUG]: Light intensity increases for each run

[mikkelmedm]

Using a combination of the bop_objects_physics_positioning and key_frames example I am trying to render multilple frames for each run. I want to have a new random light for each run, however it seems for each run, the light intensity increases, making the later runs way to bright - see images for run 1 and run 30, respectively:

Looping over each run, I create a light as in the bop example, like so:

# sample light color and strength from ceiling
  light_plane = bproc.object.create_primitive('PLANE', scale=[3, 3, 1], location=[0, 0, 10])
  light_plane.set_name('light_plane')
  light_plane_material = bproc.material.create('light_material')
  light_plane_material.make_emissive(emission_strength=np.random.uniform(3,6), replace=True,
                                  emission_color=np.random.uniform([0.5, 0.5, 0.5, 1.0], [1.0, 1.0, 1.0, 1.0]))    
  light_plane.replace_materials(light_plane_material)

  # sample point light on shell
  light_point = bproc.types.Light()
  light_point.set_energy(200)
  light_point.set_color(np.random.uniform([0.5, 0.5, 0.5], [1, 1, 1]))
  location = bproc.sampler.shell(center = [0, 0, 0], radius_min = 1, radius_max = 1.5,
                          elevation_min = 5, elevation_max = 89, uniform_elevation = True)
  light_point.set_location(location)

Hope you can help :)

[themasterlink]

Hey,

I can't see any part of the code which removes the old lights? You can just call light_point.delete() at the end of each loop.

https://github.com/DLR-RM/BlenderProc/blob/964ca3d6db33d769d4944e90c1376d373f647502/blenderproc/python/types/EntityUtility.py#L133-L135

Else you can just change the location in each run, which would be easier.

Best, Max

[mikkelmedm]

Hi @themasterlink, Thanks for this!

Wouldn't the following loop change location each run, or am I missing something?

for r in range(NUM_RUNS):
  # sample point light on shell
    light_point = bproc.types.Light()
    light_point.set_energy(200)
    light_point.set_color(np.random.uniform([0.5, 0.5, 0.5], [1, 1, 1]))
    location = bproc.sampler.shell(center = [0, 0, 0], radius_min = 1, radius_max = 1.5,
                            elevation_min = 5, elevation_max = 89, uniform_elevation = True)
    light_point.set_location(location)

[themasterlink]

Yes it does change the location every loop, but it also creates a new light every loop.

I would propose:

light_point = bproc.types.Light()
light_point.set_energy(200)
for r in range(NUM_RUNS):
    # sample point light on shell
    light_point.set_color(np.random.uniform([0.5, 0.5, 0.5], [1, 1, 1]))
    location = bproc.sampler.shell(center = [0, 0, 0], radius_min = 1, radius_max = 1.5,
                            elevation_min = 5, elevation_max = 89, uniform_elevation = True)
    light_point.set_location(location)

This only changes the location and does not create a new light in every loop.

Best, Max

[mikkelmedm]

Alright, yes I tried that already, but that leaves me with the following error

[themasterlink]

Can you please share the full code :)

[mikkelmedm]

import blenderproc as bproc
import argparse
import os
import numpy as np
from numpy.random import choice
import random
import bpy
import importlib
from mathutils import Vector
from blenderproc.python.types.MeshObjectUtility import MeshObject
import time

parser = argparse.ArgumentParser()
# Paths:
parser.add_argument('bop_parent_path', nargs='?', default="bop_dataset", help="Path to the bop datasets parent directory")
parser.add_argument('cc_textures_path', nargs='?', default="resources/cctextures", help="Path to downloaded cc textures")
parser.add_argument('output_dir', nargs='?', default="trackar_data/output", help="Path to where the final files will be saved ")

# Parameters: 
parser.add_argument('bound_box', nargs='?', default=False, help="True for BBox keypoints - False for unique keypoints")
parser.add_argument('stand_vs_lay_prob', nargs='?', default=[0.5, 0.5], help="Propability of standing object vs laying object, respectively")

parser.add_argument('scene', nargs='?', default="trackar_data/CokeCanv3.blend", help="Path to the scene.blend file")
parser.add_argument('object_name', nargs='?', default="Can", help="Name of object in .blend file with keypoints")

parser.add_argument('num_runs', nargs='?', default=30, help="Number of runs")
parser.add_argument('frames_per_run', nargs='?', default=2, help="Number of frames for each run")
parser.add_argument('out_size', nargs='?', default=(256, 512), help="Output width and output height")
parser.add_argument('cam_dist_min', nargs='?', default=0.25, help="Minimum radius of camera sampler shell")
parser.add_argument('cam_dist_max', nargs='?', default=0.7, help="Maximum radius of camera sampler shell")
parser.add_argument('distractor_objs_min', nargs='?', default=2, help="Minimum number of occluding/distractor objects to add to the scene")
parser.add_argument('distractor_objs_max', nargs='?', default=4, help="Maximum number of occluding/distractor objects to add to the scene")
args = parser.parse_args()

bproc.init()

# Set to True for faster outputs during debbuging
DEBUG = True

# If DEBUG==True it loads only a random sample of bop objects, otherwise loads all objects ONCE
# at start of script and randomly chooses n-size subset for each run to place in scene.
distractor_bop_objs = bproc.loader.load_bop(bop_dataset_path = os.path.join(args.bop_parent_path, 'hb'),
                            sys_paths = "bop_toolkit",
                            sample_objects=True if DEBUG else False,
                            num_of_objs_to_sample=2 if DEBUG else None,
                            mm2m = True)
distractor_bop_objs += bproc.loader.load_bop(bop_dataset_path = os.path.join(args.bop_parent_path, 'ycbv'),
                            sys_paths = "bop_toolkit",
                            sample_objects=True if DEBUG else False,
                            num_of_objs_to_sample=2 if DEBUG else None,
                            mm2m = True)

if DEBUG:
    bproc.renderer.set_samples(10)

else:
    bproc.renderer.set_light_bounces(  
                            diffuse_bounces = 200, 
                            glossy_bounces = 200,
                            max_bounces = 200,
                            transmission_bounces = 200, 
                            transparent_max_bounces = 200
                            )

# set segmap_class (category_id) and physics properties of distractor objects:
for j, obj in enumerate(distractor_bop_objs):
    obj.set_cp("category_id", 0)

# load the blender object(s) into the scene
load_blend_objs = bproc.loader.load_blend(args.scene)

# If we need to scale the object: 
scale = 0.1

# set segmap_class (category_id) and physics properties 
for obj in load_blend_objs:
    obj.set_cp("category_id", 1)
    if obj.get_name() == args.object_name:
        obj_of_interest = obj 
        obj.set_scale(Vector((scale, scale, scale)))

# Load cctextures:
cc_textures = bproc.loader.load_ccmaterials(args.cc_textures_path)

# Initialize light:
light_point = bproc.types.Light()
light_point.set_energy(200)

# Iteratively generate a new scene with random camera/light/object placements:        
for r in range(args.num_runs):

    bproc.utility.reset_keyframes()
    FOV = np.random.uniform(1.04, 1.277) # Range of subset of phones' FOV at GoSpooky
    bproc.camera.set_intrinsics_from_blender_params(FOV, args.out_size[0], args.out_size[1], lens_unit="FOV")

    # Sample a random size subset of distractor objects each run:
    distractor_bop_objs_sample = random.sample(distractor_bop_objs, random.randint(args.distractor_objs_min, args.distractor_objs_max))

    # Hide/unhide the different objects during renders
    for obj in distractor_bop_objs:
        obj.hide(True)

    for obj in distractor_bop_objs_sample:
        obj.hide(False)

    draw = choice([True, False], 1, p=args.stand_vs_lay_prob)
    SURFACE = draw.item(0)

    # create room
    room_planes = [bproc.object.create_primitive('PLANE', scale=[2, 2, 1]),
                bproc.object.create_primitive('PLANE', scale=[2, 2, 1], location=[0, -2, 2], rotation=[-1.570796, 0, 0]),
                bproc.object.create_primitive('PLANE', scale=[2, 2, 1], location=[0, 2, 2], rotation=[1.570796, 0, 0]),
                bproc.object.create_primitive('PLANE', scale=[2, 2, 1], location=[2, 0, 2], rotation=[0, -1.570796, 0]),
                bproc.object.create_primitive('PLANE', scale=[2, 2, 1], location=[-2, 0, 2], rotation=[0, 1.570796, 0])]

    if not SURFACE:   
        obj_of_interest.enable_rigidbody(True, friction = 100.0, linear_damping = 0.99, angular_damping = 0.99)

        for plane in room_planes:
            plane.enable_rigidbody(False, collision_shape='BOX', friction = 100.0, linear_damping = 0.99, angular_damping = 0.99)

        for obj in (distractor_bop_objs_sample):
                obj.enable_rigidbody(True, friction = 100.0, linear_damping = 0.99, angular_damping = 0.99)

    # sample light color and strength from ceiling
    light_plane = bproc.object.create_primitive('PLANE', scale=[3, 3, 1], location=[0, 0, 10])
    light_plane.set_name('light_plane')
    light_plane_material = bproc.material.create('light_material')
    light_plane_material.make_emissive(emission_strength=np.random.uniform(3,6), replace=True,
                                    emission_color=np.random.uniform([0.5, 0.5, 0.5, 1.0], [1.0, 1.0, 1.0, 1.0]))    
    light_plane.replace_materials(light_plane_material)

    # sample point light on shell

    light_point.set_color(np.random.uniform([0.5, 0.5, 0.5], [1, 1, 1]))
    location = bproc.sampler.shell(center = [0, 0, 0], radius_min = 1, radius_max = 1.5,
                            elevation_min = 5, elevation_max = 89, uniform_elevation = True)
    light_point.set_location(location)

    # sample CC Texture and assign to room planes
    random_cc_texture = np.random.choice(cc_textures)
    for idx, plane in enumerate(room_planes):
        plane.replace_materials(random_cc_texture)

    if SURFACE: # standing
        # Define a function that samples the initial pose of a given object above the ground 
        def sample_initial_pose(obj: bproc.types.MeshObject):
            obj.set_location(bproc.sampler.upper_region(objects_to_sample_on=room_planes[0:1],
                                                        min_height=1, max_height=4, face_sample_range=[0.4, 0.6]))
            obj.set_rotation_euler(np.random.uniform([0, 0, 0], [0, 0, np.pi * 2]))

        # Sample objects on the given surface
        placed_objects = bproc.object.sample_poses_on_surface(objects_to_sample=[obj_of_interest] + distractor_bop_objs_sample,
                                                surface=room_planes[0],
                                                sample_pose_func=sample_initial_pose,
                                                min_distance=0.01,
                                                max_distance=0.4)

        # BVH tree used for camera obstacle checks
        bvh_tree = bproc.object.create_bvh_tree_multi_objects(placed_objects)

    else: # laying
        # Define a function that samples 6-DoF poses
        def sample_pose_func(obj: bproc.types.MeshObject):
            min = np.random.uniform([-0.3, -0.3, 0.0], [-0.2, -0.2, 0.0])
            max = np.random.uniform([0.2, 0.2, 0.4], [0.3, 0.3, 0.6])
            obj.set_location(np.random.uniform(min, max))
            obj.set_rotation_euler(bproc.sampler.uniformSO3())

        # Sample object poses and check collisions 
        bproc.object.sample_poses(objects_to_sample = [obj_of_interest] + distractor_bop_objs_sample,
                                sample_pose_func = sample_pose_func, 
                                max_tries = 1000)

        # Physics Positioning
        bproc.object.simulate_physics_and_fix_final_poses(min_simulation_time=3,
                                                        max_simulation_time=10,
                                                        check_object_interval=1,
                                                        substeps_per_frame = 20,
                                                        solver_iters=25)
        # BVH tree used for camera obstacle checks
        bvh_tree = bproc.object.create_bvh_tree_multi_objects([obj_of_interest] + distractor_bop_objs)

    poses = 0
    while poses < args.frames_per_run:
        # Sample location
        location = bproc.sampler.shell(center = obj_of_interest.get_location(),
                                radius_min = args.cam_dist_min,
                                radius_max = args.cam_dist_max,
                                elevation_min = 5,
                                elevation_max = 45,
                                uniform_elevation = True)

        # Determine point of interest in scene
        poi = bproc.object.compute_poi([obj_of_interest])

        # Add variation to the poi (so camera is not always statically pointing at obj_of_interest)
        vari = [0,0,0]
        vari[random.randint(0, len(vari)-1)] = random.uniform(0, 0.1)
        poi += vari

        # Compute rotation based on vector going from location towards poi
        rotation_matrix = bproc.camera.rotation_from_forward_vec(poi - location, inplane_rot=np.random.uniform(-0.7854, 0.7854))
        # Add homog cam pose based on location an rotation
        cam2world_matrix = bproc.math.build_transformation_mat(location, rotation_matrix)

        # Check that obstacles are at least 0.2 meter away from the camera and make sure the view interesting enough
        if bproc.camera.perform_obstacle_in_view_check(cam2world_matrix, {"min": 0.2}, bvh_tree):
            # Persist camera pose
            bproc.camera.add_camera_pose(cam2world_matrix)
            poses += 1

    # render the whole pipeline
    data = bproc.renderer.render()
    # Render segmentation masks (per class and per instance)
    data.update(bproc.renderer.render_segmap(map_by=["class"]))
    # Write data in custom format -> "blenderproc/python/writer/CustomWriter.py"
    bproc.writer.write_custom(os.path.join(args.output_dir), data, args.object_name, args.bound_box)

[themasterlink]

This is related to #331.

Could you try remove this function:

https://github.com/DLR-RM/BlenderProc/blob/964ca3d6db33d769d4944e90c1376d373f647502/blenderproc/python/types/LightUtility.py#L29-L34

This should fix it.

Best, Max

[mikkelmedm]

That sure did work!

Thanks for your time and expertise :)

Kindly, Mikkel

[themasterlink]

No worries, that is a bug in BlenderProc, we will fix it in the next version.

Best, Max