Generating new scene from model

[AIMads]

Trained an initial model on 50 epochs, and wanted to test it out, using the generate_scenes.py script. Getting the following error when running the code, please help!

Command: python3 generate_scenes.py ../config/bedrooms_config.yaml tester /tmp/threed_front.pkl demo/floor_plan_texture_images/ --weight_file models/OSL638YTV/model_00050

Output: Running code on cpu Applying no_filtering filtering Loaded 17129 3D-FUTURE models <class 'list'> Applying no_filtering filtering Loaded 162 scenes with 21 object types: Loading weight file from models/OSL638YTV/model_00050 0 / 10: Using the 80 floor plan of scene SecondBedroom-36408 Traceback (most recent call last): File "generate_scenes.py", line 276, in main(sys.argv[1:]) File "generate_scenes.py", line 211, in main renderables, trimesh_meshes = get_textured_objects( File "/home/mil/Desktop/esoft/ATISS/scene_synthesis/utils.py", line 24, in get_textured_objects furniture = objects_dataset.get_closest_furniture_to_box( AttributeError: 'list' object has no attribute 'get_closest_furniture_to_box'

[tommaoer]

Hi， I am running the code at the same time. The second argv is the "threed_future_model.pkl", you must generate the file by yourself. I think the author missed the details. But, I find the code on the file is released in https://github.com/nv-tlabs/ATISS/blob/1acb1d5d06f343554dfc6aaf2dbebe2d366f4e93/scene_synthesis/datasets/utils.py#L134. You can run it to generate the file to replace the second argv with 'threed_future_model.pkl'. Second, you can change the last line of code "threed_future_dataset.py" from 'return dataset' to 'return cls(dataset)'. After the two operations, the errors will be solved! Jie

[AIMads]

@tommaoer Thank you so much for the response and help! I Have tried to do both things now, made the following call of the function: parse_threed_future_models("/home/mil/Desktop/esoft/ATISS/scripts/3D-FRONT/","/home/mil/Desktop/esoft/ATISS/scripts/3D-FUTURE-texture/","/home/mil/Desktop/esoft/ATISS/scripts/3D-FUTURE-model/model_info.json")

and made the change to the threed_future_dataset.py.

but now i get a new error, im concerned it is because my creation of the 'threed_future_model.pkl' is wrong?

The trace back is the following: python generate_scenes.py ../config/bedrooms_config.yaml tester/ threed_future_model.pkl demo/floor_plan_texture_images/ --weight_file models/OSL638YTV/model_00050 Running code on cpu Applying no_filtering filtering Loaded 75795 3D-FUTURE models Applying no_filtering filtering Loaded 162 scenes with 21 object types: Loading weight file from models/OSL638YTV/model_00050 0 / 10: Using the 122 floor plan of scene SecondBedroom-6261 Traceback (most recent call last): File "generate_scenes.py", line 277, in main(sys.argv[1:]) File "generate_scenes.py", line 212, in main renderables, trimesh_meshes = get_textured_objects( File "/home/mil/Desktop/esoft/ATISS/scene_synthesis/utils.py", line 25, in get_textured_objects furniture = objects_dataset.get_closest_furniture_to_box(query_label, query_size) File "/home/mil/Desktop/esoft/ATISS/scene_synthesis/datasets/threed_future_dataset.py", line 41, in get_closest_furniture_to_box return sorted_mses[0] IndexError: list index out of range

[AIMads]

Or else im getting hit with this error:

Loading dataset 6812 / 6813 Running code on cpu Applying no_filtering filtering Loaded 75795 3D-FUTURE models Applying no_filtering filtering Loaded 162 scenes with 21 object types: Loading weight file from models/OSL638YTV/model_00050 0 / 10: Using the 6 floor plan of scene MasterBedroom-75552 Traceback (most recent call last): File "generate_scenes.py", line 277, in main(sys.argv[1:]) File "generate_scenes.py", line 212, in main renderables, trimesh_meshes = get_textured_objects( File "/home/mil/Desktop/esoft/ATISS/scene_synthesis/utils.py", line 28, in get_textured_objects raw_mesh = TexturedMesh.from_file(furniture.raw_model_path) AttributeError: 'list' object has no attribute 'raw_model_path'

[tommaoer]

hi, the objects are retrieved from the same semantic category and the size of the bounding box. The above error is led by the 'oi.label' comes from the raw data, the semantic label is different with predefined in code. in this file scene_synthesis/datasets/base.py, it defines the mapping from the raw semantic label of raw data to the new labels. You can modify the 'oi.label' when you load the 'threed_future_dataset.pkl' by using the above mapping.

For the second error, you should change the last line of code "threed_future_dataset.py" from 'return dataset' to 'return cls(dataset)', which changes the type from list to a class.

[AIMads]

Thanks again for the response, is this mapping supposed to be performed in scene_synthesis/datasets/threed_future_dataset.py in the _filter_objects_by_label function? That atleast is where i see currently the label being identified? And how is it possible to make the mapping, code wise I have a hard time seeing how to do it!

I have made the change with cls(dataset) and solved one of the problems, thank you so much for helping!

[AIMads]

I made following changes: In scene_synthesis/datasets/threed_future_dataset.py return [oi for oi in self.objects if self.THREED_FRONT_BEDROOM_FURNITURE[oi.label] == label]

where THREED_FRONT_BEDROOM_FURNITURE contains the dictionary from base.py

But get this error based on it: Running code on cpu Applying no_filtering filtering Loaded 75795 3D-FUTURE models Applying no_filtering filtering Loaded 162 scenes with 21 object types: Loading weight file from models/OSL638YTV/model_00050 0 / 10: Using the 64 floor plan of scene SecondBedroom-19232

/home/mil/Desktop/esoft/ATISS/scene_synthesis/datasets/threed_front_scene.py(258)raw_model() -> print("Loading model failed", flush=True) (Pdb)

[AIMads]

I made following changes: In scene_synthesis/datasets/threed_future_dataset.py return [oi for oi in self.objects if self.THREED_FRONT_BEDROOM_FURNITURE[oi.label] == label]

where THREED_FRONT_BEDROOM_FURNITURE contains the dictionary from base.py

But get this error based on it: Running code on cpu Applying no_filtering filtering Loaded 75795 3D-FUTURE models Applying no_filtering filtering Loaded 162 scenes with 21 object types: Loading weight file from models/OSL638YTV/model_00050 0 / 10: Using the 64 floor plan of scene SecondBedroom-19232

/home/mil/Desktop/esoft/ATISS/scene_synthesis/datasets/threed_front_scene.py(258)raw_model() -> print("Loading model failed", flush=True) (Pdb)

It seems the error comes from the get_textured_objects function in scene_synthesis/utils.py

[tommaoer]

Very nice, I meet the same problem and I change the code to solve it with the following code:

def load_obj(fn):
    fin = open(fn, 'r')
    lines = [line.rstrip() for line in fin]
    fin.close()
    vertices = []; faces = [];
    for line in lines:
        if line.startswith('v '):
            vertices.append(np.float32(line.split()[1:4]))
        elif line.startswith('f '):
            faces.append(np.int32([item.split('/')[0] for item in line.split()[1:4]]))

    f = np.vstack(faces) - 1
    v = np.vstack(vertices)
    return v, f

v, f = load_obj(self.raw_model_path)
row_id = np.array(list(range(len(f))))[np.sum((f >= len(v)).astype('int'), axis=1)>0]
f = np.delete(f, row_id.tolist(), axis=0)
return trimesh.Trimesh(v, f)

this will be worked. I think this is because the 3D future model are with the error format of objects that make the trimesh load fail.

You can try the above code, good luck!

[AIMads]

Interesting thank you, where do you place this code?

Kind regards.

[tommaoer]

you can replace the original code with the above code. for example

[AIMads]

IT WORKS OMG THANK YOU SO MUCH FOR THE HELP!

[paschalidoud]

Hi all,

@AIMads I am very sorry for my late reply and for your trouble with running the code. Thanks a lot @tommaoer for your suggestions and for helping out @AIMads figuring this out. As @tommaoer pointed out in their first comment the error was related to the computation and storage of the threed_future_model.pkl that is used for efficient loading of the ThreedFutureDataset class. In order to generate this file properly, I normally do something like this in an ipython shell:

In [1]: from scene_synthesis.datasets.base import THREED_FRONT_BEDROOM_FURNITURE, \
   ...:     THREED_FRONT_LIVINGROOM_FURNITURE, THREED_FRONT_LIBRARY_FURNITURE
   ...: from scene_synthesis.datasets.common import BaseDataset
   ...: from scene_synthesis.datasets.threed_front import ThreedFront

In [2]: with open("config/invalid_threed_front_rooms.txt", "r") as f:
   ...:     invalid_scene_ids = set(l.strip() for l in f)
   ...: 
   ...: with open("config/black_list.txt", "r") as f:
   ...:     invalid_bbox_jids = set(l.strip() for l in f)
   ...: 
   ...: scenes_dataset = ThreedFront.from_dataset_directory(
   ...:      "path_to_3d_front_dataset_dir/3D-FRONT/",
   ...:      "path_to_3d_future_dataset_dir/3D-FUTURE-model/model_info.json",
   ...:      "path_to_3d_future_dataset_dir3D-FUTURE-model/3D-FUTURE-model",
   ...:      "path_to_3d_front_dataset_dir/3D-FRONT_room_layouts/",
   ...:      None,
   ...:      BaseDataset.filter_compose(
   ...:          BaseDataset.with_room("bed"),
   ...:          BaseDataset.at_least_boxes(3),
   ...:          BaseDataset.at_most_boxes(13),
   ...:          BaseDataset.with_object_types(list(THREED_FRONT_BEDROOM_FURNITURE.keys())),
   ...:          BaseDataset.with_generic_classes(THREED_FRONT_BEDROOM_FURNITURE),
   ...:          BaseDataset.with_valid_scene_ids(invalid_scene_ids),
   ...:          BaseDataset.with_valid_bbox_jids(invalid_bbox_jids),
   ...:          BaseDataset.contains_object_types(["double_bed", "single_bed", "kids_bed"]),
   ...:          BaseDataset.room_smaller_than_along_axis(4.0, axis=1),
   ...:          BaseDataset.room_larger_than_along_axis(-0.005, axis=1),
   ...:          BaseDataset.floor_plan_with_limits(6, 6, axis=[0,2])
   ...:      )
   ...: )
Loading dataset  6812 /  6813

In [3]: objects = {}

In [4]: for si in scenes_dataset:
    ...:     for bi in si.bboxes:
    ...:         objects[bi.model_jid] = bi

In[22]: future_objects = [vi for vi in objects.values()]

In [5]: from scene_synthesis.datasets.threed_future_dataset import ThreedFutureDataset

In [6]: objects_dataset = ThreedFutureDataset(future_objects)

In [7]: import pickle

In [8]: with open("/tmp/threed_future_model.pkl", "wb") as f:
    ...:     pickle.dump(objects_dataset, f)

Please note that you will have to do this process also for the living/dining rooms that contain more objects than bedrooms. I will add this explanation to the README in order to avoid future confusion. I am closing the issue for now but please feel free to reopen it in case you struggle with this again in the future.

Once again I am very sorry for not replying sooner!

Best, Despoina