Automatically collecting data from SUNCG dataset

[tailongnguyen]

Hi, I wonder how you guys can collect data (image/segmentation) from SUNCG dataset with minos simulator. There are more than 45000 scenes so you cannot just load each scene in and run the robot around collecting images. Any suggestion would be appreciated!

[msavva]

Hi @tailongnguyen , thanks for trying out MINOS. The way you would use simulation to generate data does depend on the specific problem you are trying to address. For example, in benchmarking various visual navigation methods we generated training, validation and testing episodes (see https://github.com/minosworld/scenarios) by randomly sampling starting positions and goal positions within a subset of SUNCG houses.

[tailongnguyen]

@msavva actually I thought about randomly sampling the position of the robot and taking the corresponding pair of image/segmentation but did not figure out how to do that. Could you help me about that?

[msavva]

You can generate new episodes with randomly sampled starting positions and goal positions using the script described at https://github.com/minosworld/minos#presampling-episodes-for-reproducible-testing . For example: python3 generate_episodes.py --scenes ../data/scenes.multiroom.csv --output new_episode_states.suncg.csv

[tailongnguyen]

python minos/tools/generate_episodes.py --scene_ids d0c3971878bfb42e81876c6a6501bda8 --output test_episode.txt --samples_per_scene 1

After running this line of code I encountered this error:

2018-07-18 21:28:51,772 INFO sim00:Starting sim server at /home/tailongnguyen/minos/minos/server/server.js with port 59768
2018-07-18 21:28:52,782 WARNING localhost:59768/socket.io [waiting for connection] HTTPConnectionPool(host='localhost', port=59768): Max retries exceeded with url: /socket.io/?EIO=3&transport=polling&t=1531924132779-0 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f733cb0d860>: Failed to establish a new connection: [Errno 111] Connection refused',))
2018-07-18 21:28:54,462 INFO sim00:connect
2018-07-18 21:28:54,693 INFO sim00:inited
{'position': [-39.47999954223633, 1.0799999217390397, -42.000000048428774], 'bbox': {'min': [-39.99992352724075, 0.01999997699248368, -42.090000100433826], 'max': [-38.9600755572319, 2.1399998664855957, -41.90999999642372]}, 'objectId': '0_11', 'initialOffsetFromAgent': [-0.49247267679219164, 0, -0.11660586675683934], 'cell': {'i': 31, 'j': 2, 'id': 149, 'isValid': True}}
Traceback (most recent call last):
  File "minos/tools/generate_episodes.py", line 134, in <module>
    main()
  File "minos/tools/generate_episodes.py", line 130, in main
    run(args)
  File "minos/tools/generate_episodes.py", line 58, in run
    args.samples_per_scene, i)
  File "minos/tools/generate_episodes.py", line 39, in process_scene
    write_configuration(f, edict(sim.get_scene_data()['data']), i_level)
  File "minos/tools/generate_episodes.py", line 77, in write_configuration
    % (scene_id, level, g.room, groomtype, sp[0], sp[1], sp[2], s.angle,
AttributeError: 'EasyDict' object has no attribute 'room'
2018-07-18 21:29:23,247 INFO sim00:disconnect
2018-07-18 21:29:23,248 INFO sim00:Stopping the simulator
2018-07-18 21:29:23,248 INFO Counter({'frames_received': 1})
2018-07-18 21:29:23,248 INFO sim00:Stopping child servers
2018-07-18 21:29:23,248 INFO sim00:Killing sim pid 12416
2018-07-18 21:29:23,295 INFO sim00:Stopped child servers
2018-07-18 21:29:23,296 INFO sim00:Simulator killed.

Could you tell me how to fix that? I searched over the issues and did not find something helpful.

[msavva]

Can you please try with the version of the code on the dev branch? Please note that you will need to re-install through pip and compile the server module using npm.

[tailongnguyen]

I tried on the dev version and made it works. However, I am still confused about how exactly I can use the csv file to get data from sensor data (color image, segmentation) in the generated episode. I added these lines of code to the file generate_episode.py but it did not work.

if level >= 0:  # do one level
        sim.configure({'scene': {'level': level}})
        for j in range(0, n_episodes):
            if j > 0:
                sim.reset()
            else:
                sim.start()
            ########### added
            action = {'name': 'idle', 'strength': 1, 'angle': math.radians(5)}
            response = sim.step([action], 1)
            observation = response.get('observation')
            sensor_data = observation.get('sensors')
            color_img = sensor_data['color']['data'][:, :, ::-1][..., 1:]
            color_img = color_img.reshape(
                (color_img.shape[1], color_img.shape[0], color_img.shape[2]))
            print(color_img)
            cv2.imwrite('obj_im.png', color_img)
            ###########

            write_configuration(f, edict(sim.get_scene_data()['data']), level)