Unable to CreateHouse example.py

[pushkalkatara]

Hi,

Running the example script gives the following error:

Traceback (most recent call last):
  File "/home/sirdome/EmbodiedProject/new/procthor/scripts/example.py", line 6, in <module>
    house, _ = house_generator.sample()
  File "/home/sirdome/EmbodiedProject/new/procthor/procthor/generation/__init__.py", line 287, in sample
    gfs.add_small_objects(
  File "/home/sirdome/EmbodiedProject/new/procthor/procthor/generation/small_objects.py", line 83, in default_add_small_objects
    assert event, "Unable to CreateHouse!"
AssertionError: Unable to CreateHouse!

The event error message gives:

<ai2thor.server.Event at 0x7f4fc89100a0
    .metadata["lastAction"] = CreateHouse
    .metadata["lastActionSuccess"] = False
    .metadata["errorMessage"] = "Newtonsoft.Json.JsonSerializationException: Error converting value "MediumWoodSmooth" to type 'Thor.Procedural.Data.MaterialProperties'. Path 'house.rooms[0].floorMaterial'. ---> System.ArgumentException: Could not cast or convert from System.String to Thor.Procedural.Data.MaterialProperties.   at Newtonsoft.Json.Utilities.ConvertUtils.EnsureTypeAssignable (System.Object value, System.Type initialType, System.Type targetType) [0x00067] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Utilities.ConvertUtils.ConvertOrCast (System.Object initialValue, System.Globalization.CultureInfo culture, System.Type targetType) [0x00036] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType (Newtonsoft.Json.JsonReader reader, System.Object value, System.Globalization.CultureInfo culture, Newtonsoft.Json.Serialization.JsonContract contract, System.Type targetType) [0x00117] in <f2d82e1ac3694e6399a979c083d80d79>:0     --- End of inner exception stack trace ---   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.EnsureType (Newtonsoft.Json.JsonReader reader, System.Object value, System.Globalization.CultureInfo culture, Newtonsoft.Json.Serialization.JsonContract contract, System.Type targetType) [0x00149] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x000dc] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00069] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateList (System.Collections.IList list, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonArrayContract contract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, System.String id) [0x00173] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateList (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.Object existingValue, System.String id) [0x001c4] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0007f] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.SetPropertyValue (Newtonsoft.Json.Serialization.JsonProperty property, Newtonsoft.Json.JsonConverter propertyConverter, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerProperty, Newtonsoft.Json.JsonReader reader, System.Object target) [0x00069] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject (System.Object newObject, Newtonsoft.Json.JsonReader reader, Newtonsoft.Json.Serialization.JsonObjectContract contract, Newtonsoft.Json.Serialization.JsonProperty member, System.String id) [0x00266] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x00161] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType, Newtonsoft.Json.Serialization.JsonContract contract, Newtonsoft.Json.Serialization.JsonProperty member, Newtonsoft.Json.Serialization.JsonContainerContract containerContract, Newtonsoft.Json.Serialization.JsonProperty containerMember, System.Object existingValue) [0x0006d] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType, System.Boolean checkAdditionalContent) [0x000db] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.JsonSerializer.DeserializeInternal (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00054] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.JsonSerializer.Deserialize (Newtonsoft.Json.JsonReader reader, System.Type objectType) [0x00000] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType, Newtonsoft.Json.JsonSerializer jsonSerializer) [0x00012] in <f2d82e1ac3694e6399a979c083d80d79>:0    at Newtonsoft.Json.Linq.JToken.ToObject (System.Type objectType) [0x002ff] in <f2d82e1ac3694e6399a979c083d80d79>:0    at ActionDispatcher.Dispatch (System.Object target, DynamicServerAction dynamicServerAction) [0x00180] in <688391d437854f3da453f64e1df76e9d>:0    at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.ProcessControlCommand (DynamicServerAction controlCommand, System.Object target) [0x0007d] in <688391d437854f3da453f64e1df76e9d>:0 
    .metadata["actionReturn"] = None

how do i fix this issue?

[mattdeitke]

Pushed a fix. Can you try now with: https://github.com/allenai/procthor/commit/4e60f5bb31836ce1c87fef9939fcff7bc23dc3c7 ?

[pushkalkatara]

Thanks for pushing the fix. It works now.

Which AI2THOR version / commit should I use with the generated scenes?

I'm currently using the one given in Colab on procthor-10k dataset. I tried running one of these scenes, but Initialization looks pretty weird and the sim is stuck after init.

I think the agent init location is missing in the generated scenes. For procthor-10k dataset it had a specific key Agent. How do add these to the generated jsons?

[mattdeitke]

There was a script for this, but it might be floating around somewhere. I'd have to check for it.

But, basically, use shapely.geometry to create a polygon of each of the rooms from the floorPolygons in the house jsons. Look at using buffer so there is sufficient space between the objects and the position with which the gent would teleport from.

Then, try teleporting the agent to some positions within the polygon, setting y=0.95 for its height. Try this for a bunch of x/z inside of the polygon until any succeeds.

[mattdeitke]

Using the version in the colab is recommended. The agentPoses just aren't specified, so the agent won't be teleported in.

[pushkalkatara]

Thanks for the info. If you come across the script, please do drop it in the thread. Meanwhile, I'll implement it through the logic above.

yes, I thought so, probably teleporting the agent within the polygon should make it work. Thanks!

[pushkalkatara]

Found the script: link. Dropping here in case it helps.

Scene metadata from procthor-10k:

{'agent': {'horizon': 30, 'position': {'x': 3.5, 'y': 0.95, 'z': 2.0}, 'rotation': {'x': 0, 'y': 90, 'z': 0}, 'standing': True}, 'roomSpecId': 'kitchen', 'schema': '0.0.1', 'warnings': {}, 'agentPoses': {'arm': {'horizon': 30, 'position': {'x': 3.5, 'y': 0.95, 'z': 2.0}, 'rotation': {'x': 0, 'y': 90, 'z': 0}, 'standing': True}, 'default': {'horizon': 30, 'position': {'x': 3.5, 'y': 0.95, 'z': 2.0}, 'rotation': {'x': 0, 'y': 90, 'z': 0}, 'standing': True}, 'locobot': {'horizon': 30, 'position': {'x': 3.5, 'y': 0.95, 'z': 2.0}, 'rotation': {'x': 0, 'y': 90, 'z': 0}}, 'stretch': {'horizon': 30, 'position': {'x': 3.5, 'y': 0.95, 'z': 2.0}, 'rotation': {'x': 0, 'y': 90, 'z': 0}, 'standing': True}}}

Scene metadata from procthor:

{'agent': {'position': {'x': 4.0, 'y': 0.95, 'z': 3.5}, 'rotation': {'x': 0, 'y': 0, 'z': 0}, 'standing': True, 'horizon': 30}, 'roomSpecId': 'bedroom', 'schema': '0.0.1'}

agentPoses was missing from the JSoN which was causing the issue.

Adding this in house.py resolved the issue:

    def _add_metadata(self) -> None:
        start_pose = self.choose_agent_pose()
        self.data["metadata"] = {
            "agent": start_pose,
            "roomSpecId": self.room_spec.room_spec_id,
            "schema": SCHEMA,
            "agentPoses": {'default' : start_pose}
        }

Closing the issue. Thanks, @mattdeitke for helping out.