Can I open and edit the scenes from procthor-10k dataset in Unity.editor?

[MingZwhy]

Now I can edit the scenes in ai2thor in Unity.editor, but I don't know how to edit scenes in procthor-10k dataset in Unity.editor. I guess the root of the problem is that how to build a unity scene file from a scene.json

[winthos]

So the challenge of editing a procthor scene in-editor is that all procthor scenes are generated at runtime, so they are not saved or cached in a way that you can open up a usual Unity scene.

The best way to "open" a procthor scene in the editor is to take the json and place it in the directory Assets/Resources/rooms

Then, go to Assets/Scenes/Procedural and open up the Procedural.unity scene.

While in this scene, hit Play and then navigate your cursor to the text box that shows up to the bottom left of the Game window that reads "Enter text..."

This box allows actions to be executed directly in the editor without needing to link to the python controller, and all actions are defined in the script DebugInputField.cs located in Assets/Scripts

To start, type initp into this box to initialize a default agent and press enter to execute. Then, type chp <json file's name> and hit enter again to generate a house based on the json file you saved to the Assets/Resources/rooms directory.

From here, you will be able to see and manipulate the objects created based on the json, however editing them here will not save the json. You will need to then go back into the json and edit any new values like changed object positions, by taking the values of manipulated objects in the editor and copying them over into the json itself.

[MingZwhy]

Thank you very much for your guidance！I "open" the procthor scene in unity.editor successfully following your methods.

[YandanYang]

Hi. I really appreciate this question and your answer. However, when I type chp <json file's name>, I got the error Action failed with error message 'NullReferenceException: Object reference not set to an instance of an object.trace : at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.CreaterHouse).... It happens with every json file. Can you help with this? I do not know what's wrong or what should I do.

[MingZwhy]

Hi. I really appreciate this question and your answer. However, when I type chp <json file's name>, I got the error Action failed with error message 'NullReferenceException: Object reference not set to an instance of an object.trace : at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.CreaterHouse).... It happens with every json file. Can you help with this? I do not know what's wrong or what should I do.

You can try to open it using unity editor 2021.3.11f1c1, I found same error message when I try to open it using edition 2020，my solution is to git clone the repo again and using unity editor 2021.3.11f1c1.

[YandanYang]

I have tried the version 2021.3.11f1c1 as well as 2020.3.25f1 LTS in Linux and Windows. Sadly, none of these work.

[JiMa25]

Thank you! I open the procthor scene in unity.editor successfully ，now,I want to save the scene as an Asset file,how to make it? @winthos

[winthos]

Thank you! I open the procthor scene in unity.editor successfully ，now,I want to save the scene as an Asset file,how to make it? @winthos

The ProcTHOR scenes are all generated dynamically at runtime based on the parameters specified in the json for each scene. Because of this, there is no way to save a scene in the traditional sense as a Unity scene file as the scene itself only exists at runtime. Any changes you want to make in the Unity Editor you will need to log and then make those same updates to the json for that scene in order to "save" your changes.

[Singh-sid930]

has anyone able to figure out the correct Editor version for this ? I am using linux and trying the 2021.3.11f1 version. It still throws the same error

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.NullReferenceException: Object reference not set to an instance of an object at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.CreateHouse (Thor.Procedural.Data.ProceduralHouse house) [0x00035] in /home/ANT.AMAZON.COM/hartsid/workspace/progen/src/ai2thor/unity/Assets/Scripts/BaseFPSAgentController.cs:6000 at (wrapper managed-to-native) System.Reflection.RuntimeMethodInfo.InternalInvoke(System.Reflection.RuntimeMethodInfo,object,object[],System.Exception&) at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x0006a] in <12d27ebfb46146b18b40876234894c77>:0 --- End of inner exception stack trace --- at System.Reflection.RuntimeMethodInfo.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00083] in <12d27ebfb46146b18b40876234894c77>:0 at System.Reflection.MethodBase.Invoke (System.Object obj, System.Object[] parameters) [0x00000] in <12d27ebfb46146b18b40876234894c77>:0 at ActionDispatcher.Dispatch (System.Object target, DynamicServerAction dynamicServerAction) [0x001f4] in /home/ANT.AMAZON.COM/hartsid/workspace/progen/src/ai2thor/unity/Assets/Scripts/ActionDispatcher.cs:381 at UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController.ProcessControlCommand (DynamicServerAction controlCommand, System.Object target) [0x0007d] in /home/ANT.AMAZON.COM/hartsid/workspace/progen/src/ai2thor/unity/Assets/Scripts/BaseFPSAgentController.cs:1977 UnityEngine.Debug:Log (object) UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController:ProcessControlCommand (DynamicServerAction,object) (at Assets/Scripts/BaseFPSAgentController.cs:2022) UnityStandardAssets.Characters.FirstPerson.BaseFPSAgentController:ProcessControlCommand (DynamicServerAction) (at Assets/Scripts/BaseFPSAgentController.cs:1962) UnityStandardAssets.Characters.FirstPerson.DebugInputField:Execute (string) (at Assets/Scripts/DebugInputField.cs:4018) UnityEngine.EventSystems.EventSystem:Update () (at Library/PackageCache/com.unity.ugui@1.0.0/Runtime/EventSystem/EventSystem.cs:501)

[Singh-sid930]

Also getting this error when running the Procedural.Unity scene Socket exception: System.Net.Sockets.SocketException (0x80004005): Connection refused at System.Net.Sockets.Socket.Connect (System.Net.EndPoint remoteEP) [0x000b6] in <fa31ff7c1e6a4d0894ebb6fee0a2ea37>:0 at AgentManager+<EmitFrame>d__91.MoveNext () [0x001ab] in /home/ANT.AMAZON.COM/hartsid/workspace/progen/src/ai2thor/unity/Assets/Scripts/AgentManager.cs:1045

[Singh-sid930]

seems like the default jsons present in the code do not work. including the json from the 10k dataset do.

[Inshsang]

@winthos hello，we need an indoor point cloud dataset for our recent task, and I wonder Is it possible to export the scene when Unity is running, using some of the Unity operations?I look forward to your reply.

[zjwzcx]

@winthos hello，we need an indoor point cloud dataset for our recent task, and I wonder Is it possible to export the scene when Unity is running, using some of the Unity operations?I look forward to your reply.

You can install FBX exporter in Unity's package manager. And then export your scene as fbx mesh. Editting it by 3D software like blender and meshlab, you can obtain the point cloud.