add a code editor feature

[kokushkin]

Hi! I'm wondering if it would be useful to add a code editor experience inside of the VR so you don't have to take off the helmet again and again? @mtschoen @amirebrahimi Sorry for bothering you guys I'm just trying to understand... is it still alive? (the project) because the last commit was made last autumn and I'm looking for ways to contribute. I have some more ideas, but little experience with Unity though good experience in programming in general. Anyway, don't want to deal with unfamiliar project alone)

[kyleunity]

Hi! A code editor would indeed be a cool feature. How do you imagine it would look? I'm wondering if you'd want to use a keyboard and your existing code editor (Visual Studio/Rider/etc.). With a virtual desktop overlay you can (sort of) currently do that, but in my experience it was not great to read text. Perhaps on a newer higher resolution headset that would not be so bad. The other option I imagine would be a truly in-unity text editor that you interact with using either keyboard or VR buttons/gesture. That interface could be natively designed to navigate files and methods in VR.

As to whether this project is still alive, the short answer is yes. The longer answer is that development efforts were shifted to other XR projects which slowed down updates to this one. However we do intend to update EditorXR (and have started already) with some significant refactors to things like controller input (using the new input system package) and general interface improvements. This includes separating out the VR specific functionality and using more robust "runtime authoring" (mentioned in the MARS Companion app blog post) that will allow built apps to retain some functionality that is currently only in the editor. We still have a bit of work to do in research and planning because this will affect all corners of the EditorXR codebase, but we will share updates on the forum as we have them.

[kokushkin]

Thanks for the explanations!

A code editor would indeed be a cool feature. How do you imagine it would look? I'm wondering if you'd want to use a keyboard and your existing code editor (Visual Studio/Rider/etc.).

Idk exactly, but I would say you don't want to release your controllers to start typing on your keyboard (and definitely keyboard still the best thing to type on) and then grab them back so maybe you don't need controllers at all)

With a virtual desktop overlay you can (sort of) currently do that, but in my experience it was not great to read text. Perhaps on a newer higher resolution headset that would not be so bad.

Yeah, the resolution is still poor, though we can think of it this way: was it better on very old screens in the very beginning? Yet people were typing on them))

Thanks again, I need to explore more about topics what you mentioned)

[kokushkin]

Ok, the idea will be to navigate to an object with the VR controllers, then press some button and window with the object's source code will pop up. At this moment you leave your controllers and start blind typing with the keyboard. When you're done, you press some button, the window disappears, reloading (if needed), then you grab your controllers back, check the behavior and navigate to another object. Should be something similar to https://youtu.be/SKPYx4CEIlM?t=182.

Ok, I'm going to explore more in this direction to understand how useful it might be and then will try to make some implementation.

[mtschoen-unity]

Something to keep in mind is that the Editor locks up a few times when compiling your scripts, which would also lock up VR rendering. Now that we have out-of-process VR compositors that can "fade to grey" when this happens, the experience isn't quite so nausea-inducing, but you would not be able to use EditorXR while the Editor is compiling. Unfortunately, this kind of prevents us from implementing the use case you shared. However, such a workflow would be reasonably easy with an inspector and game scripts that can update on the fly, and are set to run in Edit mode.

[kokushkin]

Thanks for the feedback!

Something to keep in mind is that the Editor locks up a few times when compiling your scripts, which would also lock up VR rendering. Now that we have out-of-process VR compositors that can "fade to grey" when this happens, the experience isn't quite so nausea-inducing, but you would not be able to use EditorXR while the Editor is compiling. Unfortunately, this kind of prevents us from implementing the use case you shared.

Yeah, I thought about this.. I think I'm gonna try one person editing scripts and another person watching in VR to see how bad it might be))

However, such a workflow would be reasonably easy with an inspector and game scripts that can update on the fly, and are set to run in Edit mode.

Not sure how it works and what the difference between these scripts (regular vs inspector) (sorry for this newbie question. I've just started exploring the area so it might be obvious to me very soon)

[mtschoen-unity]

Not sure how it works and what the difference between these scripts (regular vs inspector)

My bad on the wording there. What I'm trying to suggest is that you have a script which uses [ExecuteInEditMode] or where you somehow set runInEditMode to true so that its Update method is called in Edit mode. Then, if you select the object in EditorXR and use the inspector to modify its values, you should see it update in real time.

So for the example you showed, you might have a TreeBuilder script which checks for changes in property values in Update and rebuilds the tree if they change. Then you can select the tree, open the Inspector Workspace, and start changing values and you should see it update.

[kokushkin]

Thanks for the advice! We will definitely try this out)

[haywirephoenix]

Came here to request this feature! Ideas:

Script Inspector (asset) has come very far in creating an inspector based code editor with all the syntax highlighting and iltellisense goodies. Seems it's easier for you guys to show a Unity editor window in EditorXR than mirror an external window like VScode.

I too would be typing on a physical keyboard for sure. For non touch typers a camera feed view of the keyboard could be a nice touch.

For the compilation hang, I wouldn't mind a fade transition but I can see this is where a potential Bolt editor window would shine.

[kokushkin]

Hi! @haywirephoenix We (me and another person ;) ) haven't started the implementation yet (We were exploring Unity, took some courses, made some simple games...). But we're going to do it soon, I hope) (at least we'll try and give a feedback). @haywirephoenix ping me if you start doing something, we'd be happy to participate somehow)

[kokushkin]

Hi! We tried the concept with script inspector 3. This is what we've found.

These are problems that we've encountered with ( from more to less important , IMO)

1) When you hit play from the XR Editor the rotation is getting frozen (when you rotate your head) in the game. It stays frozen until you restart Unity Editor. 2) Nothing to grab by default to exit the game. Would be nice to have something by default (maybe hotkey instead?) 3) Have to get used to blind typing . Showing the keyboard image while typing might help greatly. Hands tracking? 4) When you press save, and then, when you return to the Unity Editor you have to choose it once again in the oculus menu(why?). 5) ~7 sec before you can continue to work vs ~4 sec in just Unity Editor . But it’s not so bad (IMO). It switches you back in XR Editor which is rather not a good behavior (takes additional time as well). 6) When an object is chosen in the XR Editor you don’t have its scripts automatically opened in Unity Editor (after pressing the oculus menu button). Have to navigate manually again.

I think I'm gonna create a separate issue about (1) because it makes sense even without code editing.