Randomize camera trajectory?

[zlou]

Hi, I am wondering if there are ways to create camera trajectories randomly so one can generate large amount of data efficiently rather than manually creating level sequences? I tried to find some way to creating level sequences randomly but did not have much luck. Any pointers would be really great. I also found this blog post. Section "Automatic navigation and screenshots" talks about how to automatic navigate through the scene. I am wondering if that would be a good way to go and if it will work with EasySynth?

[NikolaJov96]

Hi @zlou, The trajectory randomization sounds like a great idea.

Regarding the blog post, there should be two ways of capturing images from UE, screenshots and the sequencer. For EasySynth we decided to use the sequencer because it should result in much better quality images (does not drop frames, takes its time to render each image well). As a consequence we cannot program a camera to move around like we could while the game is running, instead, we have to use keyframes.

Now, generating new or randomizing existing keyframes should be possible through a plugin, but would require a lot of digging through the sequencer API. I guess it would look something similar to how camera poses are extracted in the ExtractCameraTransforms method in Source\EasySynth\Private\RendererTargets\CameraPoseExporter.cpp. The way I would approach this is to try and dig out the method inside the engine that gets triggered when manually creating a keyframe describing actor location, and try to access that method through the plugin, or mimic its functionality.

[zlou]

I spent some time last week on this. I found that one could use UE's Python editor scripting to spawn cameras, create camera track and add keyframes as what can be done manually. In other words, we could use python to create a LevelSequence with random camera trajectory. The tutorials under UE_4.27/Engine/Plugins/MovieScene/SequencerScripting/Content/Python are useful to me when I try to do this. Specifically, sequencer_examples.py and sequencer_key_examples.py.

[kzhang2]

I have implemented this if anybody is interested. I think in general generating the camera trajectory programmatically using the python API is useful, maybe I could make a pull request to add my code as a sample?

[zlou]

I have implemented this if anybody is interested. I think in general generating the camera trajectory programmatically using the python API is useful, maybe I could make a pull request to add my code as a sample?

It would be interesting to see how others implemented this. By the way, did you consider some way of collision avoidance to avoid sampling inside a building etc? I was unable to figure out a native Unreal approach to do this.

[fpanjevic]

I have implemented this if anybody is interested. I think in general generating the camera trajectory programmatically using the python API is useful, maybe I could make a pull request to add my code as a sample?

That would be great, please do.

[kzhang2]

I have implemented this if anybody is interested. I think in general generating the camera trajectory programmatically using the python API is useful, maybe I could make a pull request to add my code as a sample?

It would be interesting to see how others implemented this. By the way, did you consider some way of collision avoidance to avoid sampling inside a building etc? I was unable to figure out a native Unreal approach to do this.

I have not considered collision avoidance because I am working with simple scenes that only contain one object. I don't think there's a simple native Unreal approach to handle the camera trajectory potentially intersecting with the scene. The simplest way I can think of is to use some 3D computational geometry library to compute the morphological dilation of the scene (with respect to the radius of the smallest sphere that contains the camera) and then sample only camera positions that lie outside the dilation.

I have implemented this if anybody is interested. I think in general generating the camera trajectory programmatically using the python API is useful, maybe I could make a pull request to add my code as a sample?

That would be great, please do.

Ok, where should I put it, the Scripts directory? Also, where should I put the documentation for how to run/adapt the script, the README.md or a new file from within the Scripts directory?