Add Command for Sensor Shift and Actor Edits

[alchemz]

Is your feature request related to a problem? Please describe. Sensor shift and actor Edits

Describe the solution you'd like I could not find any information on performing sensor shifts and actor edits as described in Figure 1 of the paper. Could you please add some instructions for this? It would be very helpful to enable a launch command like --actor-edit {lateral_shift: 3} to support this.

Describe alternatives you've considered I found some actor edit options under the viewer, but it is very inconvenient to allow actor edits only in viewing mode. Additionally, it is unclear how to use the viewer for actor edits.

[amoghskanda]

RenderInterpolated class in scripts/render.py is where you can change the sensor's trajectory by adding a shift. To edit the trajectory of all actors, you can add a shift to actor_shift in the same class. If you want to change trajectories of specific actors, you can add shifts to them w.r.t to their id in the modify_actors function. You can render an interpolated video to check the results

[TurtleZhong]

RenderInterpolated class in scripts/render.py is where you can change the sensor's trajectory by adding a shift. To edit the trajectory of all actors, you can add a shift to actor_shift in the same class. If you want to change trajectories of specific actors, you can add shifts to them w.r.t to their id in the modify_actors function. You can render an interpolated video to check the results

I found the code here. so if I wanna only move the specify vehicle, I need change some codes like add the actor id property.

[amoghskanda]

yeah so essentially you iterate over the actor_positions which is a 40x47x3 tensor...40 frames, 47 dynamic objects, 3 axes and you add shift to the object wrt to it's id ranging from 0 to 46. Also, have you rendered lidar pointclouds for neurad? As mentioned in #20

[TurtleZhong]

I am sorry but I have not tried render the lidar pointclouds yet. I check the issues of neurad studio, It seems that you have implemented modifications for a single dynamic object(eg. change line, remove add new vehicle).

[amoghskanda]

I was able to operate on single dynamic objects, single object + camera sensor, multiple objects and multiple objects + camera sensor. I've mentioned how I did it in the above comment. However, I haven't tried adding new vehicles. Have you done it?

[TurtleZhong]

I was able to operate on single dynamic objects, single object + camera sensor, multiple objects and multiple objects + camera sensor. I've mentioned how I did it in the above comment. However, I haven't tried adding new vehicles. Have you done it?

No, but I have tried adding new vehicles using 3D Gaussian Splatting.

[TurtleZhong]

I was able to operate on single dynamic objects, single object + camera sensor, multiple objects and multiple objects + camera sensor. I've mentioned how I did it in the above comment. However, I haven't tried adding new vehicles. Have you done it?

I would like to ask you whether you added actor_id in the modify_actors function, for example, to only operate on actors with specific ids. I found the relevant definition of dynamic id in dyanmic_actor.py here. In addition, it seems that the author commented here that adding actor id allows us to duplicate actors during rendering. My guess is that just make a copy of the actor with a specific id and change its pose, I'm not sure if this is possible.

[amoghskanda]

I was able to operate on single dynamic objects, single object + camera sensor, multiple objects and multiple objects + camera sensor. I've mentioned how I did it in the above comment. However, I haven't tried adding new vehicles. Have you done it?

I would like to ask you whether you added actor_id in the modify_actors function, for example, to only operate on actors with specific ids. I found the relevant definition of dynamic id in dyanmic_actor.py here. In addition, it seems that the author commented here that adding actor id allows us to duplicate actors during rendering. My guess is that just make a copy of the actor with a specific id and change its pose, I'm not sure if this is possible.

No I didn't add anything. I added a shift to an actor with a specific actor id.