search

kerrj / lerf

Code for LERF: Language Embedded Radiance Fields
https://www.lerf.io/
MIT License
668 stars 65 forks source link

Rendering images for evaluation #23

Open nicolaihaeni opened 1 year ago

nicolaihaeni commented 1 year ago

Hi

Thank you for open-sourcing your great work. I was wondering if there currently exists functionality to render images and activation maps for evaluation. If so, could you provide the evaluation code?

Thank you

kerrj commented 1 year ago

Hi! We haven't had time to integrate ns-render, ns-export yet, but if you set the phrases you'd like to render in the image encoder using set_positives inside the render.py script, you should then be able to set the output type to relevancy_{0,1,2, etc} to render individual phrases. This code will be quite slow since it computes the LERF scale on a per-frame basis.

We're planning to fully integrate rendering and exporting in the future, not sure when that will be. Integrating fully would entail rendering the trajectory first downsampled in time and resolution to pick the best scale for each query, then fixing that scale and rendering a high-res video, as well as exposing query phrases as command-line args.

anas-zafar commented 1 year ago

@kerrj any update on integrating ns-render in the code base? Thanks

anas-zafar commented 1 year ago

Hi @nicolaihaeni , were you able to render images and activation maps for evaluation?