search

kerrj / lerf

Code for LERF: Language Embedded Radiance Fields
https://www.lerf.io/
MIT License
670 stars 65 forks source link

dino inference #13

Open fangli333 opened 1 year ago

fangli333 commented 1 year ago

HI,

when i view your codes on the inference step, I find that when you generate the row_relevancy output, you do not include the output from dino network. So, we are wondering the current results of the code you provided is whether the results from 'lerf w/o dino' or 'lerf w/ dino' (In fig 5 in your paper). Can you help us on it?

Best, Any

kerrj commented 1 year ago

Hi! The code is for LERF w/ DINO, the DINO features aren't used explicitly for inference, only as regularization during training. We haven't integrated visualization for DINO features through PCA or other methods yet, but this might happen in the future.

fangli333 commented 1 year ago

Hi! The code is for LERF w/ DINO, the DINO features aren't used explicitly for inference, only as regularization during training. We haven't integrated visualization for DINO features through PCA or other methods yet, but this might happen in the future.

Thank you for your answer. One more thing is I noticed that nerfstudio can generate the RGB videos of the scene, but can nerfstudio generate relevancy videos of the scene? For example, if I take prompt as 'rabbit' and I want to generate a relevancy video given the customized camera path. Thank you.

Best, Any

kerrj commented 1 year ago

Hi! Sorry for the late reply, this is related to #23. We haven't yet fully supported rendering videos but do plan to in the future. There's some information in that response which might be helpful to you as well.