I see no modifications in the edited NeRF

[benoriol]

Hello,

I run in2n and see no difference when using ns-viewer after training. In the viewer, I see the prompt "don't change the image" and I'm not sure if it has something to do with it. Also I wanted to point out that I think the command line interface for nerfstudio might have changed and I believe for in2n might have too? Maybe my issue is related to that somehow.

I first run

ns-train nerfacto --data data/face --pipeline.datamanager.camera-res-scale-factor 0.5

Note I used a different CLI arg to downscale compared to the --downscale-factor suggested in the in2n repo, which I think is now deprecated as argument in nerfstudio.

I then run

ns-train in2n --data data/face --load-dir outputs/face/nerfacto/timestamp/nerfstudio_models --prompt "give him a moustache" --pipeline.ip2p-device cuda:1

Note I didn't use the CLI to specify guidance scales and I used a slightly different argument than the one suggested in the instructions (--prompt, which seems to be an alias to --pipeline.model.prompt as opposed to the suggested --pipeline.prompt which seemed to not exist.

After it's done training I run ns-viewer --load-config outputs/face/in2n/timestamp_2/config.yml and connect to the viewer.

In the viewer I can only see the original face nerf with no editing. Moreover, I see a textbox titled "prompt" which shows "don't change the image"

I would appreciate help in trying to figure out what I might be doing wrong and why I am seeing no changes when viewing my trained nerf.

Some potentially relevant output from: outputs/face/in2n/timestamp_2/config.yml:

  edit_count: 1
  edit_rate: 10
  guidance_scale: 7.5
  image_guidance_scale: 1.5
  ip2p_device: cuda:1
  ip2p_use_full_precision: true
  lower_bound: 0.02

and then

prompt: give him a moustache

[ayaanzhaque]

I think the issue may be because the prompt isn't being properly parsed in the command line. So --pipeline.model.prompt no longer exists? @tancik have some of these arguments been depricated?

[benoriol]

Thanks for the quick response. Wanted to clarify that --pipeline.model.prompt does exist, --pipeline.prompt does not. Also, by looking at the config file, seems the prompt is being parsed correctly.

Thanks

[ayaanzhaque]

I see, sounds good. So then the downscale resolution is changed now? Could you elaborate a bit more on that

[benoriol]

I am not totally sure how to check if the resolution is actually downscaled. Should I see it in the viewer or where? What I can tell you is that the argument seems to be parsed correctly in the nerfacto config file: camera_res_scale_factor: 0.5 and also I am having no memory issues on in2n.

Do you think that not downscaling properly might be the source of my issue of not seeing any modification?

[benoriol]

Update: I patched by hardcoding my string into ip2p_pipeline.py in the Config class.

[ayaanzhaque]

So pasting it into the code directly solved your problem? that means you somehow aren't parsing the text prompt correctly.

[benoriol]

Yes, it solved the issue. Yup, I agree that might actually be the case. Weird that the prompt was parsed into the config file correctly but still did not seem to have an effect into the pipeline. Thanks for the help.

[Z-YuPeng]

@ayaanzhaque I also meet the same problem, whether I should specify the "--pipeline.model.prompt" "--pipeline.prompt" and "--prompt" at the same time when I run n2n? I find the generated training config shows the --pipeline.model.prompt = null when I use the command in the README.md.

[ayaanzhaque]

Can you try --pipeline.prompt? I believe this is what works

[Z-YuPeng]

Can you try --pipeline.prompt? I believe this is what works

yes，i try it. I got very terrible result like issue, when i try --pipeline.prompt, i found the value of another two is null as shown in the gernerated training yamls. I do not know the reason about it.

[ayaanzhaque]

then i suspect something in the newer nerfstudio versions is causing this poor quality. I will take a look soon and push some changes

[ayaanzhaque]

Let's discuss all this in #60