search

ayaanzhaque / instruct-nerf2nerf

Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions (ICCV 2023)
https://instruct-nerf2nerf.github.io/
MIT License
792 stars 70 forks source link

How can I change instruct-pix2pix to another image-edited model? #46

Closed lxmcwt closed 1 year ago

lxmcwt commented 1 year ago

This is a great work. It combines the instruct-pix2pix and NeRF models very well to achieve a very good effect. I noticed another work called HIVE, which is an improved version based on instruct-pix2pix. It adds RLHF on the basis of instruct-pix2pix, and the basic model structure remains unchanged. Now I want to try to replace the instruct-pix2pix component in instruct-nerf2nerf with HIVE to exercise my programming skills. However, since the HIVE model has not been released on Hugging Face like instruct-pix2pix, the method of loading the model needs to be changed. I have tried many methods, but none of them have been successful. This is very frustrating for me. Could you give some feasible suggestions based on your professional knowledge? Thank you very much.

ayaanzhaque commented 1 year ago

If you provide a link to their codebase I could give some suggestions. I would suggest to either wait for the model to be released on Huggingface diffusers, as their package works really well for simple inference tasks like ours. However, I think what could work for your use case is to either pip install their github codebase or move the files into a folder in this repository, then just load it in the same way they explain in their github.

ayaanzhaque commented 1 year ago

Will close this issue as there hasn't been a response in a while.