search

danielroich / PTI

Official Implementation for "Pivotal Tuning for Latent-based editing of Real Images" (ACM TOG 2022) https://arxiv.org/abs/2106.05744
MIT License
897 stars 112 forks source link

saving PTI latent vector w projection to npz file #24

Closed johndpope closed 3 years ago

johndpope commented 3 years ago

Is this achievable? I want to take result and throw it in here - https://github.com/l4rz/stylegan2-clip-approach

Or is this misguided...it's going to destroy the heart of soul of this repo? When I run the projections from pytorch stylegan2-ada the interpretation is crap. not as good as this repo...

danielroich commented 3 years ago

hey @johndpope, Of course! all you need to do is to use "w_pivot_tensor.cpu().detach().numpy()" and then save it as npz using the savez function from numpy. You can find more details about the usage here

Notice that PTI already outputs the new generator and the corresponding W vectors. In order to achieve what you are asking you need to add a saving operation for the npz output format and add a new output format for the generator. Currently is saved in the classic torch format without pickling ( I have used torch.save() to save the tuned generator).

Another option if you want free-text guided editing is to borrow inspiration from StyleCLIP Optimization. Usage demo can be found here.

In general - you can bypass outputting .npz file and pickling the generator by just changing the input format of the given github repositories. It might be easier (even though both methods should not take a long time)

I have already experimented with free-text editing + PTI. The results were truly amazing, the downside was the sensitivity of those methods to the hyperparameters.

johndpope commented 3 years ago

UPDATE - LATEST. npz

so tacking this on to end of notebook gets the data in correct format. Thanks!

plot_image_from_w(w_pivot, new_G)
np.savez(f'projected_w.npz', w=w_pivot.cpu().detach().numpy())

some of my notes here https://gist.github.com/johndpope/c5b77f8cc7d7d008be7f15079a9378bf

so I get the npz file