How to use image outside of LFW?

[geopapyrus]

Hi,

Is there a way to apply a transformation to an image that is not in the LFW dataset?

thank you

[braun-steven]

Yes, you can use the command-line argument --person-image to specify the path to the image you want to transform.

[geopapyrus]

Thanks for your quick reply!

I just tried running it with an image outside of the dataset and it throws an error

./env/bin/python dfi-tensorflow/src/main.py -d dfi-tensorflow/data/ -m vgg19.npy -g --optimizer adam --steps 2000 --lr 0.1 --rebuild-cache --k 100 --lamb 0.001 --beta 2 --alpha 0.4 -f 'Mustache' --person-image /home/xxx/Downloads/dfi/Archetypal-Male-Face-of-Beauty-embargoed-to-00.01hrs-30.03.15-589x600.jpg

`se tf.global_variables_initializer instead. Traceback (most recent call last): File "/home/xxx/Downloads/dfi/dfi-tensorflow/src/utils.py", line 53, in get_person_idx_by_path index = df[df['path'] == path].index.tolist()[0] IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "dfi-tensorflow/src/main.py", line 109, in main() File "dfi-tensorflow/src/main.py", line 105, in main dfi.run() File "/home/xxx/Downloads/dfi/dfi-tensorflow/src/DFI.py", line 46, in run phi_z = self._get_phi_z_const() File "/home/xxx/Downloads/dfi/dfi-tensorflow/src/DFI.py", line 121, in _get_phi_z_const start_img_path) File "/home/xxx/Downloads/dfi/dfi-tensorflow/src/utils.py", line 56, in get_person_idx_by_path 'Image with the path {} could not be found'.format(path)) Exception: Image with the path /home/xxx/Downloads/dfi/Archetypal-Male-Face-of-Beauty-embargoed-to-00.01hrs-30.03.15-589x600.jpg could not be found`

Am I doing something wrong?

thank you

[braun-steven]

Oh, sorry. I was a little confused. That was unfortunately not implemented yet. Though it is fairly to easily add it yourself:

Looking at this line: https://github.com/slang03/dfi-tensorflow/blob/master/src/utils.py#L42

You currently need to put the image into the data directory in it's according name directory (E.g. data/lft-deepfunneled/Max_Mustermann/Max_Mustermann_0001.jpg).

Additionally you need to add all attributes manually to the lfw_attributes.txt file.

It is sufficient to use -1/1 as attribute values.

That should hopefully work.

Our initial plan was to include an automatic svm regression for all attributes as this is how the lfw-attributes.txt was generated. Maybe if I find time I'll include in at some point in the future.

Tell me if you are successful (or if not, what went wrong).

[geopapyrus]

I was able to run it now.

I manually added -1/1 for all the attributes.

Would you be able to point me towards a way of automatic generating the lfw-attributes for the image? Is that implementation open source at another repository?

Also this is the result I got for this image. It seems to automatically crop the face...

./env/bin/python dfi-tensorflow/src/main.py -d dfi-tensorflow/data/ -m vgg19.npy -g --optimizer adam --steps 2000 --lr 0.1 --rebuild-cache --k 100 --lamb 0.001 --beta 2 --alpha 0.4 -f 'Mustache' --person-image dfi-tensorflow/data/lfw-deepfunneled/Max_Mustermann/Max_Mustermann_0001.jpg

./env/bin/python dfi-tensorflow/src/main.py -d dfi-tensorflow/data/ -m vgg19.npy -g --optimizer adam --steps 2000 --lr 0.1 --rebuild-cache --k 100 --lamb 0.001 --beta 2 --alpha 0.8 -f 'Mustache' --person-image dfi-tensorflow/data/lfw-deepfunneled/Max_Mustermann/Max_Mustermann_0001.jpg

lfw attributes line for this image:

Max Mustermann 1 1.00 -1.00 1.00 -1.00 -1.00 -1.00 1.00 -1.00 -1.00 1.00 -1.00 -1.00 -1.00 1.00 -1.00 -1.00 -1.00 -1.00 -1.00 -1.0-1.00 -1.00 -1.00 -1.00 -1.00 -1.00 -1.00 1.00 -1.00 -1.00 -1.00 1.00 -1.00 -1.00 1.00 -1.00 -1.00 1.00 1.00 -1.00 -1.00 1.00 -1.0-1.00 1.00 1.00 -1.00 1.00 -1.00 -1.00 -1.00 1.00 -1.00 1.00 1.00 1.00 -1.00 -1.00 -1.00 -1.00 -1.00 1.00 -1.00 -1.00 -1.00 -1.0-1.00 -1.00 -1.00 -1.00 -1.00 -1.00 -1.00

thank you

[braun-steven]

Would you be able to point me towards a way of automatic generating the lfw-attributes for the image? Is that implementation open source at another repository?

According to the lfw website, they have used this approach. Unfortunately I couldn't find an implementation for it.

It seems to automatically crop the face...

Oh yes, we only implemented cropping. Feel free to adjust the code to enable scaling instead of cropping here and submit a PR of you want to, it should be pretty easy. Since I don't have access to any computer in the next weeks I can only accept PRs.

[braun-steven]

Also make sure to have the facial features (eyes, mouth, nose) approximately aligned with those of the lfw dataset. As stated in the paper, the method works best for similar aligned pictures (hence the dataset is preprocessed by deepfunnel alignment)

[geopapyrus]

Ok. I manually aligned my input image to match one of Trump's images in terms of eye,nose and mouth and it worked great!

Here's my input

Asian:

Black:

Bald:

Two quick questions if you don't mind ;)

1) Why didn't the bald work? I mean why didn't he get completely bald? 2) I found a project that generates face attributes like the CelebA dataset ones for any arbitraty image (https://github.com/tornadomeet/mxnet-face) In that sense, how hard would it be to make the project use the CelebA dataset as opposed to the LFW ? In that way I wouldn't have to manually generate the attributes!

thank you

[braun-steven]

Why didn't the bald work? I mean why didn't he get completely bald?

That might have several reasons such as that the nearest neighbor search delivered bad results for that specific person. On the other side, the algorithm is quite parameter sensitive so you might find better results with lower k and higher alpha but you just need to test that out a little bit. In our experiments we found out that there is no parameter setup that "fits it all".

I found a project that generates face attributes like the CelebA dataset ones for any arbitraty image (https://github.com/tornadomeet/mxnet-face) In that sense, how hard would it be to make the project use the CelebA dataset as opposed to the LFW ? In that way I wouldn't have to manually generate the attributes!

It should be fairly easy as you only need to adjust the data loading methods in utils.py to your needs. It doesn't matter if the attributes are different from lfw as it is just used internally to find the k nearest neighbors.

I feel bad now for not making it dynamic in the first place but hardcoding the lfw paths etc. I can fix it in two months earliest so tell me if you need any help patching that yourself.