关于训练

[Jiangzl]

你好，我想请问是否可以使用你的代码对celeba人脸数据集进行训练？因为我看datasets里有imagenet和dog_and_cat的.py文件

[quolc]

Thank you for your comment! We have tested the proposed method using models trained on ImageNet (dog&cat), CUB (birds), VGG flowers datasets, but haven't tried CelebA so far (models for CUB and flowers haven't been publicized yet). We are providing the training script in this repo, so you can train the model on CelebA by yourself.

[Jiangzl]

好的，谢谢~我是否可以请问一下一般训练时长为多少呢？

[quolc]

It took about a week for training a model (450k iter) on single GeForce GTX Titan X GPU. For easier datasets like CUB and flowers (and possibly CelebA), 2 to 3 days (~150k iter) are enough to generate images with decent quality in my experience.

[Jiangzl]

哦哦，好的，还有一个问题就是我在准备数据的时候没有找到preprocess dataset中的脚本文件preprocess.sh，在生成数据集的diename_to_label.txt文件的同时dirname_to_babel_cub.txt文件内存放的是训练数据集的分类文件么？谢谢呐~

[quolc]

We are sorry for forgetting to add preprocess.sh to the repository. I'll fix it. You can use the script on sngan_projection repository for preprocessing ImageNet dataset.

In order to train a GAN on another dataset (e.g., CelebA), please prepare

• DATASET.py (e.g., cub.py) including a dataset class extending chainer.dataset.DatasetMixin and dataset preparation logic
• dirname_to_label_DATASET.txt (e.g., dirname_to_label_cub.txt) describing the correspondence between the names of image directories (class labels) and their integer labels.

It assumes the training images to be stored like

image_root/
- class_A/
    - image1.jpg
    - image2.jpg
    - ...
- class_B/
- ...

there is no assumption on the names of image files.

We are sorry for not having prepared the detailed manual for this, but I think you can reuse cub.py with very small modification (only line 7-9?).

Thank you

[Jiangzl]

好的，谢谢你的细心解答。在更新rpeprocess.sh之后请告知，我将再做尝试。其实目前还有一个疑问是关于数据集，由于人脸数据集celeba的标签只是对于眼鼻嘴的位置信息，因此将所有数据集作为一类，人这个类别进行训练你看是否可行?实在万分感谢~

[quolc]

Since our GAN is class-conditional one, I don't recommend to treat whole the dataset as a single class, or using continuous label like location of facial parts as the conditions. CelebA also provides binary attribute labels, so I think picking up some of them and using the combinations as classes (e.g., "male + eyeglasses", "female + no-eyeglasses ") is straightforward.

P.S. I use Google Translate to read your comment, so please let me know if I misunderstood what you wrote.

[Jiangzl]

好的，你对我的疑问没有理解错误。所以我可以对数据集进行标签组合的分类，比如男性+眼镜、男性+无眼镜、女性+眼镜、女性+无眼镜等多种组合对数据集进行分类然后训练是么？非常感谢你的指导和建议，我看你已更新了preprocess.sh文件，我会重新下载进行尝试。谢谢！

[quolc]

Yes. Firstly classify images based on the chosen set of attributes (e.g., eyeglasses, gender), then train the model using the classified dataset. Note that preprocess.sh is needed only for ImageNet dataset.

[Jiangzl]

所以我无法使用preprocess.sh文件celeba数据集进行处理，需要自主处理和分类数据集，然后生成dirname_to_label.txt和dirname_to_label_cub.txt文件，将分类图片放在对应的分类文件夹下就可以训练了是么？

[Jiangzl]

你好，我下来对数据集celeba进行了分类，修改了dataset中的文件路径，将dirname_to_label.txt文件修改为了celebA_txt，dirname_to_label_cub.txt修改为了celebA_cub.txt，但是在运行cub.py和imagenet_dog_and_cat_enc.py文件时依然出错，显示无法取得值和找不到字典的值，我不知道是否是由于我的文件制作有问题，或者我的路径修改不对，希望得到你的帮助，万分感谢~ 以下是我的文件和错误提示： 这是运行cub.py 这是运行imagenet_dog_and_cat_enc.py celebA.txt文件 celebA_cub.txt文件 图片路径 如果你能知道问题所在希望能给我一些指点，谢谢！

[quolc]

I'm sorry for late reply. I found some problems in your screenshots.

• Names of your image directories do not contain numeric prefix like "001.", so you can omit the prefix from your CelebA.txt like

Female_with_Bangs 1 Female_with_BlackStrightHair 2 ...

(Is Stright typo for Straight? I don't think it to be a serious problem for running the scripts.)

• KeyError 'celeba' occurred probably because of wrong path specification. try

python3.6 cub.py /media/.../datasets/celeba

Thanks

[Jiangzl]

你好，谢谢你对我的解答，有问题的文件是celebA_cub.txt，我按照你给的事例对其进行了修改，可以生成image_list_cub.txt文件，但是在运行imagenet.py, imagenet_dog_and_cat.py, imagenet_dog_and_cat_cub.py文件均出现了相同的错误，我已经对py文件中的已知txt文件进行了修改，或许是我的celebA.txt 文件依然有错误？imagenet的图片名都是以n开头，而我的图片是数字的顺序命名，你看是否是这个原因引起错误？ 非常感谢！

[quolc]

If you want to train your GAN with CelebA dataset, you don't have to run imagenet.py or imagenet_dog_and_cat.py, because they are for training GAN with ImageNet dataset. You can start training using the generated imagelist*.txt.

[Jiangzl]

哦，好的，但是训练语句需要参数文件sn_projection_dog_and_cat.yml,这个文件中的dataset_fn参数需要imagenet_dog_and_cat.py文件，所以我很迷茫~如果直接使用之前cub.py文件生成的image_list_cub.txt文件的话训练的时候不需要config_path的参数了么？

[quolc]

You should also create a new config file by modifying snprojection*.yml to adapt for your new dataset. Changing _datasetfn, _datasetname and dataset path is needed.

[Jiangzl]

我重新生成了新的yml文件，sn_projection_celeba_cub.yml，其中修改了dataset_fn:datasets/cub.py, dataset_name:celebA, path:datasets/image_list_cub.txt,训练报错显示cub模块中没有属性celebA。不知是否又是哪里的修改不对呢？很抱歉浪费你这么多的时间，同时向你对我的帮助表示万分的感谢！

[quolc]

dataset_name should be same as the class name of your dataset class. If you use cub.py without modification, it should be Cub2011Dataset. Since "Cub2011Dataset" is the name of the bird dataset, I recommend you to change it to CelebADataset, for example.

[Jiangzl]

我修改了数据集文件名和dataset_name为CelebADataset,但仍无济于事，我不是是否是因为cub.py是为Cub2011Dataset设置的原因，修改过后依然是module 'cub' has not attribute 'CelebADataset'的错误 T_T

[quolc]

That error should not be present if cub.py contains CelebADataset class. Sorry, I can't precisely grasp the state of your code and identify the problem. Please create a forked repository and upload your code.

[quolc]

I checked your code. I found datasets/cub.py unchanged from the original code. As I wrote before, you should rename the name of dataset class (Cub2011Dataset) to the one specified in the config file (CelebADataset).

[Jiangzl]

你好，我确实已经将参数文件sn_projection_celeba.yml中的dataset_name改为CelebADataset,同时在datasets文件夹下的数据集名称也是CelebADataset,请问是哪里的数据集名称还需要修改呢？

[quolc]

in dataset/cub.py, class Cub2011Dataset(chainer.dataset.DatasetMixin): (line 12) should be changed as class CelebADataset(chainer.dataset.DatasetMixin):.

[Jiangzl]

再次向你对我提供的帮助表示感谢！ 我在修改类名之后进行了尝试，提示找不到图片和'NoneType' object is not iterable，但是图片话是确实存在与数据集文件夹中的，并且不同的尝试提示找不到的图片也不相同，其中也有cub.py文件的错误，是否还是我的celebA.txt文件有错误呢？

[quolc]

I doubt that the command line argument --data_dir should be CelebADataset, not CelebA.

[quolc]

I don't have any idea about the TypeError, but it might be caused by the preceding FileNotFoundError. FileNotFoundError is obvious. Please check the file paths by yourself before posting question...

[Jiangzl]

好的，谢谢，我再看看到底是哪里出错~ :)