Multi-labels?

[Nedae]

Is there any way to define multilabels for the images in the dataset? I already saw other issues such as #53, #59, and #147 but no body provided the expected answer to those! An example is when an image has more than one set of labels, such as "old/young", "happy/sad/neutral", and "man/woman". For instance, something like below for young happy man?

{
    "labels": 
        [
             ["00000/img00000000.png", "young", "happy", "male"])
        ]
}

Any part of the code to modify to support such a behaviour?

[Nedae]

From https://github.com/NVlabs/stylegan2-ada-pytorch/blob/d4b2afe9c27e3c305b721bc886d2cb5229458eba/dataset_tool.py#L62 it seems only single labels are supported!

Also, what #147 asks seems not possible based on https://github.com/NVlabs/stylegan2-ada-pytorch/blob/d4b2afe9c27e3c305b721bc886d2cb5229458eba/dataset_tool.py#L160.

Any feedback on whether multilabel conditional training is possible in this repo is appreciated!

[thusinh1969]

Gents,

Even for the multi-class problem, have you been able to produce non-streaks results (forget faces, we have enough of so cropped-focused face dataset !!!) ? I have NEVER been able to produce a good result at all, regardless of the size of the dataset (few dozen thousand to hundred thousand images) or kimg (30-50,000 kimg). Too many weird artifacts on generated images.

Any hint? Would R1 be a real reason that we have to seek to find that value? I used 1, 2, 5, 10. Same issue.

Steve

[GHBTM]

@thusinh1969 I have had much improve results with using gamma values (R1 regularization) in accordance with Appendix D (page 33) of the ADA paper... for a 32x32 grid had to drop gamma to 0.05 to 0.005, which scales almost linearly with resolution. Check the paper out. Tweaking learning rates should also help, but I don't (yet) have experience there.

[dongyun-kim-arch]

I am also curious about multilabel...

[dongyun-kim-arch]

@Nedae Hi Nedae, did you manage to work multi-labels styleGAN? I am working on the stuff, but stuck now..

[49xxy]

I am also curious about multilabel...

Have you got the solution？

[nom57]

I have got excellent results for my project on a small dataset of 278 images with consistent art style

the project would however benefit a lot from multi labels , would be cool if someone could make support for this

[49xxy]

我的项目在一个包含 278 张图像的小数据集上获得了出色的结果，并且具有一致的艺术风格

然而，该项目将从多标签中受益匪浅，如果有人可以对此提供支持会很酷

Hi！How do you set the multi-label of your dataset？

[GilesBathgate]

Support for Multi-labels

TL;DR

It's already supported, use for example [0,1,1] inside dataset.json for label values in place of 1

Implementation Details

If you look at dataset.py https://github.com/NVlabs/stylegan2-ada-pytorch/blob/d4b2afe9c27e3c305b721bc886d2cb5229458eba/training/dataset.py#L233

It can be seen that stylegan-ada-pytorch supports raw float32 labels when the dimensionality of the label value is 2 (i.e a python list [1,2] instead of a single number 1)

In normal usage, if the dimensionality is 1, the categories are one-hot encoded for example if you have a class of fruit we could have three values: APPLE=0, ORANGE=1, PEAR=2 the one-hot encoding of these values would be:

0 = [0,0,1]
1 = [0,1,0]
2 = [1,0,0]

If you want to encode more than one category class, you can specify the raw label directly in the dataset.json. It can be encoded using any categorical encoding needed. For example, if the three classes are happiness, age, and gender a suitable encoding might be:

sad & young & male =     [0,1,0]
happy & young & male =   [1,1,0]
happy & young & female = [1,1,1]

however other categorical encodings may work better.

To put these in the JSON you simply put a list in place of the single number:

{
    "labels": 
        [
             ["00000/img00000000.png", [0,1,0]]
        ]
}

I'd imagine all the raw labels need to have the same number of elements.

To use the raw labels with generate.py the following patch can be applied:

--- a/generate.py
+++ b/generate.py
@@ -40,6 +40,7 @@ def num_range(s: str) -> List[int]:
 @click.option('--seeds', type=num_range, help='List of random seeds')
 @click.option('--trunc', 'truncation_psi', type=float, help='Truncation psi', default=1, show_default=True)
 @click.option('--class', 'class_idx', type=int, help='Class label (unconditional if not specified)')
+@click.option('--label', 'raw_label', type=num_range, help='Raw label')
 @click.option('--noise-mode', help='Noise mode', type=click.Choice(['const', 'random', 'none']), default='const', show_default=True)
 @click.option('--projected-w', help='Projection result file', type=str, metavar='FILE')
 @click.option('--outdir', help='Where to save the output images', type=str, required=True, metavar='DIR')
@@ -51,6 +52,7 @@ def generate_images(
     noise_mode: str,
     outdir: str,
     class_idx: Optional[int],
+    raw_label: Optional[List[int]],
     projected_w: Optional[str]
 ):
     """Generate images using pretrained network pickle.
@@ -105,9 +107,12 @@ def generate_images(
     # Labels.
     label = torch.zeros([1, G.c_dim], device=device)
     if G.c_dim != 0:
-        if class_idx is None:
-            ctx.fail('Must specify class label with --class when using a conditional network')
-        label[:, class_idx] = 1
+        if class_idx is None and raw_label is None:
+            ctx.fail('Must specify class label with --class or --label when using a conditional network')
+        if class_idx is not None:
+            label[:, class_idx] = 1
+        if raw_label is not None:
+            label = torch.unsqueeze(torch.tensor(raw_label,device=device),0)
     else:
         if class_idx is not None:
             print ('warn: --class=lbl ignored when running on an unconditional network')

Then the raw label argument could be specified as so: --label=0,1,0