How to get the interesting results in Fig. 16?

[Sharpiless]

Awesome work! But I wonder how to get the awesome shape generation via ViT feature manipulation as shown in Fig. 16? Could you provide more detailed instructions? Thanks!

[Zhengxinyang]

Thank you for your interest in our work! As described in the paper, we simply swap ViT patch features of two existing sketches.

you can use the following code to swap features.

1. For swapping bottom and top features:

sketch_c = np.concatenate([sketch_c_1[0:129], sketch_c_2[129:]],0)

2. For swapping left and right features:

def get_blend_feature(feature_1, feature_2):
    res = [feature_1[0:1]]
    for i in range(16):
        res.extend([feature_1[i*16+1 :i*16+9], feature_2[i*16+9:i*16+17]])
    return np.concatenate(res,0)

sketch_c = get_blend_feature(sketch_c_1, sketch_c_2)

Here, sketch_c_1 and sketch_c_2 are features extracted from the ViT, and sketch_c is the condition for the sample_with_sketch function.

[Sharpiless]

Thanks!