Set: Set-to-Vector, Vector-to-set

[JisuHann]

Set-to-Vector

1. Deep Sets(NIPS2017)
2. Learning Functions over Sets via Permutation Adversarial Networks(ACCV2020)
3. Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks(ICML2019)

Vector-to-Set

1. DSPN: Deep Set Prediction Networks (NIPS2019)
2. TSPN: Conditional Set Generation with Transformers (ICMLW2020)

[JisuHann]

Deep Sets(NIPS2017)

paper

Idea

• Objective functions defined on sets that are invariant to permutations
  • Function acting on sets must be permutation invariant to the order of objects in the set

Deep Sets

[JisuHann]

Set Transformer: A Framework for Attention-based Permutation-Invariant Neural Networks(ICML2019)

paper

Idea

• Set structured data - permutation invariant set of solution
• set-input problem
  • Critical requirements
    • permutation invariant: the output of the model should not change under any permutation of the elements in the input set
    • able to process input set of any size
  • Application (set-input problem)
    • multiple instance learning: (set of instances) -> (label for the entire set)
    • 3D shape recognition, sequence ordering
    • various set operations

      Model

• Encoder-decoder architecture that rely on attention mechanism
  • Attention scheme inspired by inducing point methods from sparse Gaussian process literature
    • self-attention
      • every element in an input set -> naturally encode pairwise- or higher-order interactions btw elements in the set
      • aggregate feature: each cluster center X heavily depends its location relative to the other clusters
• Encoder
• Decoder

tasks

1. Maximum Value Regression
2. Counting Unique Characters
3. Amortized Clustering with Mixture of Gaussinas
4. Set anomaly detection
5. Point cloud classification

[JisuHann]

Learning Functions over Sets via Permutation Adversarial Networks(ACCV2020)