Why the original feature is called ``sparse features`` and the mean and std features are called ``target_encodings``?

[Yonggie]

Hi, we see in the data preprocessing you joined mean&std *4 to the original X, and in the transformer model, you named the original features as sparse_features and the mean&stds as target feautres. I'm having hard time reading this, could you kindly explain why is it?

class CustomTransformer_v3(nn.Module):  # mean + std
    def __init__(self, num_features, num_labels, d_model=128, num_heads=8, num_layers=6, dropout=0.3):
        super(CustomTransformer_v3, self).__init__()
        self.num_target_encodings = 18211 * 4
        self.num_sparse_features = num_features - self.num_target_encodings

        self.sparse_feature_embedding = nn.Linear(self.num_sparse_features, d_model)
        self.target_encoding_embedding = nn.Linear(self.num_target_encodings, d_model)
        # .......
    def forward(self, x):
        sparse_features = x[:, :self.num_sparse_features]
        target_encodings = x[:, self.num_sparse_features:]

        sparse_features = self.sparse_feature_embedding(sparse_features)
        target_encodings = self.target_encoding_embedding(target_encodings)
        #.....

[Eliorkalfon]

Those dense aka target encoding features were derived from the target values. Target encoding is a method of utilizing prior knowledge to enhance model performance.