Inquiry on Audio Prompts Implementation in musicgen Model

[LiuZH-19]

I am currently exploring the musicgen model and have some questions regarding the application of audio prompts within the model's architecture, particularly in relation to the cross_attention layers:

1. Role of Audio Prompts: Is the audio prompt used as a cross-attention signal within the cross_attention layers of the musicgen model?

   musicgen :
   (transformer): StreamingTransformer(
   (layers): ModuleList(
     (0-47): 48 x StreamingTransformerLayer(
       (self_attn): StreamingMultiheadAttention(
         (out_proj): Linear(in_features=2048, out_features=2048, bias=False)
       )
       (linear1): Linear(in_features=2048, out_features=8192, bias=False)
       (dropout): Dropout(p=0.0, inplace=False)
       (linear2): Linear(in_features=8192, out_features=2048, bias=False)
       (norm1): LayerNorm((2048,), eps=1e-05, elementwise_affine=True)
       (norm2): LayerNorm((2048,), eps=1e-05, elementwise_affine=True)
       (dropout1): Dropout(p=0.0, inplace=False)
       (dropout2): Dropout(p=0.0, inplace=False)
       (layer_scale_1): Identity()
       (layer_scale_2): Identity()
       (cross_attention): StreamingMultiheadAttention(
         (out_proj): Linear(in_features=2048, out_features=2048, bias=False)
       )
       (dropout_cross): Dropout(p=0.0, inplace=False)
       (norm_cross): LayerNorm((2048,), eps=1e-05, elementwise_affine=True)
       (layer_scale_cross): Identity()
     )
   )
   )

   2. Request for Training Code: Could you provide examples or documentation on how to properly use audio prompts as model inputs during training?

   Thank you for your time and assistance.