Tasks to do with Hernan

[behzadhaki]

1. Tasks before VAE development

---

1A) Sigmoid for Vel/Timing

• [ ] Use sigmoid for velocity/micro-timing and use BCE loss instead of MSE
• [ ] Retrain the best four models (without any h-param sweeps)

1B) Account for imbalance in training data

• [ ] Add penalty to loss based on genre and per voice hit distributions
• [ ] Retrain the best four models (without any h-param sweeps)

1C) Mixed Gridline

• [ ] Use mixed gridline (triplet mixed with 16th note)
• [ ] Ensure GrooveEvaluator works with the new mixed gridline
• [ ] Retrain the best four models (without any h-param sweeps)

1D) Learning Rate

• [ ] Experiment with adaptive learning rate

2. Tasks for VAE development

---

2A) Implement a general transformer VAE:

• [ ] Figure out the code for the VAE loss in magenta source code (KL Loss and Free Bits Implementation)
• [ ] Develop a generalized Variational Auto-Encoder Transformer (Encoder-Encoder and Encoder-Decoder)

2AA) Training process familiarization

• [ ] Study the helpers/BasicGrooveTransformer_train.py code for training utils/guides

2B) Use the implementation to do h-param sweeps for two cases:

• [ ] VAE for Groove2Groove
• [ ] VAE for Groove2Drum

[behzadhaki]

Tasks for @hdperezn for the next few days:

• [x] Use your university email to get the full version of pyCharm
• [x] Study the papers related to our work (most importantly the AIMC paper from last week) https://zenodo.org/record/7088343 https://youtu.be/XOTksswM_4U

Transformer Architecture for Generating Short Drum Loops Given a Performed Monotonic Groove.pdf

<<Ignore this one for now!>>

Infilling_Audio_Drum_Loops_with_Transformer_Neural_Networks.pdf

• [x] Focus of 2A.1. (VAE loss in magenta source code)

• [x] Review the notes on setting up the cluster (don't set up yet, this will change a bit, just familiarize with the concepts as much as possible) https://github.com/behzadhaki/ENVIRONMENT-SETUP-TUTORIALS/blob/main/tutorials/UPF_HPC_Tutorial.md

• [x] Review the notes on transformers https://github.com/behzadhaki/Transformer_Tutorials/blob/main/Tutorial%203%20-%20Transformer%20Decoder%20and%20Positional%20Encoding/Attention%20-%20Condensed%20Notes%20Session%203.pdf

[behzadhaki]

Tasks for @hdperezn for Oct 6-13

VAE development

2A) Implement a general transformer VAE:

• [ ] Figure out the code for the VAE loss in magenta source code (KL Loss and Free Bits Implementation)
• [ ] Develop a generalized Variational Auto-Encoder Transformer (Encoder-Encoder and Encoder-Decoder)

2AA) Training process familiarization

• [ ] Study the helpers/BasicGrooveTransformer_train.py code for training utils/guides