[5] MACHINE LEARNING - As a developer I want to optimize the preprocessing steps so that the modeling is easier for neural sequence model

[localaization]

Description: We are evaluating the A. Event-based Representation for Music of the document MuseMorphose. We need to check with Jaime this method.

Documentation: From MuseMorphose and example midi file.zip

From This Time with Feeling: Learning Expressive Musical Performance The piano is used in this example

From SEQUENCE-TO-SEQUENCE PIANO TRANSCRIPTION WITH TRANSFORMERS

From MT3: MULTI-TASK MULTITRACK MUSIC TRANSCRIPTION

From Twitter

Piano mapping

Does not univocally map any written note.

Pentagrom mapping

It maps univocally any written note.

Solution

1. Read A. Event-based Representation for Music chapter of the MuseMorphose doc.
2. Take a look at the documentation (in the section above).
3. Identify and check if the variables and tokens are suitable for the pentagrom structure.
4. Map the identified elements which are suitable to the actual pentagrom elements.

Q&A:

1. Check this paper and see if a 7x3 matrix instead of 8x1 make sense and solve the questions: [Piano Genie] (https://arxiv.org/pdf/1810.05246.pdf)

Definition of Done: To have an agreement whether or not section "A. Event-based Representation for Music" from MuseMorphose is a good approximation.

[localaization]

From This Time with Feeling: Learning Expressive Musical Performance

[TuWebO]

There is a lot of information that we could potentially get from: https://magenta.tensorflow.org/

I didn't know about it until now. I will take note about some interesting links: https://magenta.tensorflow.org/ddsp-vst-blog https://openreview.net/forum?id=B1x1ma4tDr&noteId=lcOwh022Vta https://github.com/magenta/mt3 https://archives.ismir.net/ismir2021/paper/000030.pdf https://openreview.net/pdf?id=iMSjopcOn0p

[TuWebO]

I've updated the documentation with the latest articles I've seen. I will meet Jaime tomorrow and leave this task unassigned for now.

[localaization]

Jaime and myself have been talking about this issue. We have red the A. Event-based Representation for Music from the document MuseMorphose, also another papers and pianogenie project. We think the pentagrom system could have one main advantage:

• It can univocally map any note in the matrix to any written note in the stave, something that does not happen with the piano or any other music controller so far.

[localaization]

@isamu-isozaki this is the task, if you think it is solved feel free to mark it as done, otherwise assign it to me and back to in progress.

[isamu-isozaki]

@localaization yup nice work! I think it's done. I think for machine learning, we just go through your format -> remi -> model which should do the job. Possibly midi in between