search

Audiveris / audiveris

Latest generation of Audiveris OMR engine
https://audiveris.github.io/audiveris
GNU Affero General Public License v3.0
1.47k stars 218 forks source link

Drum notation? #33

Open ghost opened 6 years ago

ghost commented 6 years ago

Hello, wonderful developers of Audiveris!

I try to scan my drums sheets from my music school but after a processing, the hi-hat notes (noted with crosses), do not appear on the resulting file; am I missing some configuration? I tried to search for "drums" in your wiki and documentation, is it an expected behavior, a limitation or an error I did?

Thanks for your help :)

~SPJ

hbitteur commented 6 years ago

I'm afraid Audiveris does not recognize drum sheets. Could you point to a typical example of such drum notation, so that I can confirm?

chenlung commented 6 years ago

Hi Hervé,

Wikipedia has an article, but there isn't a universal standard for drum notation (noteheads, voices, etc), so I would suggest the ability to define a drumset (i.e. MuseScore).

I've attached two examples of full scores from different books (same typesetting software and transcribing company, I believe) to give an idea: key to percussion notation example 1 excerpt example 1 key to percussion notation example 2 excerpt example 2

They use a double sharp sign for some note heads, but Elaine Gould ('Behind Bars', page 11) discourages this.

hbitteur commented 6 years ago

Wow! It's a whole bunch of notations! Some are very similar to (non-drum) standard notations, some are specific. From the point of view of current Audiveris possibilities, we would need to handle specific note heads (reading your top diagram: open hit-hat, crash cymbal, ride cymbal), plus some symbols like tremolos (see your buzz strokes).

On top of that, we'd have to check how MusicXML export format can support all these drum notations. Please have a look at MusicXML 3.1

My guts feeling, based on Audiveris architecture, is that this is technically feasible. However it would be a significant task, and we lack time and human resources right now. Perhaps some external contribution could focus on this development.

When you say "there isn't a universal standard for drum notation (noteheads, voices, etc), so I would suggest the ability to define a drumset", I hear a small voice saying "beware, this sounds like a niche requirement". Before investing large efforts into this, make sure the need is crucial and the "standard" widely shared.

ghost commented 6 years ago

I couldn't take the time to point out those problems, but I'm aware there is no standard notation for it and that it could take a large amount of work to implement it. My original question is therefore answered, it's indeed a limitation, but given the state of the art concerning drum notation(s), I also understand perfectly that it's not exactly an urgent issue :) I'll close the issue. Thanks again!

chenlung commented 6 years ago

I was originally inspired by certain sophisticated published transcriptions (hidden empty staves, tablature, drums, etc), so I feel this is an area I could assist (though I am not a coder). Such material has helped improve the scoring tool I use - MuseScore.

@stephanepjeannin: Regardless of the scale of challenge, I still think it's a valid issue and we should leave it open.

@hbitteur: It might be a matter of perspective, but defining drumsets may not be as niche as you think.🙂 May I email you with some private examples? I will also post some public ones so they can be discussed, etc.

ghost commented 6 years ago

Well, I closed the issue because I got my answer... However, I agree that the limitation could be addressed. Therefore I reopen it :)

hbitteur commented 6 years ago

@chenlung You can use my private mail (herve dot bitteur at audiveris dot org) for any private example

hallvors commented 3 years ago

I need these funny triple line things - might be called "buzz strokes" per the above? Skjermbilete_2021-03-25_23-19-11 This score also uses "square standing on tip" noteheads described as ride cymbals above, though it is used for the crash cymbals part.

I see MusicXML has things like <note><notehead>X</notehead></note> to permit things like the percussion x notehead. However, I don't know if this handles all the special things for percussion notation. I tried to cheat and created a MuseScore file with some percussion instruments and odd note heads, but it failed silently when I tried to export to MusicXML so I guess either the standard or the implementation is not quite ready.

hallvors commented 3 years ago

Well, actually I already have some MusicXML with drum notation around.. Drum Set.xml.zip

chenlung commented 3 years ago

@hallvors I couldn't open your file in MuseScore 3.6.2 (crashes).

I've got a mscz with several drumsets if you want to experiment with it?

hallvors commented 3 years ago

Ops.. Probably my mistake, it is extracted from a bigger MusicXML score, but it does render fine with OSMD (Opensheetmusicdisplay JS library).

I can't really speak for any Audiveris developers here since I'm just a random user :) But if Musescore exports that drum set file nicely to MusicXML I think it might be useful to them as an example of percussion notation in MusicXML.

Jojo-Schmitz commented 3 years ago

MuseScore crashes on that xml file, in a Debug build with a failed assertion somewhere deep in Qt: Fatal: ASSERT failure in QList::at: "index out of range", file C:/Qt/5.15.2/mingw81_64/include/QtCore/qlist.h, line 571

hbitteur commented 2 years ago

FYI, Audiveris template matching was recently extended to support cross heads (x-shaped heads). I'm putting this here to help contributors (Brian...) on drum notation.

image

The image above shows how we can interactively apply a head-template for a given shape (here shape NOTEHEAD_CROSS, with a stem located on the right head side) on a given score location. This was meant for Audiveris developer to visually check how the template worked (or not).

To play with this, you will need: to

In the Templates tab, use mouse right button to position the template at a desired location. Then read the matching result in the template board on the right column (where you can also select which template to use). Mouse left button is used to read the distance from mouse location to the nearest template foreground point.

OK, that's a bit complex, but this feature was never meant for the end-user :-)

Then browse Audiveris code, beginning with classes:

They should be good starting points. /Hervé

brian-math commented 2 years ago

@hbitteur thanks for the welcome and the tips for getting started! As Hervé implied, I want to get involved with improving Audiveris' recognition of drum notation: specifically, 5-line unpitched percussion scores. I am brand new to GitHub, so please forgive and correct any mistakes like posting under the wrong heading of the forum.

Let me first address the issue of MusicXML. It seems that MusicXML 4.0 already supports all the needed elements of unpitched percussion for Audiveris export; see https://www.w3.org/2021/06/musicxml40/tutorial/percussion/ for a good introduction. In particular, it supports encoding of:

Admittedly there are some choices to make with that last point: in the example posted near the top of this thread by chenlung on Jan 17 2018, an x on the space above the score is depicted as an open hi-hat, but many programs that create drum score notation (and many drummers) would use that symbol for a closed hi-hat. Ultimately Audiveris should probably provide a choice to the user among several common schemes for assigning MIDI sounds to each (notehead, score position) pair. But initially for development purposes, it makes sense to choose one of them; I would propose the one used by MuseScore.

Maybe that's enough for now. After I've had a chance to try Hervé's suggestions from the previous post, I'll probably have some questions. And at some point I'll post an overview of what I think will need to be done, in terms of new or improved code development, shape recognition, training, etc., for Audiveris to support unpitched 5-line percussion scores. [GitHub newbie question: should all that be posted here in Issue #33, or somewhere else?]

chenlung commented 2 years ago

@hbitteur thanks for the welcome and the tips for getting started! As Hervé implied, I want to get involved with improving Audiveris' recognition of drum notation: specifically, 5-line unhitched percussion scores.

Hi Brian,

I have a score which you may use for testing (specially designed for this). Can I email it to you?

brian-math commented 2 years ago

@chenlung, thanks for the offer. I have dozens if not hundreds of PDF drum scores already, from MuseScore and various other sources. Is there something special about yours? I’m not sure how email works with GitHub—I got an email copy of your posting, does that mean if you post your score I’ll get it by email?

hbitteur commented 2 years ago

@brian-math

[GitHub newbie question: should all that be posted here in Issue https://github.com/Audiveris/audiveris/issues/33, or somewhere else?]

Yes you can use this location for questions, suggestions, etc... When your modifications get more concrete, it will be time to use a PR (pull request) to include them.

Since you are a GitHub "newbie", to get up and running on Audiveris code, do the following from the command line level in a local folder of your choice:

# Download Audiveris code and put it by default in a sub-folder named audiveris
$ git clone https://github.com/Audiveris/audiveris.git

# Move to the just created/populated sub-folder
$ cd audiveris

# Select development branch
$ git checkout development

# Show current status
$ git status

Before that, you'll need to install Git and Gradle tools, plus Java JDK 17 and your favorite IDE. What's your OS?

chenlung commented 2 years ago

@chenlung, thanks for the offer. I have dozens if not hundreds of PDF drum scores already, from MuseScore and various other sources. Is there something special about yours? I’m not sure how email works with GitHub—I got an email copy of your posting, does that mean if you post your score I’ll get it by email?

@brian-math You might be covered overall, but I think the score will still serve you in some form, even beyond five-line drumsets (should you get to those).

And I think that's just a notification, so feel free to drop me a line.

brian-math commented 2 years ago

Hervé, I want to say that the existing instructions on the site for installing Audiveris were very clear, and I had almost no problems getting it up and running a week ago. The only problems arose regarding the tesseract language files. It seemed that the default was to download version 4.1.0 of those files, whereas it was (exceedingly!) clear from the instructions that only v. 3.0.4 would work. When I first ran Audiveris, I got an error "Could not initialize Tesseract with language deu" [or eng or fra or ita] and suggesting that the environment variable TESSDATA_PREFIX needed to be set. Eventually I discovered that the language files (on a Mac) are saved in /opt/local/share/tessdata/xxx.traindata (where xxx is deu, eng, fra, ita, etc), and after adding that environment variable definition to my .login file, the error message went away. But I still had to figure out how to find and download the tesseract 3.0.4 language files and substitute them for the 4.1.0 files in the aforementioned directory. Some better explanation of this process might have helped. But otherwise it was no problem! Mac OS Mojave 10.14.6.

hbitteur commented 2 years ago

I suppose you are using Audiveris development branch. And so, you are living on the "bleeding edge" of this software, taking advantage (?) of every modification made on this branch. One of them was the migration from Tesseract 3.x to Tesseract 4.x used in "legacy" mode, which means still using the old (3.x) Tesseract data files.

During this switch, I discovered by chance that TESSDATA_PREFIX environment variable that was supposed (in 3.x) to point to the parent folder of tessdata folder is now supposed in (4.x) to point directly to tessdata folder. This might explain your problem.

This needs to be documented before the next release is published...

hbitteur commented 2 years ago

Template generation

In Audiveris 5.x, heads are recognized by template matching while most of the other fixed-shape symbols are handled by a trained neural network. This is likely to change in 6.x where the new patch classifier should be able to handle heads as well as the other symbols. But for the time being, we need to extend templates so that even the drum-related heads get recognized.

For this, the key class is org.audiveris.omr.image.TemplateFactory. It works with a specific free true-type font named "MusicalSymbols" which does not need to be installed on the user machine, since it is provided in Audiveris res folder (res/MusicalSymbols.ttf file). See for example https://fonts2u.com/musicalsymbols.font to have a look at this font.

With this font, in TemplateFactory we simply specify the code to use for every specific head shape. For example:

Drum notation will require a few additional codes. A quick search gave:

But I could not find any circled fat cross head (crash cymbal) in this font. What can we do?

  1. Use another musical font? This means 1/ finding out a new suitable font and 2/ checking and perhaps modifying the font codes used in Audiveris software. Fortunately, these codes are kept hidden in 2 key classes, TemplateFactory class to get the images needed to generate templates and Symbols class which handles the symbols display on Audiveris images. However, besides the font code, the program sometimes needs additional information such as the x and y shift of a symbol with respect to font origin. And this involves a bunch of classes.
  2. Combine 2 font symbols, one for the fat cross and another one for the circle around it. For example code 111, a small circle which unfortunately is significantly smaller than the cross, so it should be picked up from a scaled variant of the font. A bit complex, but doable. I prefer this solution because it doesn't break anything.

Any other idea?

hbitteur commented 2 years ago

image

image

image

image

You can play with this small tool integrated in Audiveris. It is launched by Debug | Launch Symbols Ripper, provided that you have selected the debug features in the Tools | Advanced Topics Mind the fact that this tool allows to select a font among the ones installed on your machine. So MusicalSymbols font must be installed in that case.

brian-math commented 2 years ago

@hbitter Thanks for this detailed information about the music font used for note heads; it is very interesting.

I would suggest not to worry too much right now about the circled x. It seems to me that ~95% of drum notation uses only standard oval and x note heads. The next most common head seems to be a diamond. I have never encountered a circle-x used for a crash cymbal; rather, the main crash is normally an x on the first ledger line above the staff (or perhaps the space above that, for a 2nd crash cymbal). I have seen a circle-x used for an open hi-hat cymbal, but there are at least two other common notations in use for the open hi-hat, and we can probably reasonably translate to one of them if we can recognize an x-in-a-circle glyph. This should be good enough until we get to version 6.x. But if not, I like your idea of combining two existing symbols from the font.

By the way, as mentioned at the bottom of @chenlung 's Jan 17 2018 post, the double sharp symbol should not be used for drum cymbal notation, even though it certainly appears in some scores (probably because it was an existing music notation symbol to which the transcriber had easy access). So I think we should stick to the Code 192 x and not use the Code 220 double sharp.

I've spent a good deal of today preparing 1) a detailed analysis of how Audiveris performs, step by step, when given a drum score as input, and 2) an outline of a proposed set of tasks to get Audiveris able to recognize 5-line drum scores. I'll post those later this evening (US time).

chenlung commented 2 years ago

I have never encountered a circle-x used for a crash cymbal; rather, the main crash is normally an x on the first ledger line above the staff (or perhaps the space above that, for a 2nd crash cymbal).

I think MuseScore has the circle-x?

Barnes Music Engraving used Amadeus Music Software for these transcriptions, by the way.

brian-math commented 2 years ago

Yes, MuseScore allows you to assign a circle-x to a notehead (and many other shapes, including the Aiken 7-shape do re mi...). But it is not used in their default Drumset notation.

chenlung commented 2 years ago

But it is not used in their default Drumset notation.

I can't speak for the developers, nor am I fully aware of on-going developments (will let someone else answer properly), but I don't think the drumsets of previous versions were always properly designed; I actually have a few custom ones (one for emulating the published scores, and another containing my preferences).

Anyway, I'll trust your judgement on this (I'm a non-dev!). One thing that may help is having a conversation with the personnel at MuseScore (or indeed others), influencing the decisions of all parties for the better?

brian-math commented 2 years ago

Here are some results of a trial run using Audiveris 5.3 (Development) on a 5-line drum score (original PDF score attached).

Remarks on score:

But includes some challenges:

Executive Summary of Audiveris results: Audiveris did a surprisingly good job of recognizing much of this score! Things that did not go well:

============================================================ Here are my detailed observations after each Audiveris step. (My Audiveris settings are listed at the very end.)

GRID All barline inters detected (blue), even the double thick/thin ones at repeats. But in the “section-only” selection mode, showing outlines of contiguous black pixels, the drum clef symbols all have portions of the staff lines amalgamated with them. Also the stems shared by a regular oval note and an x note look different from most others–segmented and filled in black.

HEADERS No header clef detected in any staff! The 2/4 time signature seems to be considered most likely 2/2 by the glyph classifier (but no inter assigned). The 4/4 time signature seems to be ignored completely. The header at the beginning of staff#2 is mistakenly classified as an F_CLEF. This is attached to a glyph consisting of the drum clef, double bar line, and parts of the 3 middle staff lines. When selecting the drum clef at the beginning of each staff in “glyph + inter” mode, typically it is in six pieces instead of the expected two (above, below, and on the middle staff line; left and right parts, instead of just left and right). But sometimes two of these pieces are connected by a piece of the 2nd staff line (e.g. the left and right bottom parts in staff#3).

[Should I select each drum clef and assign PERCUSSION_CLEF from the Glyph Classifier list (it always has the highest grade, except in staff#2) before proceeding with more steps?]

[I unassigned the F_CLEF from the header of staff#2, but not sure how to break apart the drum clef from the double barline in the compound glyph there.]

STEM_SEEDS I guess this worked. Clicking on a few random stems highlights them in black, and they seem to have approximately the correct length and thickness, including the long stems attached to both an x and an oval note head. But they don’t show up as inters; is this expected behavior?

BEAMS Seems to work almost perfectly–all the beams and beam hooks appear to be correctly highlighted in pink. The middle of an “s” in the song title is incorrectly identified as a beam hook.

LEDGERS Most ledger lines are correctly recognized, but not the three supporting x note heads in staff#3, for some reason. Selecting them reveals no inter, and the Glyph Classifier thinks they are tenuto marks with very high grade. (I just noticed that the first and second ending horizontal and vertical lines became selectable at some point, but they don’t seem to be recognized as any sort of inter. The two horizontal top lines get selected as one object.)

HEADS Pink stuff: The x noteheads almost all seem to be correctly recognized as NOTEHEAD_CROSS inters. The ordinary solid oval noteheads all seem to generate at least two inters–a NOTEHEAD_BLACK, plus one or more NOTEHEAD_CROSS inters at or near the same location. For example, selecting the first notehead of bar 47, bottom space, and right-clicking, reveals 6 NOTEHEAD inters: two _BLACK, three _CROSS, and one _VOID! The open half note x at the end is designated as a NOTEHEAD_CROSS; I’m assuming there is not yet a NOTEHEAD_CROSS_VOID shape defined. There are a lot of spurious NOTEHEAD_CROSS inters on the time signatures and drum clefs (and, as mentioned, each NOTEHEAD_BLACK).

Green stuff: There are a lot of spurious WHOLE_NOTE inters: several on each drum clef (combined with a bunch of NOTEHEAD_CROSS inters); a couple near the 4/4 time signature; and one in the midst of the second x-head triplet in the 2nd-last bar.

STEMS Dark green note heads and beams, with light green connections between most stems and at least one of the corresponding note heads (both _BLACK and _CROSS when these overlap), and beams when relevant. Three slanted beams in staff#3 are still pink, where the ledger line of one of the attached notes was missed in LEDGERS step. Same for the erroneous beam hook in the middle of the “s” in the title.

REDUCTION Most of the overlapping symbols from the HEADS step are now gone. Unfortunately, all the x heads which share a stem with an ordinary oval note are also gone. This is incorrect. The grace notes in the last bar are incorrectly regarded as x heads. [I should have turned on support for Small Black Note Heads in Book Parameters.] The drum clefs are now regarded as one whole note on top of another, except in staff#2. The beams and beam hook that were pink after the STEMS step are gone (incorrectly, for the three in staff#3). Both time signatures are unrecognized.

CUE_BEAMS

TEXTS The header text is all recognized correctly. The tempo setting “quarter note = 123” has been rendered as the SENTENCE “J: 123”. All but two of the bar numbers have been recognized correctly. The 1. in the first ending has been transcribed as “II.” and the 2. in the second ending has become “Hz.” The triplets have been rendered as a SENTENCE of 3’s, long dashes, underscores, colons, and quotes. The horizontal lines of the first and second endings are now rendered as a long WORD of underscores. (This probably happened here, but I didn’t notice it until later.)

MEASURES 52 raw measures seems correct.

CHORDS Turning on Show chord IDs, it appears as if every note has an individual chord number. Probably this is not correct. Maybe because of the missed time signatures early on?

CURVES The log reports 142 slurs and 34 segments. Evidently the beams which were deleted because of missing ledger lines were turned into SLUR_ABOVEs and TUPLET_THREEs. There are numerous spurious slurs attached to single or consecutive x notes (some of these note heads were already deleted at the REDUCTION step).

SYMBOLS This apparently picked up most of the stems and flags. One grace note acquired a flag but the other didn’t. The REPEAT_DOTs are recognized, except at the beginning of staff#2. The stems whose beams were lost because of the missing ledgers have each become part of a TUPLET_THREE. The accents in the first staff were never recognized. There are a few spurious (faint) natural signs, slurs, dots, etc., which I just noticed, but these may have been introduced in an earlier step.

LINKS Some of the deleted x note heads have become augmentation dots. Most of the spurious natural signs disappeared, along with some other spurious marks.

RHYTHMS The 1st and 2nd endings have been fixed. The triplets still aren’t correct.

PAGE A bunch of errors about “No target duration for measures …, please check time signatures.” [I should have fixed the time signatures after the HEADERS step.]

============================================================ Here is some info about my Audiveris settings when I ran the above test:

% git status On branch development Your branch is up to date with 'origin/development'. nothing to commit, working tree clean

Audiveris settings Selected all advanced topics in Tools | Advanced Topics In Tools | Options, set the constant DistancesBuilder.displayTemplates to true Book | Set Book Parameters:

Twist_And_Shout_TheBeatles_musescore.pdf

brian-math commented 2 years ago

Partly based on the above trial run, here is a proposed list of steps for adapting Audiveris to recognize and handle 5-line unhitched percussion scores. Comments and suggestions for improvement of this list are of course welcomed.

Recognize drum clef

Recognize common drum note heads

Chords

Voices

Other special drum notation to recognize

Export to MusicXML 4.0

Some glyphs we should never see in a drum score: key signatures or accidentals. Maybe we can make the grade of these inters be very low when the user selects drum mode? Or (in a score with other pitched instrument parts) at least low score on the drum staff.

brian-math commented 2 years ago

Anybody know why this single-line percussion score does not process correctly on Audiveris with the Book parameter: Support for percussion staves (1 line) turned ON? I get an error at the SCALE step: "SCALE step failed. Too large interline value: 299 pixels. This sheet does not seem to contain staff lines."

Can't_Take_My_Eyes_Off_of_You-Hand_Clap.pdf

hbitteur commented 2 years ago

Case of Twist and Shout:

None of the percussion clefs were recognized.

But in the “section-only” selection mode, showing outlines of contiguous black pixels, the drum clef symbols all have portions of the staff lines amalgamated with them.

There is a bug, due to recent fix Reduced minRunLength for LinesRetriever in GRID step

Temporary workaround is to set constant LinesRetriever.minRunLength in Tools | Options back to 1 (I had set it to 0.25). Then the sections will get correctly erased and all the drum clefs recognized.

Why some staff sections stay stuck to the staff lines when minRunLength is set to a very low value is the bug I need to fix ASAP. Sorry for that, I'm on several urgent tasks at the same time...

hbitteur commented 2 years ago

[From now on, I'll assume that the workaround in my message above is used.]

HEADERS step deals only with header part of staves, trying very hard at the beginning of every staff to recognize a "CKT" sequence made of Clef + possible Key + possible Time, all this right after a starting barline (more reliable that line pixels). Hence:

To fix this manually:

STEM_SEEDS step: These are just glyph seeds, no Inters are created yet.

BEAMS step: A false beam was detected in the title. No problem, it won't survive long, because it lacks stems on its side(s).

hbitteur commented 2 years ago

LEDGERS step: I don't know why those above staff 3 were not recognized. Beside shape checks there are other tests on location and distribution of vertical distance from next staff/ledger line. I suspect the distribution criteria did not correctly work because this is a synthetic (too perfect) image. To be debugged.

hbitteur commented 2 years ago

HEADS step: This produces head inter as candidates. Additional information will come later and impact their final contextual grade. Notice that the missing ledgers on staff 3 prevent the detection of heads.

STEMS step: Most glyphs from STEM_SEEDS step are transformed into real Inter instances. Still on staff 3, no head was detected around the missing ledgers, and since a stem needs a head, we are missing stems as well. The related beams remain in red, because they lack stems on one side...

REDUCTION step: Here a lot of checks are run, exclusions are detected, contextual grades are computed, etc...

TEXTS step: The tempo setting (quarter note = ...) currently can't be recognized by Tesseract because the quarter note symbol is not a text symbol. This is a known issue on Audiveris forum. There should be a way to add it to Tesseract input, if someone has the answer?

CHORDS step: Except for whole notes of course, it's "one stem, one chord", regardless of the number of heads left on the stem. But remember that Xs and ovals are (wrongly) considered as mutually exclusive on the same stem...

SYMBOLS step: Many candidate Inters get created.

LINKS step: This is another reduction step, hence many false symbols get discarded thanks to links (or lack of links).

RHYTHMS step: Measures in pink: consequence of missing beams or excluded X's mistaken for augmentation dots.

All in all:

On my todo list:

Then, we'll get back to the same input file and check the new behavior.

hbitteur commented 2 years ago

Point 1 (fix on minRunLength bug fix) is now done via commit d666582361a4541177db091e39286ec2e6a235d9. Please make a "git pull --all" from development branch.

hbitteur commented 2 years ago

Ledgers problem now fixed by commit 8cf38c20bb3c47a00a042358b127b4f58f20a300 Please pull.

hbitteur commented 2 years ago

Mutual exclusion between X and Oval heads now fixed by commit a7f3f9da7b230696c4292aafdc54c8f0ed5a94a1

Almost all measures in Twist & Shout sheet are now correctly processed up to PAGE step. Remaining manual fixes concern a false quarter rest (measure 41), a tuplet not recognized (measure 52), and a few quarter notes missed here and there.

Please pull.

That's enough for today... :-)

brian-math commented 2 years ago

@hbitteur Wow, that is some impressive troubleshooting and code revision in 24 hours! Here's how Twist & Shout goes for me with the new pulls:

HEADERS: It now finds all the drum clefs except in staff 2, which is still (incorrectly) recognized as bass clef. I deleted that, assigned it to drum clef instead. After a little experimentation, I assigned the double bar line plus repeat dots to the LEFT_REPEAT_SIGN inter. If I instead assigned one thick and one thin bar line, these repeat dots never got recognized when all the others in the score did.

HEADS: I turned on the book option "Support for small black note heads" and now the grace note heads in the last bar get recognized.

TEXTS: The hollow (half-note) x head in the last bar finally got removed. Need to make a head template for this case? I bet it's not in the MusicSymbols font though :-(. What to do?

SYMBOLS: The 4/4 time signature in bar 2 was never recognized, so I assigned it manually. I had to assign the grace note flags manually, too (SMALL_FLAG_SLASH).

RHYTHMS/PAGE: Even after adding several missed oval and x heads, bars 45, 46, and 52 are still pink. Two of the triplet 3s at the end were incorrectly linked to the staffs above at the RHYTHMS stage, which would cause all three of those bars to have the wrong rhythm (and also bar 51, although it was not pink). One of these triplet 3's (bar 51) got correctly allocated to the bar below at PAGE step, but bar 45 above is still pink. I'm not sure why. There are still a whole bunch of small curves (ties? slurs?) on several of the x heads.

All in all, very significant improvement!

I tried exporting this book and then importing the .mxl file into MuseScore. There was an error, but it still rendered. Looks pretty decent. The main noticeable bad artifact is lots and lots of extraneous ties and slurs. Do you know how to get rid of them at the Audiveris processing stage?

What to work on next? Some ideas:

Thoughts/suggestions?

hbitteur commented 2 years ago

Anybody know why this single-line percussion score does not process correctly on Audiveris with the Book parameter: Support for percussion staves (1 line) turned ON? I get an error at the SCALE step: "SCALE step failed. Too large interline value: 299 pixels. This sheet does not seem to contain staff lines."

Can't_Take_My_Eyes_Off_of_You-Hand_Clap.pdf

The input image contains only 1-line staves.

The key scaling data for Audiveris is the "interline" value, which is the vertical distance, from center to center, between 2 consecutive lines of a staff. This value is key to determine the actual scale of the symbols to be found in the sheet at hand. Note that Audiveris supports up to 2 different interline values in the same sheet (it's the case of two populations of staves, one large and one small). Interline value is the main output of SCALE step.

This value can of course vary from one input to the other, but it is generally around 20 pixels.

In the input at hand, the SCALE step measured a vertical distance of 299 pixels, detected this was not a reasonable value and then decided there was no music in this sheet. This happens usually when the sheet at hand is just an illustration sheet in a larger book.

The interline value(s) are determined from the histogram of lengths of vertical combo runs (either white run followed by black run, or black run followed by white run). The problem with your input sheet is: how can the OMR engine measure the actual scale used in the sheet?

FYI, here is the combo histogram on this image: There is a main peak at 299 and a much smaller one at 196.

image

hbitteur commented 2 years ago

As a quick fix, manual entering of scale data has been improved via commit e89ef1d672ae94df2954aca33ab4c92a34b372f2. Please pull.

You can stop right before SCALE step (at BINARY step) and determine the scaling values on your own by inspecting the binary view (read the precise values via the Pixel board). You can then enter them manually via the Sheet | Set Scaling Data dialog.

For example, in the case at hand with only 1-line staves, you would fill the dialog as follows:

image

Then proceed to the following steps as usual (the SCALE step will simply keep you manual values).

BTW, the multi-measure rests are still to be implemented in Audiveris. They are on the todo list.

hbitteur commented 2 years ago

New commit 666232209d2073a84e2e8753ff3106d7ba0d7ab6. You can also modify the scaling data after the SCALE step. Any value <= 0 will nullify the scaling item.

For example, running SCALE step on the case at hand will complain about no staves found. Don't discard the sheet as proposed by the engine. But call Sheet | Set Sheet Scaling and you get this pre-populated dialog: image

Of course, these are not the desired values:

This gives: image

Commit the modifications by pressing the OK button, and in the log window you get:

Interline set to 21
Small interline set to null
Beam thickness set to 10
hbitteur commented 2 years ago

Back to Twist & Shout (we are chasing two hares at the same time...)

TEXTS: The hollow (half-note) x head in the last bar finally got removed. Need to make a head template for this case? I bet it's not in the MusicSymbols font though :-(. What to do?

Yep! That's a font/template problem. No solution so far.

SYMBOLS: The 4/4 time signature in bar 2 was never recognized, so I assigned it manually. I had to assign the grace note flags manually, too (SMALL_FLAG_SLASH).

Strange, 4/4 time sig got recognized by the SYMBOLS step on my side. Perhaps I have a more recent version of the classifier model file. To be checked.

RHYTHMS/PAGE: Even after adding several missed oval and x heads, bars 45, 46, and 52 are still pink. Two of the triplet 3s at the end were incorrectly linked to the staffs above at the RHYTHMS stage, which would cause all three of those bars to have the wrong rhythm (and also bar 51, although it was not pink). One of these triplet 3's (bar 51) got correctly allocated to the bar below at PAGE step, but bar 45 above is still pink. I'm not sure why. There are still a whole bunch of small curves (ties? slurs?) on several of the x heads.

In fact, the same glyphs can lead to triplets linked to staff above and to triplets linked to staff below, because both systems try to use the same glyphs that lie in the inter-system gutter. Look carefully by selecting a tuplet with a lasso (not by simply clicking on it), and then use popup menu Inters, you'll see inters from both systems. These "gutter cases" get fixed at the end, at the PAGE step, IIRC.

The last triplet was not recognized as such, perhaps because it lies vertically too far from candidate chords. If you manually assign it to Tuplet-3 shape, it gets immediately connected to the right chords!

I indeed noticed that many tiny slurs get "recognized" by the CURVES step at the same location of cross-heads. And some tiny slurs get a higher grade than the underlying cross-head which is then discarded. This is something I really need to work on.

hbitteur commented 2 years ago

I indeed noticed that many tiny slurs get "recognized" by the CURVES step at the same location of cross-heads. And some tiny slurs get a higher grade than the underlying cross-head which is then discarded. This is something I really need to work on.

Done in commit 40492b9644639f4d11b3fc1b8de1402cae54d40f Too easy! :-)

brian-math commented 2 years ago

[Moved to Issue #565]

@hbitteur I'm going to have to work harder and faster to keep you supplied with puzzles to solve and bugs to squash!

Back to "hare#2" I decided to add a 5-line drum part above the 1-line hand clap and see how the software performs. The new score is attached. I won't go through the blow-by-blow of each STEP, but here are the main problems I encountered on page 1 (I did not attempt to process pages 2 and 3):

Can't_Take_My_Eyes_Off_of_You-Drums+Clap.pdf

brian-math commented 2 years ago

[Moved to Issue #566]

Here's a drum score with tremolo marks (2-stroke roll) on some of the stems, and ghost notes (standard oval snare drum noteheads enclosed in parentheses, beginning on sheet 2).

[BTW, if anyone knows this song well, you may notice that the hi-hat groove is not transcribed quite how Steve Gadd plays it. He plays a couple of the hi-hat notes with the foot pedal instead of the sticks. The transcriber apparently found it a lot easier to play all the hi-hat notes with the sticks.]

Main problems on p.1:

To Do: Recognize tremolo marks

On p.2:

The problems on sheet 2 were similar to those on sheet 1 (though perhaps fewer in number). A bunch of the accents were also recognized (incorrectly) as text ">" (greater than signs). These were all flagged in the log at the PAGE step, but evidently the code couldn't decide which was correct so kept both. Same for a bunch of pedal marks versus stem or beam.

The parentheses around the ghost notes were never detected.

To Do: Recognize ghost notes

50_Ways_To_Leave_YourLover-_Paul_Simon-Drumset.pdf .

hbitteur commented 2 years ago

@brian-math There are too many input files and small or big problems here.

I'd suggest:

  1. To keep this issue 33 only for general work on drum notation
  2. To create a separate issue for every precise separate topic or input, so that we can keep track of what has been fixed and what is still to be addressed.
brian-math commented 2 years ago

@hbitteur: Hervé, I submitted a pull request #568 on Sunday to add the capability to recognize diamond-shaped percussion noteheads, following a procedure suggested by Rémi Marche when my initial attempt to generate a pull request failed. I'm just checking whether I did this correctly and it went to the right place/person (presumably you). Naturally I'm happy to make revisions to those commits if anything is not in line with the Audiveris conventions or would conflict with other aspects of the code...just let me know. Thanks--Brian.

hbitteur commented 2 years ago

Hi Brian, looking at your PR page, I can read at the top of the page that "This branch is 68 commits ahead, 6 commits behind Audiveris/audiveris:master." Are you sure you created your "drum-notation" branch off of Audiveris "development" branch rather than off of default "master" branch? Remember our Wiki article about Git Workflow, we develop only on development branch or temporary sub-branches of development branch. And we merge development to master only at release time. Could you confirm?

brian-math commented 2 years ago

Hi Hervé, no I am not sure. I do know that new branches are supposed to branch off development, and I was on the development branch when I created my drum-notation branch, but I am new at git and I'm not sure I did it right. Can you tell me how to check? And how to fix it if it's wrong?

When I posted the PR, I made sure to point the merge to development rather than master: "brian-math wants to merge 2 commits into Audiveris:development from brian-math:drum-notation", and I saw a message saying this would create no new conflicts.

But it would certainly be best if I could get my local working branch to come off development if that's not already the case.

Thanks as always for your assistance.

Jojo-Schmitz commented 2 years ago

Try git pull --rebase upstream development (provided your upstream's name is 'upstream') Then a git push -f

brian-math commented 2 years ago

I'm nervous to do this without really understanding it.

It seems the assumption is that I probably have:

o---o---o---o---o  master
    |            \
    |             o---o---o  drum-notation
     \
      o---o---o---o---o  development

[Is there a way to check that assumption? Maybe the PR message "This branch is 68 commits ahead, 6 commits behind Audiveris/audiveris:master" just reflects the fact that that is where the development branch is?]

But what I want is

o---o---o---o---o  master
         \
          o---o---o---o---o  development
                           \
                            o---o---o  drum-notation

What about doing

git rebase --onto development master drum-notation ?

Do I need to insert some version of upstream somewhere in there? And based on the following, I think my upstream is actually called origin, correct?

% git remote -v origin https://github.com/brian-math/audiveris.git (fetch) origin https://github.com/brian-math/audiveris.git (push)

How is 

git pull --rebase upstream development
git push -f

different from that? Thanks for helping a git newbie!