{define copy ...} for keyboard inversions

[gwyndaf]

This is probably an enhancement, but might conceivably be a bug.

{define: G* copy G/D display G/D}
Simple form [C/E]
Indirect form [G*]

gives the error (when processed with defaults and a keyboard config)

"Slash+Copy.cho", line 2, Unknown chord to copy: G/D
    {define: G* copy G/D display G/D}
"Slash+Copy.cho", line 4, Unknown chord: "G*"
    Indirect form [G*]
"Slash+Copy.cho": No chord diagram defined for "G*" (skipped)

Although the inversion in the song body is handled nicely, that doesn't seem to happen with copy, but it would be useful if inversions/bass were also handled when copying keyboard chords.

Use case is referencing keyboard inversions indirectly, so an inverted chord gets shown for keyboard players, and a basic form for fretted instruments.

Reason for trying that approach is that, if I try to flag a 'slash' chord as a passing chord with a suffix like [G/D*], the suffix confuses the chord parser when transposing. So, I'm exploring alternative approaches.

And, the reason for using a suffix to indicate passing chords is 1) to display them smaller in most output (with {define ... format ... }) and 2) to create additional, 'simple' output without passing chords, using some preprocessor handling.

v6.050 (release)

[sciurius]

A passing chord distinguishes itself from an ordinary chord in the way it is used, similar to alternative chords (written abd shown as chords between parens). So the ultimate solution is to have a way to flag a chord as passing, and have a separate formatting rule for passing chords. And an option to leave out the passing chords from the output. That would be a nice enhancement.

In the mean time you are searching for alternative ways, and stumble upon some interesting anomalities.

One, inversions automagically exist for keyboards when used, but not when copyed. The current implementation says: you can only copy chords that are defined (known). Also, C/E could mean an inversion, or a chord with an explicit bass note, or a power chord).

{define X copy C/G}

What would you expect X to be? What do you expect X/E to be? And what if not keyboard? While redefining slash chords for string instruments makes sense, I don't think it is useful for keyboards.

Maybe some form of shorthand, e.g.

{meta X C/G}
Shorthand [%{X}]

(Set metadata.strict to false in your config.)

[gwyndaf]

Thanks Johan. Handling/styling of passing or optional chords would be a great feature. Right now, nearly all my usages of {define ... format ...} in songs are for that purpose, so it would streamline things a lot if there was a way to globally define flags and formatting for such chords.

I suspect that handling of flagged chords might require chord parsing to also recognise a suffix (after recognised root, qualifier, extension and bass). That might actually remove the issue with transposing flagged slash chords, e.g. [C/E*], which is essentially the thing I'm trying to work around.

Meanwhile, thanks for the metadata suggestion: I'll explore what that would make possible.

[sciurius]

Something experimental for you to play with.

If you check out the 'parens' branch from git, you can:

• Define a default format for parenthesised chords
• Suppress all parenthesised chords with a config setting.

If you use parens to denote your passing chords, I think this will provide all you need. (We may decide on a different notation for passing chords but you'll get the idea.)

paren.zip

[gwyndaf]

Oh, that's fantastic.

suppress is really useful and saves me several complex preprocessor lines!

One observation is how it interacts with inline chords.

With this config

{    "settings" : {
        "notenames" : true,
        "suppress-paren-chords" : false,
        "inline-chords" : "(%s)",
    },
      "chord-formats" : {
        "common" :    "%{root|%{}%{qual|%{}}%{ext|<span size=80% rise=30%>%{}</span>}%{bass|<span size=90%>/%{}</span>}|%{name}}",
        "prnfmt" :    "<small>(%{formatted})</small>"
      },
}

I get

I can see that's because %s in inline-chords takes the whole prnfmt value, including small parentheses, then adds full-size parentheses around the whole thing.

Of course, I can omit the ( ) from prnfmt, and that achieves a reasonable result, with a smaller chord name, but the surrounding parentheses remain at full size. I think the ideal might be that it's possible to have small parentheses (only) around optional/passing chords that are inline.

I wonder if a solution might be to shift away (revert) from including formatting in settings.inline-chords, to something like chord-formats.inline, which allows regular inline and parenthetical chords to be formatted independently of each other?

I guess that would mean that settings.inline-chords should then default to %s rather than [%s]. In fact, if formatting is handled elsewhere, true/false values might be sufficient.

Minor point for consideration: might something like paren be clearer? I could imagine prnfmt maybe being mis-read as 'print format'.

[sciurius]

Good observations and suggestions. I'll try to get something working like that this evening. stdfmt and prnfmt (and the new inlfmt) are cryptic since they're most likely going to change anyway. As opposed to the others these are appearance-specific formats so maybe they end up somewhere else.

[gwyndaf]

That makes sense. The existing chord-formats group of variables seems to be global (for song body and diagrams), while the group of standard, parenthetical and inline formats relates only to body output.

Strictly speaking, I guess the inline-annotations format also fits into that latter group, although it could become redundant, depending on how you implement inlfmt.

Further testing confirms that chord forms like [(D7(13))] don't seem to encounter problems, despite their use of parentheses in a different way.

[sciurius]

As I see it now, there are three levels of formatting. The result of each level is available as %{formatted} to the next level.

• Base level: formatting of a chord from components like root, qual, etc. Depends on the chord notation system used.
  • common: %{paren|(}%{root|...|%{name}}%{paren|)}
  • roman: %{paren|(}%{root|...|%{name}%{paren|)}
  • nashville: %{paren|(}%{root|...|%{name}}%{paren|)
• Custom level: The format as {define}d (default none → %{formatted})
• Presentation level: formatting for
  • plain chord: %{formatted}
  • inline chord: [%{formatted}]
  • grid chord: %{formatted}

(And indeed, settings.inline-chords would become a toggle.)

In your specific case this will become something like:

• Base (common): %{paren|<span size="70%" raise="30%" color="grey">(}%{root|...|%{name}}%{paren|)</span>}
• Custom: Your call.
• Presentation level:
  • plain chord: %{formatted}
  • inline chord: %{paren||(}%{formatted}%{paren||)}
  • grid chord: %{formatted}

It may even be achievable to define the parsing rules in the config, e.g.

    "parser" : {

        // Parsing chord presentations.
        "chords" : {
            // The default entry is optional.
            "default" : {
                "pattern" : "(.*)",
                "priority" : 0, // lowest
            },
            "paren" : {
                "pattern" : "\\((.+)\\)",
                "priority" : 10,
            },
            "passing" : {
                "pattern" : "(.+)\\*",
                "priority" : 10,
            },
            "annotation" : {
                "pattern" : "\\*(.*)",
                "priority" : 5, // higher than passing
            },
        }

This would parse (Bm) as Bm with property paren, and A7* as A7 with property passing. You define the rules and properties, and the associated formats. Interesting how nice annotations fit in.

Would we still need the custom level of formatting? I'd be inclined to say no, but I bet you'll find some nifty purposes for it ☺.

[gwyndaf]

That seems to make sense.

Right now, all my songs use {define ... format "<small>\%{formatted}</small>"} purely to format passing or optional chords consistently, which would be taken care of with paren functionality in chord formats.

I don't have any current use case for the 'custom' level, but it makes sense, given the {define ... format ...} functionality. I suppose one case might be in a structured learning setting, to highlight a new or unfamiliar chord in bold or colour, which formatting might need to be defined ad hoc per song.

I like the way 'base level' formats can depend on paren status (e.g. so chord diagrams can have smaller or () labels. An interesting (and real) case is where the same chord gets used as both a principal and an optional/passing chord in the same song. It makes sense that its appearance at different points in the song reflects the (highest priority) property matched by the parser for that particular usage.

But what form should the chord diagram and label take, i.e. what's the applicable property for the chord in the overall song?

1. Should there be two identical diagrams, but with differently-formatted name labels? I think 'no'.
2. If one diagram, how should the name label be formatted? I think the ideal is default format if the chord is used as a principal chord somewhere in the songlines or grids, and paren format if it's only ever used in (). Maybe that complicates matters, or requires two passes through the chords in use.

The parser aspect looks very powerful for customising how different chord usages get flagged. Is that limited to 'built-in' chord usages, or would that allow the definition of further ones, such as (hypothetically) staccato?

[sciurius]

Right now, all my songs use {define ... format "<small>\%{formatted}</small>"} purely to format passing or optional chords consistently, which would be taken care of with paren functionality in chord formats.

Good.

But what form should the chord diagram and label take, i.e. what's the applicable property for the chord in the overall song? ... if the chord is used as a principal chord somewhere in the songlines or grids, and paren format if it's only ever used in ().

The idea is that presentation is independent from the chord. Playing a (Bm) is identical to playing a Bm. There's a single chord diagram for Bm.

The parser aspect looks very powerful for customising how different chord usages get flagged. Is that limited to 'built-in' chord usages, or would that allow the definition of further ones, such as (hypothetically) staccato?

Of course:

           "loud" : {
                "pattern" : "(.+)!",
                "priority" : 10,
            },

Presentation level: formatting for plain chord: %{loud|<b>}%{formatted}%{loud|</b>}.

Do we need combinations? E.g. (Bm*)! ?

[gwyndaf]

Hmm. For the original purpose, of optional or passing chords, it might be enough for each chord usage to have only one property, e.g. paren or passing.

However, the underlying mechanism seems more generic and flexible, so it could support wider applications, which might involve combinations. An example might be to format a chord usage to represent both loud and staccato playing.

Personally, I have very few cases where a chord is used with two suffix flags, for alternative voicings (*, ¹ or ²) and as optional (&). However, the voicing flags are really part of the chord name, for {define} purposes, so don't need to be handled (and stripped off the name) as additional properties.

It might depend on the effort involved, but I suspect it'll future-proof the functionality if it does support combinations from the outset.

[gwyndaf]

Another possible application of chord suffixes, if they get handled generically...

I'd been experimenting with showing strum patterns in chord grids, alongside chords. Not the usual convention, but uses less page space. For instance:

|: Bm~*\~ ~~G~*/~ . . | A~*\~~~  ~Bm~*/ . . :|  x2

Although the result is visually ok, it needs trial and error, and abuses the ~ symbol, so it loses meaning as a sub-beat separator. I've pretty much abandoned the approach in this form.

However, by handling \ and / as chord suffixes (e.g. downstrum, upstrum properties), and formatting to add ↓ and ↑ suffixes in output, this might become possible:

|: Bm\ ~G/ . . | A\  ~Bm/ . . :|  x2

I might need to rethink my choice of symbols, as / could get lost in parsing bass notes, but the general idea seems worth flagging, as a potential application of chord suffixes for conveying additional performance information.

[sciurius]

Very ingenious. You know what? This worked out of the box... paren.zip

As you can see, the trailing / and \ form no problem.

[sciurius]

BTW You'll notice that JSON configs are now relaxed. Much easier to write. On top of this, I've added the possibility to break strings over multiple lines.

[sciurius]

I've checked in the latest developments into the parens branch to give you something to play with.

The project required a bit more that I anticipated, but it allowed me to remove a lot of special-cases for situations that now fit nicely in the new concept. There are stil some rough edges, though...

[sciurius]

I'm a bit in doubt as how to treat these:

<b>(Bm)</b>
(<b>Bm</b>)

The first case is clear, it is a bold parenthesised chord, a Bm with presentation property parenthesised and format <b>%{formatted}</b>. It will be shown as (Bm).

The second case is a parenthesised bold chord. A Bm with format (<b>%{formatted}</b>). But it can not have the parenthesised property, or it would come out as ((Bm)). Alternatively, with property and format <b>%{formatted}</b>, it would come out as (Bm) just like the first case.

I'm inclined to restrict the parens to immedeately surrounding the chord, making the second case invalid. What do you think?

[gwyndaf]

I think that conclusion seems reasonable.

If I understand the multi-level chord-formats functionality, the situation arises because (with default config), ( ) get added at basic level, which passes %{formatted} (including parentheses) to the presentation level for output. So there's no way to format the ( ) independently of the chord name at presentation level.

The gap in my knowledge is this: where in the process is ad hoc markup applied? From your example, I think it's maybe applied to %{formatted} after basic-level chord-formats, but before presentation-level formatting, e.g. pdf.chord-formats.

If so, and if it's important to achieve (Bm) without bold parentheses, I'd explore creating a custom config that doesn't add parentheses at basic level, but adds them at presentation level. I think that would mean %{formatted} and parenthesised reach presentation level independently, and could be combined at that level. But I think it's a niche case best served by custom configs, and don't see any need in default config.

More generally, I'm inclined to wonder why this might arise. If the bold appearance is a visual representation of something like 'loud', it might be preferable to flag the chord's function (e.g. with ! suffix parsed as loud property). Those could then be (custom) formatted at presentation level.

[gwyndaf]

I'm not sure whether my current testing/exploration is valid, because I'm getting Result: FAIL when building the latest update, and version is still reported as 6.050_19.

I don't know the make process well enough to know whether failing some tests means it rolls back to the previous version, or I do in fact have the latest update, but with the rough edges you mentioned.

Also, another possible language ambiguity: parenthesised uses UK-style 'ised' ending. However, Pango tends to use US conventions like color, so confusion might happen. Maybe clearer to use parentheses?

[sciurius]

Yes, one of the test still fails (it's the one testing (Bm)...). git log should tell you that you are on the parens branch, at commit 6c18cf6521aea454ed6a0809fa4a94c5db193e4f. This is the rough edges version.

Good suggestion to use parentheses instead of parenthesized.

You reasoning in https://github.com/ChordPro/chordpro/issues/352#issuecomment-1992275599 is correct. There are several layers of formats, from lowest ('common') via chord-specific and ad-hoc to presentation format.

I'll first make (Bm) an error, and see how far we can go. I'm also not sure whether the chord specific formats (from define) still work properly. We'll see.

[sciurius]

I've checked in 6.050_020. The tests should pass. Still some rough edges with annotations.

[gwyndaf]

Thanks Johan.

A few observations/queries so far. Any or all might be bugs or my failure to grasp the functionality!

1. As you mention, annotation issues: I still get PDF output with a * prefix, in standard chord font/colour.
2. Unicode values not interpreted in format string, e.g. I can get property strumdown from a \ suffix, but %{strumdown|<span face=dingbats>\u2798</span>} gives a string of Dingbats characters, rather than the expected ➘.
3. I think some variables might be getting stuck in memory or similar (sorry that's a bit vague). I sometimes need to re-run the same command on unchanged test file and config before changes appear in PDF output. In fact, if I keep re-running the same command, PDF seems to toggle back to previous appearance.
4. I'm not having much success with combinations of properties, such as (Dm)! for a loud, but optional chord, although each property seems to parse and behave as expected in isolation. Should I expect combinations to be recognised, or is current functionality to handle just one property for each chord, as determined by priority in parsing?
5. One regular message (about 30+ occurrences), although output is still generated:

   chord_display called on chord object at /media/nfs/Shared/Music/Technology/ChordPro/git/chordpro/script/../lib/ChordPro/Chords/Appearance.pm line 53.

I also notice that, in my latest git pull, chordpro.json is near-empty, and chordpro.prp seems to have no references to the present functionality, which appear to match the files on github (though I do have 6.050_020). Although I'd previously copied some relevant settings into my testing config, it's possible some of those might now have changed, which might explain some of the above.

[sciurius]

1. Yes, annotations need a bit more work.
2. It seems that the new relaxed json parser does not handle \uXXXX escapes, I've added a patch.
3. Interesting... Can you reproduce this and share a test program with some of the PDFs?
4. No combinations are not recognised. If you want (Dm)! you need a pattern and presentation for this combo.
5. Yes, this message indicates there's something that still needs to be adjusted. It should occur only with make test. Does you get the message when processing a normal song?

[gwyndaf]

5. Seems triggered by transposition, e.g. processing this with default (-X) config:

   {title: Parentheses test}
   {transpose: +10}
   [Dm]Regular

For 2, 3, 4, I'll do some more testing/tweaking with latest update and let you know if they persist.

[gwyndaf]

2. Unicode now handled nicely.
3. Doesn't seem an issue with the latest update.
4. Might be my application or understanding of the functionality. I'm currently exploring custom patterns and presentation formatting, and either making mistakes or discovering issues...

My current testing sample and config (otherwise processed with -X defaults Parenthetical_chords.zip

With inline chords, parens property for optional chords works well. I use parentheses to delimit standard inline chords, want optional chords just lighter and smaller. To avoid double parentheses around those, I've customised config settings, so parentheses are added at presentation level instead of basic level.

a) This works nicely for inline chords but not for above-line chords, even though I've also customised the presentation level default to reflect changes to basic level formatting. That seems to suggest that pdf.chord-formats.default settings aren't being applied, although those for inline and grid are.

b) All single custom properties seem to give expected results, e.g. loud and strums, but combinations universally don't. The interesting thing is that neither property in the combination gets applied. With a (Dm)!, I might expect it to be either smaller (paren) or red (loud), but I get neither. In fact, I notice that even basic level chord formatting (e.g. superscript 7) is ignored.

To fill my knowledge gap:

• When you say I need a pattern and presentation for this (Dm)! combo, does mean separate patterns for each aspect, i.e. ( ) and !, or a single combination pattern like "loudparen" : { \\((.*)\\)\\! }?
• Am I right in thinking that the chord parser is applied only once, and should return the property corresponding to the highest-priority property pattern match? Or it is applied repeatedly until no more patterns are matched?

Much of this is stress-testing the functionality, and might not see much real-life application, but I can see myself using parenthesised chords with strum flag to indicate optional riffs.

[sciurius]

Just a quickie:

• You do indeed need a single pattern that matches the combination.
• Chord parser is applied once, with a combined pattern that contains all patterns in descending order of priority (highest first, default = 0 last).

[gwyndaf]

Great, thanks Johan. That'll save me trying to experiment with things that aren't supported!

[gwyndaf]

Just confirming I can successfully match combinations like

"optstrumdown" : {
    "pattern" : "\\((.*)\\)\\\\",
    "priority" : 40,
},

and use that property to control presentation formats. As previously, referencing the property seems to work in grid and inline formats, but not default.