rism-digital / verovio

🎵 Music notation engraving library for MEI with MusicXML and Humdrum support and various toolkits (JavaScript, Python)
https://www.verovio.org
GNU Lesser General Public License v3.0
650 stars 179 forks source link

Support for MIDI export of repetitions #235

Open craigsapp opened 8 years ago

craigsapp commented 8 years ago

Is there a way of encoding repeats in MEI which verovio, and more particularly the verovio MIDI export understands?

Example work with one simple repeat (no separate second ending): http://verovio.humdrum.org/?file=chorales/chor001.krn

screen shot 2016-07-11 at 5 40 56 pm

The next complication would be first/second endings. And then cases such as da capo repeats (such as in minuet and trios).

Click to view example MEI data. ``` xml Aus meines Herzens Grunde From the Depths of My Heart Bach, Johann Sebastian Craig Stuart Sapp 2016-07-11 17:44:19

Transcoded from Humdrum with Verovio version 0.9.12-dev-48e949d

reference COM Bach, Johann Sebastian reference CDT 1685/02/21/-1750/07/28/ reference OTL Aus meines Herzens Grunde reference OTL From the Depths of My Heart reference SCT BWV 269 reference PC# 1 reference AGN chorale reference hum2abc -Q '' reference title @{PC#}. @{OTL@@DE} reference YOR1 371 vierstimmige Choralgesänge von Johann Sebastian Bach, reference YOR2 4th ed. by Alfred Dörffel (Leipzig: Breitkopf und Härtel, reference YOR3 c.1875). 178 pp. Plate "V.A.10". reprint: J.S. Bach, 371 Four-Part reference YOR4 Chorales (New York: Associated Music Publishers, Inc., c.1940). reference SMS B&H, 4th ed, Alfred Dörffel, c.1875, plate V.A.10 reference EED Craig Stuart Sapp reference EEV 2009/05/22
```
lpugin commented 7 years ago

Repeats should not be too difficult. Endings will indeed be more complicated but OK, and Da Capo and so on even more. For these I think there is a need for discussion in MEI.

craigsapp commented 3 years ago

Technically this features has now been implemented, since verovio can now expand a score and output a MIDI file (using the --expand option).

But a second stage of implementation would be to render an expanded MIDI file while keeping the score in compressed format. This would allow live playback of the MIDI data with WildWebMidi with the same score in compressed mode, where the note highlighting would jump around in the score according to the repeats (currently the score can be expanded and the MIDI file is a straight pass through the expanded score).

Here is a test case:

Screen Shot 2021-02-02 at 11 22 32 PM

This score would be displayed, but the MIDI file would play the sequence C D C E F F, and the notes would highlight in that sequence. (also there could be another expansion for only second endings, which would produce C E F in the MIDI).

The output timemap for the compressed score is currently (this should probably remain the default if no MIDI expansion is requested):

[
    {
        "tstamp":   0.000000,
        "qstamp":   0.000000,
        "tempo":    60,
        "on":   ["note-L7F1"]
    },
    {
        "tstamp":   4000.000000,
        "qstamp":   4.000000,
        "on":   ["note-L10F1"],
        "off":  ["note-L7F1"]
    },
    {
        "tstamp":   8000.000000,
        "qstamp":   8.000000,
        "on":   ["note-L13F1"],
        "off":  ["note-L10F1"]
    },
    {
        "tstamp":   12000.000000,
        "qstamp":   12.000000,
        "on":   ["note-L16F1"],
        "off":  ["note-L13F1"]
    },
    {
        "tstamp":   16000.000000,
        "qstamp":   16.000000,
        "off":  ["note-L16F1"]
    }
]

Here is what the timemap (and matching MIDI file) should be produced if an expanded MIDI file is requested:

[

    {
        "tstamp":   0.000000,   //  C (note-L7F1)
        "qstamp":   0.000000,
        "tempo":    60,
        "on":   ["note-L7F1"]
    },
    {
        "tstamp":   4000.000000,    //  D (note-L10F1)
        "qstamp":   4.000000,
        "on":   ["note-L10F1"],
        "off":  ["note-L7F1"]
    },
    {
        "tstamp":   8000.000000,    //  C (note-L7F1)
        "qstamp":   0.000000,
        "tempo":    60,
        "on":   ["note-L7F1"],
        "off":  ["note-L10F1"]
    },
    {
        "tstamp":   12000.000000,   //  E (note-L13F1)
        "qstamp":   8.000000,
        "on":   ["note-L13F1"],
        "off":  ["note-L7F1"]
    },
    {
        "tstamp":   16000.000000,   //  F (note-L16F1)
        "qstamp":   12.000000,
        "on":   ["note-L16F1"],
        "off":  ["note-L13F1"]
    },
    {
        "tstamp":   20000.000000,   //  F (note-L16F1)
        "qstamp":   12.000000,
        "on":   ["note-L16F1"],
        "off":  ["note-L16F1"]
    },
    {
        "tstamp":   24000.000000,   //  end
        "qstamp":   16.000000,
        "off":  ["note-L16F1"]
    }
]

The tstamp variable always increased in the timemap (millisecond time in the MIDI file), but the qstamp will decrease whenever there is a repeat happening.

Here is the distilled timemap:

real    score
0   0
4000    4
8000    0
12000   8
16000   12
20000   12
24000   16

In the MEI score, the time data would be reversed, with potentially multiple realTime values stored for each note:

score   real
0   [0, 8000]
4   [4000]
8   [12000]
12  [16000, 20000]

So for example, the first note would have two realTime values: 0 for the first repeat, and 8000 for the second repeat. Likewise, the note in the last measure at scoreTime 12 would have an array containing the two realTime values 16000 and 20000.

If there were an expansion list for the second endings only, then the score's note would have the values related to the timings of the notes in the shortened MIDI file:

score   real
0   [0]
4   []
8   [4000]
12  [8000]

Notice that the realTime values for the first ending note (D) is never played, so it has an empty realTime.

Test MEI data:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="4.0.0">
 <meiHead>
  <fileDesc>
   <titleStmt>
    <title />
   </titleStmt>
   <pubStmt />
  </fileDesc>
  <encodingDesc>
   <appInfo>
    <application isodate="2021-02-02T23:23:59" version="3.2.0-dev-8f1ece3-dirty">
     <name>Verovio</name>
     <p>Transcoded from Humdrum</p>
    </application>
   </appInfo>
  </encodingDesc>
  <workList>
   <work>
    <title />
   </work>
  </workList>
 </meiHead>
 <music>
  <body>
   <mdiv xml:id="mdiv-0000001923652048">
    <score xml:id="score-0000001831078766">
     <scoreDef xml:id="scoredef-0000001634015516" midi.bpm="60">
      <staffGrp xml:id="staffgrp-0000001108141941">
       <staffDef xml:id="staffdef-0000000076573858" n="1" lines="5">
        <clef xml:id="clef-L2F1" shape="G" line="2" />
        <meterSig xml:id="metersig-L3F1" count="4" unit="4" />
       </staffDef>
      </staffGrp>
     </scoreDef>
     <section xml:id="section-L1F1">
      <expansion plist="#label-A #label-A1 #label-A #label-A2 #label-B #label-B" xml:id="expansion-L4F1" />
      <section xml:id="label-A">
       <measure xml:id="measure-L1" n="1">
        <staff xml:id="staff-0000000685011685" n="1">
         <layer xml:id="layer-L1F1N1" n="1">
          <note xml:id="note-L7F1" dur="1" oct="4" pname="c" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
      <ending xml:id="label-A1" n="1">
       <measure xml:id="measure-L8" right="rptend" n="2">
        <staff xml:id="staff-L8F1N1" n="1">
         <layer xml:id="layer-L8F1N1" n="1">
          <note xml:id="note-L10F1" dur="1" oct="4" pname="d" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <ending xml:id="label-A2" n="2">
       <measure xml:id="measure-L11" n="3">
        <staff xml:id="staff-L11F1N1" n="1">
         <layer xml:id="layer-L11F1N1" n="1">
          <note xml:id="note-L13F1" dur="1" oct="4" pname="e" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <section xml:id="label-B">
       <measure xml:id="measure-L14" left="rptstart" right="rptend" n="4">
        <staff xml:id="staff-L14F1N1" n="1">
         <layer xml:id="layer-L14F1N1" n="1">
          <note xml:id="note-L16F1" dur="1" oct="4" pname="f" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
     </section>
    </score>
   </mdiv>
  </body>
 </music>
</mei>

And a test containing a second-ending expansion:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="4.0.0">
 <meiHead>
  <fileDesc>
   <titleStmt>
    <title />
   </titleStmt>
   <pubStmt />
  </fileDesc>
  <encodingDesc>
   <appInfo>
    <application isodate="2021-02-02T23:23:19" version="3.2.0-dev-8f1ece3-dirty">
     <name>Verovio</name>
     <p>Transcoded from Humdrum</p>
    </application>
   </appInfo>
  </encodingDesc>
  <workList>
   <work>
    <title />
   </work>
  </workList>
 </meiHead>
 <music>
  <body>
   <mdiv xml:id="mdiv-0000000048699960">
    <score xml:id="score-0000001258871766">
     <scoreDef xml:id="scoredef-0000001634015516" midi.bpm="60">
      <staffGrp xml:id="staffgrp-0000002139912416">
       <staffDef xml:id="staffdef-0000000014370509" n="1" lines="5">
        <clef xml:id="clef-L2F1" shape="G" line="2" />
        <meterSig xml:id="metersig-L3F1" count="4" unit="4" />
       </staffDef>
      </staffGrp>
     </scoreDef>
     <section xml:id="section-L1F1">
      <choice xml:id="choice-0000001179785592">
       <orig xml:id="orig-0000001713763553">
        <expansion plist="#label-A #label-A1 #label-A #label-A2 #label-B #label-B" xml:id="expansion-L4F1" />
       </orig>
       <reg xml:id="reg-0000000163708006" type="norep">
        <expansion plist="#label-A #label-A2 #label-B" xml:id="expansion-L5F1" type="norep" />
       </reg>
      </choice>
      <section xml:id="label-A">
       <measure xml:id="measure-L1" n="1">
        <staff xml:id="staff-0000000725373844" n="1">
         <layer xml:id="layer-L1F1N1" n="1">
          <note xml:id="note-L8F1" dur="1" oct="4" pname="c" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
      <ending xml:id="label-A1" n="1">
       <measure xml:id="measure-L9" right="rptend" n="2">
        <staff xml:id="staff-L9F1N1" n="1">
         <layer xml:id="layer-L9F1N1" n="1">
          <note xml:id="note-L11F1" dur="1" oct="4" pname="d" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <ending xml:id="label-A2" n="2">
       <measure xml:id="measure-L12" n="3">
        <staff xml:id="staff-L12F1N1" n="1">
         <layer xml:id="layer-L12F1N1" n="1">
          <note xml:id="note-L14F1" dur="1" oct="4" pname="e" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <section xml:id="label-B">
       <measure xml:id="measure-L15" left="rptstart" right="rptend" n="4">
        <staff xml:id="staff-L15F1N1" n="1">
         <layer xml:id="layer-L15F1N1" n="1">
          <note xml:id="note-L17F1" dur="1" oct="4" pname="f" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
     </section>
    </score>
   </mdiv>
  </body>
 </music>
</mei>
faelwar commented 1 year ago

I'm trying to use the expand option on the RenderToTimemap on the JS lib. However, it always returns the same result as using it without it.

timemap = tk.renderToTimemap({
        expand: "expansion-L4F1"
    });

However on the command line I can see the expect result while using the --expand option.

There is something that I could be using wrong here?

lpugin commented 1 year ago

You need to set the option before loading the data.

faelwar commented 1 year ago

Like this?

console.log("Verovio has loaded!");
    options = {
        pageWidth: pageWidth,
        scale: zoom,
        expand: "expansion-L4F1",
        // Add an option to pass note@pname and note@oct as svg @data-*
        svgAdditionalAttribute: ["note@pname", "note@oct"]
    };
    tk.setOptions(options);
    let svg = tk.renderData(meiXML, {
    });
faelwar commented 1 year ago

That code will generate the following error on my Browser:

Screenshot 2023-03-03 at 14 32 05

wergo commented 1 year ago

I am getting the same error.

(I have posted this on the Slack Verovio channel earlier.)

I have a problem with getting expansions work in the toolkit. In this example, the toolkit crashes when having to expand “expansion-repeat” or “expansion-default”. It works correctly when expansion set to “expansion-minimal”.

It seems when the toolkit has to add elements it crashes, when it removes elements, it works. (All these functions work correctly from the command line.)

wergo commented 1 year ago

Ah, I just see that it was raised again in #3314 and fixed through a90b43bb54b62331c2018dbe3699617c0fd62581.

Muromi-Rikka commented 2 weeks ago

Technically this features has now been implemented, since verovio can now expand a score and output a MIDI file (using the --expand option).

But a second stage of implementation would be to render an expanded MIDI file while keeping the score in compressed format. This would allow live playback of the MIDI data with WildWebMidi with the same score in compressed mode, where the note highlighting would jump around in the score according to the repeats (currently the score can be expanded and the MIDI file is a straight pass through the expanded score).

Here is a test case:

Screen Shot 2021-02-02 at 11 22 32 PM

This score would be displayed, but the MIDI file would play the sequence C D C E F F, and the notes would highlight in that sequence. (also there could be another expansion for only second endings, which would produce C E F in the MIDI).

The output timemap for the compressed score is currently (this should probably remain the default if no MIDI expansion is requested):

[
  {
      "tstamp":   0.000000,
      "qstamp":   0.000000,
      "tempo":    60,
      "on":   ["note-L7F1"]
  },
  {
      "tstamp":   4000.000000,
      "qstamp":   4.000000,
      "on":   ["note-L10F1"],
      "off":  ["note-L7F1"]
  },
  {
      "tstamp":   8000.000000,
      "qstamp":   8.000000,
      "on":   ["note-L13F1"],
      "off":  ["note-L10F1"]
  },
  {
      "tstamp":   12000.000000,
      "qstamp":   12.000000,
      "on":   ["note-L16F1"],
      "off":  ["note-L13F1"]
  },
  {
      "tstamp":   16000.000000,
      "qstamp":   16.000000,
      "off":  ["note-L16F1"]
  }
]

Here is what the timemap (and matching MIDI file) should be produced if an expanded MIDI file is requested:

[

  {
      "tstamp":   0.000000,   //  C (note-L7F1)
      "qstamp":   0.000000,
      "tempo":    60,
      "on":   ["note-L7F1"]
  },
  {
      "tstamp":   4000.000000,    //  D (note-L10F1)
      "qstamp":   4.000000,
      "on":   ["note-L10F1"],
      "off":  ["note-L7F1"]
  },
  {
      "tstamp":   8000.000000,    //  C (note-L7F1)
      "qstamp":   0.000000,
      "tempo":    60,
      "on":   ["note-L7F1"],
      "off":  ["note-L10F1"]
  },
  {
      "tstamp":   12000.000000,   //  E (note-L13F1)
      "qstamp":   8.000000,
      "on":   ["note-L13F1"],
      "off":  ["note-L7F1"]
  },
  {
      "tstamp":   16000.000000,   //  F (note-L16F1)
      "qstamp":   12.000000,
      "on":   ["note-L16F1"],
      "off":  ["note-L13F1"]
  },
  {
      "tstamp":   20000.000000,   //  F (note-L16F1)
      "qstamp":   12.000000,
      "on":   ["note-L16F1"],
      "off":  ["note-L16F1"]
  },
  {
      "tstamp":   24000.000000,   //  end
      "qstamp":   16.000000,
      "off":  ["note-L16F1"]
  }
]

The tstamp variable always increased in the timemap (millisecond time in the MIDI file), but the qstamp will decrease whenever there is a repeat happening.

Here is the distilled timemap:

real  score
0 0
4000  4
8000  0
12000 8
16000 12
20000 12
24000 16

In the MEI score, the time data would be reversed, with potentially multiple realTime values stored for each note:

score real
0 [0, 8000]
4 [4000]
8 [12000]
12    [16000, 20000]

So for example, the first note would have two realTime values: 0 for the first repeat, and 8000 for the second repeat. Likewise, the note in the last measure at scoreTime 12 would have an array containing the two realTime values 16000 and 20000.

If there were an expansion list for the second endings only, then the score's note would have the values related to the timings of the notes in the shortened MIDI file:

score real
0 [0]
4 []
8 [4000]
12    [8000]

Notice that the realTime values for the first ending note (D) is never played, so it has an empty realTime.

Test MEI data:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="4.0.0">
 <meiHead>
  <fileDesc>
   <titleStmt>
    <title />
   </titleStmt>
   <pubStmt />
  </fileDesc>
  <encodingDesc>
   <appInfo>
    <application isodate="2021-02-02T23:23:59" version="3.2.0-dev-8f1ece3-dirty">
     <name>Verovio</name>
     <p>Transcoded from Humdrum</p>
    </application>
   </appInfo>
  </encodingDesc>
  <workList>
   <work>
    <title />
   </work>
  </workList>
 </meiHead>
 <music>
  <body>
   <mdiv xml:id="mdiv-0000001923652048">
    <score xml:id="score-0000001831078766">
     <scoreDef xml:id="scoredef-0000001634015516" midi.bpm="60">
      <staffGrp xml:id="staffgrp-0000001108141941">
       <staffDef xml:id="staffdef-0000000076573858" n="1" lines="5">
        <clef xml:id="clef-L2F1" shape="G" line="2" />
        <meterSig xml:id="metersig-L3F1" count="4" unit="4" />
       </staffDef>
      </staffGrp>
     </scoreDef>
     <section xml:id="section-L1F1">
      <expansion plist="#label-A #label-A1 #label-A #label-A2 #label-B #label-B" xml:id="expansion-L4F1" />
      <section xml:id="label-A">
       <measure xml:id="measure-L1" n="1">
        <staff xml:id="staff-0000000685011685" n="1">
         <layer xml:id="layer-L1F1N1" n="1">
          <note xml:id="note-L7F1" dur="1" oct="4" pname="c" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
      <ending xml:id="label-A1" n="1">
       <measure xml:id="measure-L8" right="rptend" n="2">
        <staff xml:id="staff-L8F1N1" n="1">
         <layer xml:id="layer-L8F1N1" n="1">
          <note xml:id="note-L10F1" dur="1" oct="4" pname="d" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <ending xml:id="label-A2" n="2">
       <measure xml:id="measure-L11" n="3">
        <staff xml:id="staff-L11F1N1" n="1">
         <layer xml:id="layer-L11F1N1" n="1">
          <note xml:id="note-L13F1" dur="1" oct="4" pname="e" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <section xml:id="label-B">
       <measure xml:id="measure-L14" left="rptstart" right="rptend" n="4">
        <staff xml:id="staff-L14F1N1" n="1">
         <layer xml:id="layer-L14F1N1" n="1">
          <note xml:id="note-L16F1" dur="1" oct="4" pname="f" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
     </section>
    </score>
   </mdiv>
  </body>
 </music>
</mei>

And a test containing a second-ending expansion:

<?xml version="1.0" encoding="UTF-8"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://relaxng.org/ns/structure/1.0"?>
<?xml-model href="https://music-encoding.org/schema/4.0.0/mei-all.rng" type="application/xml" schematypens="http://purl.oclc.org/dsdl/schematron"?>
<mei xmlns="http://www.music-encoding.org/ns/mei" meiversion="4.0.0">
 <meiHead>
  <fileDesc>
   <titleStmt>
    <title />
   </titleStmt>
   <pubStmt />
  </fileDesc>
  <encodingDesc>
   <appInfo>
    <application isodate="2021-02-02T23:23:19" version="3.2.0-dev-8f1ece3-dirty">
     <name>Verovio</name>
     <p>Transcoded from Humdrum</p>
    </application>
   </appInfo>
  </encodingDesc>
  <workList>
   <work>
    <title />
   </work>
  </workList>
 </meiHead>
 <music>
  <body>
   <mdiv xml:id="mdiv-0000000048699960">
    <score xml:id="score-0000001258871766">
     <scoreDef xml:id="scoredef-0000001634015516" midi.bpm="60">
      <staffGrp xml:id="staffgrp-0000002139912416">
       <staffDef xml:id="staffdef-0000000014370509" n="1" lines="5">
        <clef xml:id="clef-L2F1" shape="G" line="2" />
        <meterSig xml:id="metersig-L3F1" count="4" unit="4" />
       </staffDef>
      </staffGrp>
     </scoreDef>
     <section xml:id="section-L1F1">
      <choice xml:id="choice-0000001179785592">
       <orig xml:id="orig-0000001713763553">
        <expansion plist="#label-A #label-A1 #label-A #label-A2 #label-B #label-B" xml:id="expansion-L4F1" />
       </orig>
       <reg xml:id="reg-0000000163708006" type="norep">
        <expansion plist="#label-A #label-A2 #label-B" xml:id="expansion-L5F1" type="norep" />
       </reg>
      </choice>
      <section xml:id="label-A">
       <measure xml:id="measure-L1" n="1">
        <staff xml:id="staff-0000000725373844" n="1">
         <layer xml:id="layer-L1F1N1" n="1">
          <note xml:id="note-L8F1" dur="1" oct="4" pname="c" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
      <ending xml:id="label-A1" n="1">
       <measure xml:id="measure-L9" right="rptend" n="2">
        <staff xml:id="staff-L9F1N1" n="1">
         <layer xml:id="layer-L9F1N1" n="1">
          <note xml:id="note-L11F1" dur="1" oct="4" pname="d" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <ending xml:id="label-A2" n="2">
       <measure xml:id="measure-L12" n="3">
        <staff xml:id="staff-L12F1N1" n="1">
         <layer xml:id="layer-L12F1N1" n="1">
          <note xml:id="note-L14F1" dur="1" oct="4" pname="e" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </ending>
      <section xml:id="label-B">
       <measure xml:id="measure-L15" left="rptstart" right="rptend" n="4">
        <staff xml:id="staff-L15F1N1" n="1">
         <layer xml:id="layer-L15F1N1" n="1">
          <note xml:id="note-L17F1" dur="1" oct="4" pname="f" accid.ges="n" />
         </layer>
        </staff>
       </measure>
      </section>
     </section>
    </score>
   </mdiv>
  </body>
 </music>
</mei>

Has there been any new progress on this issue?