craigsapp
commented
7 years ago It depends what you mean. If you mean a physical timestamp such as seconds or milliseconds, I would say that is bad: the score can represent multiple performances, each potentially with their own timings. Likewise, if the tempo of the MIDI conversion from MEI changes you would potentially have to change all of the timings in the SVG (but this is not much of a problem, as you could regenerate the SVG images at the same time as the new MIDI file). But if you are dynamically generating new timestamps, then that means that you also have access to the MEI data, which defeats the purpose of why you are asking.
Instead, a score-based timestamp system would definitely be of general use to be embedded in SVG output from verovio. This could be used to look-up a physical time that that location in the score should be played. Note that a single point in the score can have multiple realtime positions in seconds due to repeats (this is another reason to not use physical timestamps since a note would require multiple labels for each repeat in the score). Unlike physical timestamps in seconds, the score-based timestamps would be immutable: it specifies a generalized time for the note, not the actual time in seconds. The timemaps could be embedded in the original MEI data, or extracted from the MEI data, or stored separately from the MEI data (as I do it).
I use quater-note timestamps ("qstamps") which are durations from the start of the musical score to the current note in terms of quarter notes. Here is a demo page: http://www.humdrum.org/vhv-demos/recordings There is a menu in the top right where you can select a particular recording. Then press the space bar or the play button at the bottom left to start playing. Also you can click on any note to start playing the recording at that note. Note that you must stop a recording before switching to a new recording, otherwise both will be playing, and I do not enable automatic scrolling when the music goes off of the page (this is only a demo, so no bother with refinements). Also note there is no server involved in this case (recordings are just random URLs from the internet).
For this demo, I insert qstamps into the SVG @class, which you can see if you inspect the SVG in a browser developer tools window:
Verovio inserts a <g> element for each note that wraps the graphics for drawing that note. On this element, I extract the xml::id for the note via JavaScript, and then I refer back to the original score for the qstamp of the start and stop of the note which gets inserted into the @class tags for the <g> element. I do this dynamically, but it could also be done statically to prepare an svg score.
Specifically, I extracted the id:
id="note-L40F3"
from the SVG note, then went back to the original score and identified when that note with the same xml::id should start and stop in terms of quarter notes since the beginning of the piece, then I inserted the class tags:
class="noteon-3d5 noteoff-4"
which means the note should be turned on at quarter note 3.5 and turned off at quarter note 4. Note that I encode the decimal as ".", as class tags are not allowed to have "." characters. Also you cannot start class tags with digits.
I do all of this outside of verovio (in JavaScript), but it would be handy if verovio added these qstamps to the SVG as verovio creates it.
Then to assign a physical time, I have a lookup table from "qstamp" (quarter-note timestamp) to "tstamp" (physical timestamp in seconds):
{
"work-title": "Chopin, Etude in F minor, Op. 10, No. 9",
"performer": "Mehmet K. Okonsar",
"audio": { "file":"http://imslp.org/images/7/73/PMLP01969-Complete-Opus10_128Kbps.mp3", "type":"audio/mpeg" },
"timemap": [
{"qstamp":0, "tstamp":1215.498},
{"qstamp":1.5, "tstamp":1216.505},
{"qstamp":3, "tstamp":1217.303},
{"qstamp":4.5, "tstamp":1218.102},
{"qstamp":6, "tstamp":1218.873},
{"qstamp":7.5, "tstamp":1219.617},
{"qstamp":9, "tstamp":1220.486},
{"qstamp":10.5, "tstamp":1221.564},
{"qstamp":12, "tstamp":1222.628},
{"qstamp":13.5, "tstamp":1223.523},
{"qstamp":15, "tstamp":1224.307},
{"qstamp":16.5, "tstamp":1225.127},
{"qstamp":18, "tstamp":1225.919},
{"qstamp":19.5, "tstamp":1226.646},
{"qstamp":21, "tstamp":1227.503},
{"qstamp":22.5, "tstamp":1228.533},
{"qstamp":24, "tstamp":1229.863},
{"qstamp":25.5, "tstamp":1230.702},
{"qstamp":27, "tstamp":1231.433},
{"qstamp":28.5, "tstamp":1232.155},
{"qstamp":30, "tstamp":1232.907},
{"qstamp":31.5, "tstamp":1233.636},
{"qstamp":33, "tstamp":1234.564},
{"qstamp":34.5, "tstamp":1235.524},
{"qstamp":36, "tstamp":1236.461},
{"qstamp":37.5, "tstamp":1237.3},
{"qstamp":39, "tstamp":1238.059},
{"qstamp":40.5, "tstamp":1238.799},
{"qstamp":42, "tstamp":1239.536},
{"qstamp":43.5, "tstamp":1240.295},
{"qstamp":45, "tstamp":1241.333},
{"qstamp":46.5, "tstamp":1242.396},
{"qstamp":48, "tstamp":1243.765},
{"qstamp":49.5, "tstamp":1244.974},
{"qstamp":51, "tstamp":1245.994},
{"qstamp":52.5, "tstamp":1246.92},
{"qstamp":54, "tstamp":1247.781},
{"qstamp":55.5, "tstamp":1248.625},
{"qstamp":57, "tstamp":1249.495},
{"qstamp":58.5, "tstamp":1250.429},
{"qstamp":60, "tstamp":1251.593},
{"qstamp":61.5, "tstamp":1252.603},
{"qstamp":63, "tstamp":1253.508},
{"qstamp":64.5, "tstamp":1254.416},
{"qstamp":66, "tstamp":1255.301},
{"qstamp":67.5, "tstamp":1256.136},
{"qstamp":69, "tstamp":1256.972},
{"qstamp":70.5, "tstamp":1257.837},
{"qstamp":72, "tstamp":1258.691},
{"qstamp":73.5, "tstamp":1259.603},
{"qstamp":75, "tstamp":1260.368},
{"qstamp":76.5, "tstamp":1261.142},
{"qstamp":78, "tstamp":1261.977},
{"qstamp":79.5, "tstamp":1262.707},
{"qstamp":81, "tstamp":1263.396},
{"qstamp":82.5, "tstamp":1263.877},
{"qstamp":84, "tstamp":1266.069},
{"qstamp":85.5, "tstamp":1267.023},
{"qstamp":87, "tstamp":1268.589},
{"qstamp":88.5, "tstamp":1269.802},
{"qstamp":90, "tstamp":1271.333},
{"qstamp":91.5, "tstamp":1272.284},
{"qstamp":93, "tstamp":1273.699},
{"qstamp":94.5, "tstamp":1274.967},
{"qstamp":96, "tstamp":1276.569},
{"qstamp":97.5, "tstamp":1277.629},
{"qstamp":99, "tstamp":1278.874},
{"qstamp":100.5, "tstamp":1280.353},
{"qstamp":102, "tstamp":1281.831},
{"qstamp":103.5, "tstamp":1282.842},
{"qstamp":105, "tstamp":1284.349},
{"qstamp":106.5, "tstamp":1285.863},
{"qstamp":108, "tstamp":1288.178},
{"qstamp":109.5, "tstamp":1289.141},
{"qstamp":111, "tstamp":1289.85},
{"qstamp":112.5, "tstamp":1290.653},
{"qstamp":114, "tstamp":1291.359},
{"qstamp":115.5, "tstamp":1292.143},
{"qstamp":117, "tstamp":1292.941},
{"qstamp":118.5, "tstamp":1293.768},
{"qstamp":120, "tstamp":1294.739},
{"qstamp":121.5, "tstamp":1295.536},
{"qstamp":123, "tstamp":1296.295},
{"qstamp":124.5, "tstamp":1297.029},
{"qstamp":126, "tstamp":1297.813},
{"qstamp":127.5, "tstamp":1298.58},
{"qstamp":129, "tstamp":1299.384},
{"qstamp":130.5, "tstamp":1300.38},
{"qstamp":132, "tstamp":1301.624},
{"qstamp":133.5, "tstamp":1302.541},
{"qstamp":135, "tstamp":1303.274},
{"qstamp":136.5, "tstamp":1304.073},
{"qstamp":138, "tstamp":1304.803},
{"qstamp":139.5, "tstamp":1305.532},
{"qstamp":141, "tstamp":1306.366},
{"qstamp":142.5, "tstamp":1307.182},
{"qstamp":144, "tstamp":1308.132},
{"qstamp":145.5, "tstamp":1309.005},
{"qstamp":147, "tstamp":1309.813},
{"qstamp":148.5, "tstamp":1310.621},
{"qstamp":150, "tstamp":1311.597},
{"qstamp":151.5, "tstamp":1312.446},
{"qstamp":153, "tstamp":1313.151},
{"qstamp":154.5, "tstamp":1313.943},
{"qstamp":156, "tstamp":1315.363},
{"qstamp":157.5, "tstamp":1316.489},
{"qstamp":159, "tstamp":1317.274},
{"qstamp":160.5, "tstamp":1317.985},
{"qstamp":162, "tstamp":1318.675},
{"qstamp":163.5, "tstamp":1319.306},
{"qstamp":165, "tstamp":1320.014},
{"qstamp":166.5, "tstamp":1320.887},
{"qstamp":168, "tstamp":1322.701},
{"qstamp":169.5, "tstamp":1324.083},
{"qstamp":171, "tstamp":1325.399},
{"qstamp":172.5, "tstamp":1326.76},
{"qstamp":174, "tstamp":1327.988},
{"qstamp":175.5, "tstamp":1329.013},
{"qstamp":177, "tstamp":1330.292},
{"qstamp":178.5, "tstamp":1331.46},
{"qstamp":180, "tstamp":1332.857},
{"qstamp":181.5, "tstamp":1334.011},
{"qstamp":183, "tstamp":1335.371},
{"qstamp":184.5, "tstamp":1336.935},
{"qstamp":186, "tstamp":1338.256},
{"qstamp":187.5, "tstamp":1339.38},
{"qstamp":189, "tstamp":1340.902},
{"qstamp":190.5, "tstamp":1342.364},
{"qstamp":192, "tstamp":1343.849},
{"qstamp":193.5, "tstamp":1344.737},
{"qstamp":195, "tstamp":1345.389},
{"qstamp":196.5, "tstamp":1346.043},
{"qstamp":198, "tstamp":1347.367},
{"qstamp":199.5, "tstamp":1349.34}
]
},In this case the performance of the etude is embedded in a longer recording, so the time stamps may seem large since I am skipping over the previous etudes in the opus on the recording.
If a "qstamp" is not found in the list, then I do an interpolation based on adjacent qstamps. If you look at the source code there are more parameters in the lookup table, but these are only for future display of the current measure and beat in the score, but not used in the actual score/audio alignment.
Once an SVG score is marked up with qstamp on/off times, I can do audio/score alignment with output from any notation software. For example here is the same system being used on graphic notation from another program: www.ccarh.org/haydn/op20n5/mvmt2
For this example, notice that there are repeats. You can see in the embedded lookup table that the tstamps are unique, but the qstamps repeat.
For this example, if you type "1" before clicking on a note, that will play the first repeat version. If you type "2" and click, then the second repeat will be played. For the minuet, there is also a "3" version which is the da capo repeat.
rettinghaus
notator
lpugin
ahankinson
serejahh
pe-ro
k-ljung
Is it a bad idea to be able to generate the time stamp for each note in the SVG file?
I want to generate the SVG file on a server application and use it on a client and be able to select active note(s) at a specific time, just as getElementsAtTime (milliseconds) works, without having to load the EMI file.