Ill timber/volume in specific tone range

SAKryukov commented 7 years ago

Abstract

Suddenly, at first attempt, I came across weird sound in one specific interval, only within one specific semitone corresponding to MIDI notes 50 to 51. So far, I could not reproduce the problem in any other tonal range.

Steps to Reproduce

Run this and try microtonal sounds between MIDI notes 50 to 51, compare with other sounds:

<html>
    <head>
        <script src='https://surikov.github.io/webaudiofont/npm/dist/WebAudioFontPlayer.js'></script>
        <script src='https://surikov.github.io/webaudiofontdata/sound/0001_GeneralUserGS_sf2_file.js'></script>
        <script>
            var selectedPreset=_tone_0001_GeneralUserGS_sf2_file;
            var AudioContextFunc = window.AudioContext || window.webkitAudioContext;
            var audioContext = new AudioContextFunc();
            var player=new WebAudioFontPlayer();
            player.adjustPreset(audioContext,selectedPreset);
            function play(pitch) {
                player.queueWaveTable(
                    audioContext, audioContext.destination, selectedPreset,
                    audioContext.currentTime, pitch,
                    0.4);
            } // play
        </script>
    </head>
    <body>

        <a href='javascript:play(2.0+12*4);'>MIDI note 50: E</a><br/>
        Moving up by 10 cents:</br>
        <a href='javascript:play(2.1+12*4);'>50.1</a><br/>
        <a href='javascript:play(2.2+12*4);'>50.2</a><br/>
        <a href='javascript:play(2.3+12*4);'>50.3</a><br/>
        <a href='javascript:play(2.4+12*4);'>50.4</a><br/>
        <a href='javascript:play(2.5+12*4);'>50.5</a><br/>
        <a href='javascript:play(2.6+12*4);'>50.6</a><br/>
        <a href='javascript:play(2.7+12*4);'>50.7</a><br/>
        <a href='javascript:play(2.8+12*4);'>50.8</a><br/>
        <a href='javascript:play(2.9+12*4);'>50.9</a><br/>
        Compare the sound!</br>     
        <a href='javascript:play(3.0+12*4);'>MIDI note 51: ♯F</a><br/><br/>
        But beyond the ill range shown above (between 50 and 51), everything is fine:</br>
        <a href='javascript:play(3.1+12*4);'>51.1</a><br/>
        <a href='javascript:play(3.2+12*4);'>51.2</a><br/>
        <a href='javascript:play(3.3+12*4);'>51.3</a><br/>
        <a href='javascript:play(3.4+12*4);'>51.4</a><br/>
        <a href='javascript:play(3.5+12*4);'>51.5</a><br/>
        <a href='javascript:play(3.6+12*4);'>51.6</a><br/>
        <a href='javascript:play(3.7+12*4);'>51.7</a><br/>
        <a href='javascript:play(3.8+12*4);'>51.8</a><br/>
        <a href='javascript:play(3.9+12*4);'>51.9</a><br/>
        Done!</br>      

    </body>
</html>

The Problem Background

I do understand that this work is not intended for microtonal use. But! As I can see, microtonal sound works even without any change in the library. Also, I seriously think that the main value of this library is its microtonal potential.

Presently, there is a great activity in music around microtonal field: development of novel instruments and tone systems, and so on. There is a lot of 12-tone software these days, but the microtonal capabilities present a considerable problem. It's much better to concentrate on support of microtonal systems now.

My Questions

Can you explain it?
Can you suggest some other way of using the microtonal tones?
Would you consider further development to offer microtonal versions of the API? If you can consider it, I can provide some help, as the very interested person qualified in the field. Alternatively, I could fork it and publish the fork if you don't mind and if you can support me with just answering some questions. In turn, I can explain what I would like to achieve.

Thank you. —SA

surikov commented 7 years ago

1.there are about 1000 of instrumets. Not all samples have been tested yet. Choose any other sample.

you can use any frequency, see 'sound fx' in 'Code examples' of project description.
all you need for microtonal music implemented already.

feel free to ask any help

SAKryukov commented 7 years ago

Thank you very much, but your judgement is not quite correct.

I say so because I found a bug. And I mean it: it's a bug, even though it's probably completely hidden by the fact that your pitches are all integer. It's methodically wrong. And I already try different instruments — the problem is different. And it's your claim that "all you need for microtonal music implemented already". (I am very, very glad to hear that.) Than it needs to be fixed. Look, it's not hard to fix.

I'll explain: for some pitches, Zone is not found, and further processing goes as it was #0. Why not found? Because zones should be, say A... B, then B to C, but what is see is A... B, then B+1 to C... What I say will be just a bit more complicated, but correct: you need to have zones without those gaps and search by comparison which is strict comparison operator on one side and not on another, for example: Left < pitch && pitch <= Right. Isn't it clear?

Now, the rough work-around works: I simply pass fractional pitch in notes, but round it only inside findZone. It sounds good now, but my Question One for you is: is my assumption correct: the choice of zone slightly outside the zone, close to its boundary is still "good enough". If so, my rounding work-around is also "good enough". More serious change would be fixing the zones, but I do understand how much work it can be.

Question Two: why there are zones with the same MIDI note, overlapping, and why some zones have lower limit of the range higher than the upper limit? Does it make any sense? If I'm not much mistaken, such zones cannot be found. It is right?

Look, you have done great and big work. It would be pity to block its use (microtonal, in particular) due to small bug. I really would like to get it going. I checked some other libraries, they impose near-prohibitive limitations on microtonal application (MIDI.js is one such example), so it's quite possible that your work is the only reasonable one.

Thank you very much. —SA

surikov commented 7 years ago

what is 'good enough'? you can use any zone to produce any pitch
there are about of 1000 instruments in library. may be some instruments has errors

p.s. create new issue add link to JSBin code to reproduce bug add short description be more specific

SAKryukov commented 7 years ago

No, this is a regular problem. Yes, my guess was that it came from instruments. As I am not quite well understand how you produced the instruments' .js (I see you batch files, etc.), I'm not quite sure. I tried to explain the problem in this.findZone, even if it's not the root of the problem. From my guess and your note, the root of the problem is in instrument. We need to do something about it. The problem is obvious.

"Good enough" is the key here. Let's try to understand. Any zone of any pitch? The why having the zones? As I can see, you can use any zone, but quality of sound suffers. I guess, zones make sense.

Say, you have range of pitches for zone. Let me look:

Object {midi: 0, originalPitch: 2600, keyRangeLow: 0, keyRangeHigh: 27, loopStart: 35640, …}
preset.zones[1]
Object {midi: 0, originalPitch: 2900, keyRangeLow: 28, keyRangeHigh: 31, loopStart: 47869, …}

This is the root of the problem. This is repeated everywhere. It works for integer pitches, but wrong in principle. It should be, say 0 to 27 and 27 to 31 (not 28 to 31).

My guess about "good enough" is: say, pitch is 27.5. My idea was: isn't zone 0..27 good for this pitch, as well as 28..31? I can hear by sound, that far-away zone produce bad sound. After all, you wrote findZone and use it. I thought, you are doing it for good reason. Only the search may fail for fractional note values. Ideally, I would post-process zones. Let's see further:

Object {midi: 0, originalPitch: 3800, keyRangeLow: 36, keyRangeHigh: 35, loopStart: 49318, …}
preset.zones[4]
Object {midi: 0, originalPitch: 2600, keyRangeLow: 36, keyRangeHigh: 27, loopStart: 35640, …}
preset.zones[5]
Object {midi: 0, originalPitch: 2900, keyRangeLow: 36, keyRangeHigh: 31, loopStart: 47869, …}

As you can see, for the same midi 0, we get some zones with keyRangeLow > keyRangeHigh, which may or may not be considered as nonsense, but can't you see they will be ignored by your code, as a result of the call to your findZone. Maybe it's better remove all such zones after loading, to improve performance...

Again, the problem occurs when zone is not found. Then the zone becomes preset.zones[0], and it produces bad sound. "Bad sound" is not a particular bug (to report), I just can hear it's wrong.

I'm not creating new issue yet, want to get consulted with you first.

Thank you. —SA

surikov commented 7 years ago

This isn't a regular problem

open https://jsbin.com/qupupogeyu/edit?html,output
test it

no sound artefacts

SAKryukov commented 7 years ago

You could not hear sound artifacts maybe because you are using different instrument. Besides, some can hear some defects and some do not. It does not mean that the defect does not exist. Finally, I pointed out wrong behavior in code; if such wrong behavior is found, it does not matter if the software "works" or not; it still needs to be fixed — the problem is way too obvious.

Problem is fully analyzed, fixed and solution is tested, presented as a pull request. Any questions, concerns?

I found couple more very apparent bugs which need to be fixed and can be fixed simply. I did not present fixed to avoid breaking of separation of concerns.

Thank you. —SA

surikov commented 7 years ago

Thanks you for investigating of problem. Those are bugs of .sf2 converter but not player. I will fix converter and reconvert all instruments. It will take few days.

notator commented 7 years ago

I'm also looking closely at (my derivative of) the gree sf2 parser to see if I can find any bugs. See https://github.com/notator/WebMIDISynthHost/issues/34. Lets keep in touch. All the best.

SAKryukov commented 7 years ago

Great. I understand that it's a converter problem. More exactly, it's a problem of the font file I can see, no matter how you obtained it. Of course, my solution is a sheer work-around, which works for me as soon as I have to use the font files with this problems.

There is much more serious problem which I don't know how to work around. The instruments with damping does not damp out, they are looped. This is wrong. I modified your code to have "infinite" sound, a must-have thing if you have to create an instrument. If the sound damps, such as in piano or guitar, it should damp out to silence, not loop infinitely at lower volume. Your samples are not long enough, they are cut prematurely...

Thank you. —SA

notator commented 7 years ago

Looping is handled wrongly in the gree code. (The code is a good start for parsing soundFont files, but its incomplete, and needs debugging at a low level...)

At line 141 in sound_font_synth_note.js it says: bufferSource.loop = (this.channel !== 9); So all presets loop if they are not percussion! I've changed the line as follows: bufferSource.loop = (this.channel !== 9 && (keyLayer.loopFlags === 1 || keyLayer.loopFlags === 3)); whereby the loopFlags are retrieved from the "sampleModes" sf2 generator. I cant find the code for parsing "sampleModes" in gree. Maybe you can? Looping is controlled by the two least significant bits in the sampleModes flags. Hope that helps!

SAKryukov commented 7 years ago

James,

Thank you for your message. Sorry, I have no idea of gree operation, perhaps due to lack of documentation in a language I could understand. All I know is that the JavaScript font data provided by the project webaudiofontdata.

From that standpoint, the major problem with looping is not incorrect looping, but insufficient data in the sample -- there is not enough data to loop correctly or wait for the completion of damping. Should I understand from your comment that some part of sample was lost in processing? In all cases, I don't know or have original font source and don't know how surikov's JavaScript form of font is created.

If I had original sources of fonts of reasonably good quality and completeness, knew all the formats, etc., I would probably think of parsing and processing them by myself. It could be better than looking for some undocumented software being not sure that it does the job properly. I already experimented with alternative ways of sample-based synthesis (Fourier, wavelets) but still don't know good solution.

I already created fully-functional [https://github.com/SAKryukov/microtonal-chromatic-lattice-keyboard](microtonal instrument), but its timber repertoire is tiny and could use better quality of sounds.

Thank you. —SA

surikov commented 7 years ago

Now all soundfonts works well. Download and test again.

SAKryukov commented 7 years ago

Thank you very much.

How about the other problem, looping of the sounds which should damp out completely by certain time but instead are cut when they still sound (guitars, pianos, bells)?

—SA

surikov commented 7 years ago

zone with loopStart and loopEnd plays in loop zone.ahdsr:true defines default AHDSR envelope redefine zone.ahdsr with your own array of volumes

defaultAHDSR array is [{duration : 0,volume : 1}, { duration : 0.5,volume : 1}, { duration : 1.5,volume : 0.5}, { duration : 3,volume : 0}]

There are hundreds of instruments in library. Some instrument has looping. Some instrument has looping and ahdsr. Choose carefully.

PS Create new issue if you have new question.

SAKryukov commented 7 years ago

Done. Before going to the looping/AHDSR problems...

I downloaded new audiofontdata (is it what you were talking about?). And I see exact same thing. For example, keyRangeLow, keyRangeHigh go: 35-41, 42-47, 48-52, 53-75.

Totally wrong, conceptually. This is not what can work with, say, fractional tone values expressed in 12-TET. Should be, for example, 35-42, 42-48, 48-53. But my pull request fixed it dynamically, anyway. This part of the problem is very simple. And the sound artifact breaks the ear. I explained how to reproduce it.

Now, the problem is: "there are hundreds of instruments in library". Let's say, I would ask you: "Do you have just one which sounds correctly with either your `webaudiofont' player or anything else? I need just one, but it should be piano". What would you answer? "Go find one in hundreds of instruments?" But why would I hope for success and decide to spend time on that, if tens of examples failed to sound correctly? The problem is: all samples I saw are just too short; they are cut well before the full damp ("full damp" is a questionable notion, I know, but this is not the point; the point is: too short).

So, what do we have here:

I cannot create correct damping using AHDSR — silly me.
But your player also does not show this behavior. In your demo, you simply use too short duration, everywhere. Same thing goes with SSSynthesiser.js.

Any suggestions?

Thank you. —SA

surikov commented 7 years ago

player works fine with noninteger pitch values. Test again. look to code examples. Find AHDSR example. Test sound, look to the code.

surikov / webaudiofont