Midi learn built-in - Githubissues

micbuffa commented 7 years ago

Hi g200kg, we're currently working on adding midi learn capabilities to webaudio-controls. I'll keep you informed, when finished we'll add a pull request.

Meanwhile, you can watch this video: https://www.youtube.com/watch?v=WckGJNSNd3c

We are adding in the components 👍

Right click menu with midi learn/midi clear
Exporting a global mapping JSON object for all the mappings between midi controllers and all webaudio controls in the current webapp. -Loading and setting up midi mappings globally from all webaudio controls in the current webapp, using a JSON file (the one exported by previous step).

Currently it works with code outside of the web components. We're now adding this into the web-audio control components. If it works as intended, just updating the components in any app that use them should add midi learn to this app's webaudio controls.

micbuffa commented 7 years ago

Hi! We will have a version with integrated (and switchable on/off) midi learn by the end of this week. Would you be interested in trying it? Then we'd be glad to make a pull request once it's stable.

g200kg commented 7 years ago

Hi, I am currently not sure whether it is a generic one that should be defaultly integrated, but I would like to investigate if you send me a pull request. Thank you !!

micbuffa commented 7 years ago

Actually, we're still working on it but for sure there is an option setMidiLearn(false) in the webAudioControls. In that case the code will be ignored. In case it's activated, then you can build a midi device menu, all webaudio-knobs, sliders, switches will pop up a context menu on right click for assigning a midi controller + the webaudio controls + will expose the mapping too. I prepare some demos for next week so that you can try/look how it works. I'll do a pull request too. Just to illustrate the concept: I have written a tube amp guitar simulator that uses WebAudioControls. I just updated the html I import with the new version with midi learn in it, and all my knobs have a context menu, I was able to insert a "midi device" menu in my GUI. And all knobs can expose in JSON their mapping + keep working the same as before (midi event turns the knob and generate an onchange event, so existing listeners still work). There is also an API method for getting all midi events and do some app specific treatment. For example, with the amp I assigned the knobs from a novation lauch control pad to the amp knobs + I assigned the flat buttons from a Behringer midi guitar pedal board to Program Changes events I process directly in the app for changing the global sound presets in the amp. I do this through the webaudio controls midi passthrough.

If I have a webaudio app with my own midi processing, I can just use the webaudioControls "as is", as midi learn is disabled by default.

Good thing is: you have a complex app with lots of webaudio controls, just update the lib and you get free midi learn and midi event listeners for free + the possibility to save the midi mapping.

I'll post demos + videos by the end of the week.

micbuffa commented 7 years ago

I just made a pull request. I'm open to all improvements (hot plug / unplug of midi devices for example, should be ok soon).

micbuffa commented 7 years ago

Hi! I saw you merged the pull request. Things I'm gonna do these next days: add hot plug/unplug of midi devices support, and complete the webaudiocontrolsManager object (with an api to get all midi devices, to save+restore in JSON all midi mappings, etc.)

g200kg commented 7 years ago

hello, thank you for the pull-request.

Now I fix MIDI handling to limit only for control-change. Because in some environment, MIDI active sensing or some other messages disturbing midi learn.

micbuffa commented 7 years ago

Thanks, you're more expert than me about MIDI. The work I submitted in the pull request is not completed, as you might have seen, I added a webaudiocontrolManager object in webaudiocontrols.html, in the script at the end, that I need to complete. This object is meant to expose the list of MIDI devices, and to allow global save/restore of MIDI mappings for all knobs, sliders, switches. Did you see the change I've done in the JS code for the switches:

pointerdown: function(e) { // For midi learn if(e.type !== 'mousedown' || e.button !== 0) return;

I'm not sure about the last line against tactile events. I added this because a right click on a switch was actioning the switch. Maybe we should also test for touch event ?

g200kg commented 7 years ago

Some points :

Some globals (midiAccess , handleMIDIMessage, etc...) may conflicts to other scripts.
Not only MIDI-learn, predefined MIDI CC# assign may be more useful for app developers. I think. for example, <webaudio-knob midicc="2"></webaudio-knob> will assign this knob to control-change 2 or maybe <webaudio-knob midicc="3.2"></webaudio-knob> if use midi ch(3) + midi cc#(2)
About pointerdown of switches, current code will simply ignore touch devices. To enable MIDI-learn on touch devices, I think touch-and-hold handling or something is needed.

micbuffa commented 7 years ago

Ok, I'll try to fix these globals (encapsulate in the webaudiocontrolsManager object), I never used midi CC, how do they work? I have a Behringer pedalboard and the switches on the pedal board did not work with the current midilearn code, maybe this is the reason? Do you think that it's possible to assign these automatically (with a midi learn like behavior). Or are you talking about an optional way to make assignment by giving the midi codes directly?

What can I do to help?

Le dim. 30 avr. 2017 à 22:35, g200kg notifications@github.com a écrit :

Some points :

-

Some globals (midiAccess , handleMIDIMessage, etc...) may conflicts to other scripts.

Not only MIDI-learn, predefined MIDI CC# assign may be more useful for app developers. I think. for example, will assign this knob to control-change 2 or maybe if use midi ch(3) + midi cc#(2)

About pointerdown of switches, current code will simply ignore touch devices. To enable MIDI-learn on touch devices, I think touch-and-hold handling or something is needed.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/g200kg/webaudio-controls/issues/15#issuecomment-298255347, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnl2G79z6-4h-hLlj1_1bFrnaDnSqV_ks5r1PB0gaJpZM4LONmp .

g200kg commented 7 years ago

Hi, midi CC means just 'MIDI control-change' messages, that has 3 bytes [0xbX nn vv] X=channel nn=control number (0-119) vv=value (0-127). these are normally used for control parameters.

Behringer MIDI pedalboards are also configurable to assign appropriate midi CC to each switches/pedals. But probably factory preset may use other messages. e.g. program-change or something.

Because current code limit to learn only midi CC, it may not work with initial state of Behringer pedal. You can extend the MIDI learning function beyond control changes, but I think you should exclude system messages and channel mode messages to avoid conflict. if ((data[0]&0xf0)==0xf0) || ((data[0]&0xf0)==0xb0 && data[1]>=120)) return;

Anyway, what I wanted to say is to directly specify default corresponding MIDI messages in HTML code instead of doing MIDI learn each time.

Thank you

micbuffa commented 7 years ago

Hi! I removed all externals and now have only one single ES5 constructor function with everything encapsulated. I also implemented the midicc attribute you described. Just one question: I now get the different midi properties from the midi event like that:

   data = event.data,
    cmd = data[0] >> 4,
    channel = data[0] & 0xf,
    type = data[0] & 0xf0, // channel agnostic message type. Thanks, Phil Burk.
    note = data[1],
    velocity = data[2];

My question is: is cc the note? if I use "3.2" as a value for midicc, with 3 being the channel and 2 the cc, how do I get the cc from the midi event.data value?

g200kg commented 7 years ago

Yes, when message type is control-change, ( means... (data[0]&0xf0)==0xb0), data[1] is control number (cc#) and data[2] is value.

type	data[0]	data[1]	data[2]
NoteOn/Off	cmd (0x90 or 0x80) +ch	note	velo
ControlChange	cmd (0xb0) +ch	control number (cc#, 0-119)	value
ProgramChange	cmd (0xc0) +ch	prg#	not exist
ChannelPressure	cmd (0xd0) +ch	value	not exist
PitchBend	cmd (0xe0) +ch	value LSB	value MSB
SystemMessages	0xf0 - 0xff	depends on data[0]

Then midicc="3.2" means : [0xb0 + (3-1), 2, val] (note that ch is usually represented as 1-16)

Actually, messages other than ControlChange is not so suitable for this purpose. PrgChange / ChannelPressure has not data[2], PitchBend's data[1] and data[2] are part of contiguous 14bit value, and SystemMessage format is depends on each command.

g200kg commented 7 years ago

Hi, The current code does not filter the message in the case of the switches, is it because of the behavior with the Beringer pedal? Can you see what kind of messages are issuing when Behringer pedal switch pressing?

In some environment, SystemMessges are automatically issued that prevent normal MIDI learning.

micbuffa commented 7 years ago

I haven't re-tested with my Behringer pedalboard. I'll try tomorrow. I removed the filter because when I tried with some Novation Launchpad and launchcontrol devices I wanted the switches from my WebAudio guitar amp to be assigned to some pressure pads like that (and whose type was returning true to the test if ((data[0]&0xf0)==0xf0) || ((data[0]&0xf0)==0xb0 && data[1]>=120)) return;

I got event.data[0] = 144 (decimal) that did not pass the filter (I got this value when I pressed the last pressure pad on my Novation launchcontrol)

https://us.novationmusic.com/launch/launch-control and https://us.novationmusic.com/launch/launchpad

g200kg commented 7 years ago

Well, 144 (decimal) is 0x90, NoteOn message. It can not be caught on ((data [0] & 0xf0) == 0xf0) || ((data [0] & 0xf0) == 0xb0 && data [1]> = 120)).

Anyway, using NoteOn as a switch is not basically assumed by the MIDI standard, so inconsistency is likely to occur if you try to operate it in all cases.

As a fundamental problem, when using control change as a switch under MIDI standards, value must be off: <64, on:> = 64. Since the current implementation does not conform to this rule, unexpected behavior occurs when assigning a physical controller sending control change to webaudio-switch.

micbuffa commented 7 years ago

Ok I see what you mean. I fix that, test a little more and submit a pull request.

g200kg commented 7 years ago

Thank you! merged and slightly changed.

delete alert() that blocks program execution
midi channel representation is changed to 1-16 instead of 0-15 in midicc attrib.

g200kg commented 7 years ago

I'm not sure but on Windows, sometimes the browser seems be frozen about 1 sec when using webaudio-controls. your AmpSim sample also sounds sometimes interrupted. maybe WebMIDI related issue...

micbuffa commented 7 years ago

Ah, even sample1.html does that? Can you tell me how to reproduce this? It's true that I only tested on a Mac with Google Chrome...

micbuffa commented 7 years ago

Can you try https://wasabi.i3s.unice.fr/AmpSim3/ and tell me if this one too has this behavior... the one with all frequency analyzers + oscilloscopes uses lots of cpu just for that.

micbuffa commented 7 years ago

A recent video of the amp with the new version of webaudiocontrols + midi: https://www.youtube.com/watch?v=NlcNXdRk6qM

g200kg commented 7 years ago

Tested on Windows10 + Chrome58 sample1.html does not that. AmpSim3 still does that.

It seems that not only normal CPU usage problem. Just by opening AmpSim3, the mouse cursor on the PC will not move smoothly and the sound of the application in another Chrome window will run out.

I will check it a bit more

micbuffa commented 7 years ago

Can you compare then with mainline.i3s.unice.fr/AmpSim3/ that uses an old version of the webaudiocontrols without midi support?

I also discovered that on FF, the webaudiocontrols work, but the script at the end of webaudiocontrols is not executed correctly (very, very strange behavior due to the webcomponent-lite.js polyfill I guess). I'l still investigating.

This does not prevent webaudiocontrols to work. It's just that this code does not seem to be executed at all...

g200kg commented 7 years ago

In my environment,

Win10 + Chrome 58: https://wasabi.i3s.unice.fr/AmpSim3/ : Almost work but glitch noise mixes http://wasabi.i3s.unice.fr/AmpSim3/ : redirected to https https://mainline.i3s.unice.fr/AmpSim3/ : NET::ERR_CERT_AUTHORITY_INVALID cant access at all http://mainline.i3s.unice.fr/AmpSim3/ : DOMException: Only secure origins are allowed, getUserMedia() no longer works on insecure origins.

Win10 + Firefox 54: https://wasabi.i3s.unice.fr/AmpSim3/ : SEC_ERROR_UNKNOWN_ISSUER can't access at all http://wasabi.i3s.unice.fr/AmpSim3/ : Almost work but glitch noise mixes https://mainline.i3s.unice.fr/AmpSim3/ : SEC_ERROR_UNKNOWN_ISSUER can't access at all http://mainline.i3s.unice.fr/AmpSim3/ : Almost work but glitch noise mixes

Unfortunately, non-MIDI version on Win+Chrome does not work. When I tried these again, I did not feel mouse cursor freeze in Chrome. It might be another problem.

micbuffa commented 7 years ago

Ok, I don't have paid for https certificate for the mainline.i3s.unice.fr domain, but you can nevertheless access the application, just accept the security exception. It should work after that.

g200kg commented 7 years ago

Win10+Chrome : http://mainline.i3s.unice.fr/AmpSim3/ In this case, graphs are not displayed and sounds will be played without any effects. 20170505ampsim2

micbuffa commented 7 years ago

On try again the last URL with https and accept the security exception

g200kg commented 7 years ago

non-MIDI version on Win+Chrome : Ignoring security error, it can almost work but glitch noise mixes same as MIDI version.

And, I found both non-MIDI / MIDI versions has audible glitches but all contiguous audio samples can be captured from audioI/F monitor loopback. I think it maybe a problem about audio output devices.

micbuffa commented 7 years ago

You tried with the integrated audio player or with the mic input? On my win 10 machine, an hp840 elite, I just tried with the integrated audio i player :and internal sound card : no glitches. With a guitar and external sound card : huge latency and no glitches. What hardware did you use?

Apparently is not the midi code...

g200kg commented 7 years ago

Confirmed. The glitches are because of Behringer 302USB mixer builtin audio-i/f. on-board integrated audio output has no problem.

Thank you

micbuffa commented 7 years ago

Hmmm this is really strange as my webaudio graph is not so big nor complicated...

micbuffa commented 7 years ago

Anyway, thanks a lot for testing my app. I can't thank you enough for your work on webaudiocontrols!!! I wrote several other apps that can be useful, that use them, but they are still in a very prototype stage...

A guitar pedal fx designer: http://i.imgur.com/FWULsoK.jpg in which each pedal is a Polymer WebComponent that uses webaudiocontrols. You can choose themes/templates/, buttons/switches look'n'feel, char fonts, colors, sizes etc.

A pedalboard for connecting the above pedals: http://i.imgur.com/y905v2l.jpg, plain JS with SVG + polymer component pedals.

This is an ongoing work. The amp is also being rewritten to become a polymer web component.

g200kg commented 7 years ago

This Audio I/F has not had any problem so far, but I suppose it was that I installed 'Windows 10 Creators Update' about a week ago. Basically it works, but there may still be some problems and performance may be decreasing.

guitar pedal fx looks awesome :) I still have something I'd like to do with midicc attribute related functions, so I may make changes little by little.

Thank you

g200kg commented 7 years ago

Although it is not yet compatible with Polymer 2.0,

Unfortunately, since there are cases where the MIDI function conflicts with the application itself, it is turned off by default and you need to specify the flag before the tag when using it : <script>UseWebAudioControlsMIDI=1</script>

If the application itself use the WebMIDI API, the library side may receive MIDI messages unintentionally. Also, since we use multiple MIDIAccess within one application, the behavior of the browser is not stable yet (especially in Windows).

I think that using MIDIAccess in a library, that is hard to see from a programmer, needs to make it clearly conscious.

micbuffa commented 7 years ago

Ok, no problem. So you midified the code so that if we plan to use the midi learn builtin, we just add this one line script?

g200kg commented 7 years ago

Yes, add just one line like this :

<script>UseWebAudioControlsMidi = 1</script>
<link rel="import" href="bower_components/webaudio-controls/webaudio-controls.html" >

micbuffa commented 7 years ago

Perfect, this seems a good approach.

Le jeu. 8 juin 2017 à 09:26, g200kg notifications@github.com a écrit :

Yes, add just one line like this :

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/g200kg/webaudio-controls/issues/15#issuecomment-307022268, or mute the thread https://github.com/notifications/unsubscribe-auth/AAnl2Oz2ps6_uEePmxlRu_5Ils6Jx1lTks5sB6IogaJpZM4LONmp .

g200kg commented 6 years ago

I made it possible to set flag with WebAudioControlsOptions object at the same time with other options

g200kg / webaudio-controls

Midi learn built-in #15

Some globals (midiAccess , handleMIDIMessage, etc...) may conflicts to other scripts.

Not only MIDI-learn, predefined MIDI CC# assign may be more useful for app developers. I think. for example, will assign this knob to control-change 2 or maybe if use midi ch(3) + midi cc#(2)