AI Jam js not working

[ghost]

Hi there, I'm trying to run ai-jam-js on my Ubuntu 16.04 from my Magenta conda environment.

I want to use an Akai LPK25 as a MIDI controller, so I plug it into my PC with the environment active and I run sh RUN_DEMO.sh in the appropriate directory. By accessing to 127.0.0.1:8080 the interface is shown but it seems like it is not receiving any kind of signal because the piano keys only work when I use the mouse and they don't emit any sound at all (even with the mouse).

I don't know what's wrong but I suspect that it has to be sth related to the hardware and synth connections. Here's the log:

File http://download.magenta.tensorflow.org/models/attention_rnn.mag already present
File http://download.magenta.tensorflow.org/models/performance.mag already present
File http://download.magenta.tensorflow.org/models/pianoroll_rnn_nade.mag already present
File http://download.magenta.tensorflow.org/models/drum_kit_rnn.mag already present
 * Running on http://127.0.0.1:8080/ (Press CTRL+C to quit)
WARNING:tensorflow:No input port specified. Capture disabled.
INFO:tensorflow:Opening 'magenta_in' as a virtual MIDI port for output.
2018-02-01 01:04:04.104307: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2018-02-01 01:04:04.166210: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
WARNING:tensorflow:The saved meta_graph is possibly from an older release:
'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'.
INFO:tensorflow:Restoring parameters from /tmp/tmpIxuYso/model.ckpt
Loaded 'attention_rnn' generator bundle from file './attention_rnn.mag'.
WARNING:tensorflow:The saved meta_graph is possibly from an older release:
'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'.
INFO:tensorflow:Restoring parameters from /tmp/tmpZlPHdN/model.ckpt
Loaded 'drum_kit' generator bundle from file './drum_kit_rnn.mag'.
INFO:tensorflow:Opening 'magenta_drums_in' as a virtual MIDI port for input.
INFO:tensorflow:Opening 'magenta_clock' as a virtual MIDI port for input.
INFO:tensorflow:Opening 'magenta_out' as a virtual MIDI port for output.

Instructions:
Start playing  when you want to begin the call phrase.
When you want to end the call phrase, stop playing and wait one clock tick.
Once the response completes, the interface will wait for you to begin playing again to start a new call phrase.

To end the interaction, press CTRL-C.
INFO:tensorflow:Restoring parameters from /tmp/tmpAEHW5P/model.ckpt
Loaded 'rnn-nade_attn' generator bundle from file './pianoroll_rnn_nade.mag'.
WARNING:tensorflow:The saved meta_graph is possibly from an older release:
'model_variables' collection should be of type 'byte_list', but instead is of type 'node_list'.
INFO:tensorflow:Restoring parameters from /tmp/tmpd8bQ9A/model.ckpt
Loaded 'performance' generator bundle from file './performance.mag'.
INFO:tensorflow:Opening 'magenta_in' as a virtual MIDI port for input.
INFO:tensorflow:Opening 'magenta_out' as a virtual MIDI port for output.

Instructions:
Start playing  when you want to begin the call phrase.
When you want to end the call phrase, stop playing and wait one clock tick.
Once the response completes, the interface will wait for you to begin playing again to start a new call phrase.

To end the interaction, press CTRL-C.
127.0.0.1 - - [01/Feb/2018 01:04:13] "GET /build/1.js HTTP/1.1" 200 -
open: opción incorrecta -- «a»
Uso: open [OPCIONES] -- comando

Please help @adarob @cghawthorne :pray: :smiley:

[ghost]

I get this js error in Magenta.js i when I press a key from the keyboard (both in Chrome and Firefox):

And this when I change from Drums to Piano again:

[JunShern]

@turingnianos did you manage to get this solved? I am facing the same issues.

[lmartak]

@turingnianos @JunShern me too!

I also noticed that magenta_piano_in interface fails to launch as I wanted to test it with my Novation Impulse (hardware MIDI keyboard) and custom audio fonts synthesized with FluidSynth.

This also corresponds to following behavior --> when Piano instrument is chosen, there is warning message in console on each key press and key release event I do physically on the keyboard, while with Drums, there is no activity in the console (errors again correspond to GUI piano key click/release as reported by @turingnianos ).

But still nor audible neither visible reaction except these in console were observed.

Also worth mention might be that with this conda env setup on this same machine I am able to run Call-Response interaction with single NN performer in terminal using magenta midi interface if it helps to exclude some possible issues from consideration..

[lmartak]

After resolving original symptoms with #30 I have a follow-up issues (unless someone calls for a separate issue I'll continue here since the issue name still applies).

I installed magenta-gpu in separate conda envs magenta for python3.5 and magenta2 for python2.7.

Since python 3 version exhibited crashing on an Import Error, I work with python 2 for now.

ImportError: No module named 'Queue'

After runing sh RUN_DEMO.sh in magenta2 env, when I open localhost:8080 in Chrome, my GPU load raises from ~0% to ~50% and stays there no matter what I do in the app. I can now hear and see what I played, but I can't get any Response to any of my Calls.

I also attach the current behavior as a short YT video for quick demonstration.

I'd love to hear if someone gets a different behavior. @adarob any clues or suggestions where to look at next?

Cheers!

EDIT: Additional observation - when RUN_DEMO.sh is launched with qjackctl and fluidsynth running in background it seems that note off events don't make it to the web app (this holds for all note events generated by magenta models as well as by player through UI and external MIDI controller). After some experimental routing we can hear note events generated by magenta models and we hear them start and finish. In web app we only see them start, never finish, like there was a sustain pedal being held. Again some video evidence.

[adarob]

I have a fix for the Python3 issue that I'll push shortly. It would be important to see what's happening in the terminal you started RUN_DEMO.sh in. It should be logging messages that it is receiving clock signals.

[adarob]

https://github.com/tensorflow/magenta/pull/1117 will fix the python3 issues once the pip package is updated.

[lmartak]

For the no Response case, I attach terminal output of RUN_DEMO.sh in this GIST. I don't see any logs even when I play Calls and interact with the UI.