Closed createcandle closed 5 years ago
Since it uses a notifier, it would be best to set that to a minimum of 0.9.
Very cool work! However, as previously discussed, we're currently working on pulling the voice add-on from our main add-on list, specifically because of all of the system dependencies. This would be a great first add-on for your own new add-on list.
If you'd like help with that, please let me know. Essentially, you'll want to do the following:
~/.mozilla-iot/config/local.json
. For example:{
"addonManager": {
"listUrls": [
"https://api.mozilla-iot.org:8443/addons",
"https://<yourdomain>/addons"
]
}
}
Also, just a tip. This add-on would be "linux-arm"
, not "linux-arm64"
. Raspbian is 32-bit.
That's really sad. I want it to be as easy as possible for my users to get started. Creating an alternative image of the Gateway is yet something else I would need to explore, and it would cut off a lot of potential users.
Can you explain in more detail why this direction is being taken? E.g.
I understand wanting to make it easy for users to do what they want, which is why we at least added the config file option. In the future, we could potentially add some UI to make that easier, but that's not currently planned.
- What were the things weighed in that decision?
- Would it be possible to make it a 'use at your own risk' type of thing, where users have to set that they want to see "add-ons that also install things" in the add-on list?
Yes. That was the point of adding the config option. Users have to explicitly opt-in to doing things that we don't necessarily support. Modifying a config file will be a conscious, deliberate action a user will have to take.
- Would it be possible to allow Raspberry Pi users a bit more room?
All users are afforded the same level of configurability. We're not preventing anyone from doing what they want. We just don't want to promote add-ons which modify the entire system
- Are add-ons still allowed to have binaries if they don't install anything to the rest of the system but contain everything within the add-on folder?
Yes. In many times, that's a hard requirement, as with the Z-Wave adapter.
Thanks for the explanations.
I understand you may want to guarantee some interoperability for a selection of add-ons. But I'd like to make a very American point: why not let users decide if they want to be exposed to that risk by checking a checkbox or some other 'explicit act'?
Currently the add-on system has shifted from total freedom to you have to manipulate core files. That's quite a big jump. I guess I'm advocating for something in the middle?
In the future, we could potentially add some UI to make that easier, but that's not currently planned.
Perhaps it would it be possible to put that higher on the list?
I am reminded of how Synology does it. They allow you to explicitly add links to third party lists of sources, and it has allowed the community to build on top of the Synology platform without burdening Synology.
Alternatively, would you allow for an add-on that, once installed, makes it easier to install a wider range of add-ons? Eg. an add-on that would run an add-on proxy and that would let users easily append new URLs to the listUrls"
? That would very much be a conscious 'your warranty is now void`act.
Some other practical questions here:
.deb
files. But it doesn't seem like I could point it to local shared libraries.All users are afforded the same level of configurability. We're not preventing anyone from doing what they want. We just don't want to promote add-ons which modify the entire system
I get your point, and I can understand that the focus on the router platform would make a focus on more security and predictability something to care about. But I don't fully agree with your statement. "preventing it" has shades of grey. While technically it's still possible, the user experience for the average geek has shifted quite radically, and quite binary.
as we're moving further down the OpenWrt road, add-ons that do Raspbian-specific things or assume Raspberry Pi hardware are becoming less desirable
It seems to me that the Raspberry Pi and a router are very divergent platforms. Not so much in hardware, but in philosophy. Creating a way for users to say "I want to have more choice" on the world's most popular maker platform is something I would like to plead wholeheartedly for. I feel that the WebThings Gateway could support both philosophies, and I hope it will.
In the future, we could potentially add some UI to make that easier, but that's not currently planned.
Perhaps it would it be possible to put that higher on the list?
@benfrancis do you want to weigh in on this? I know you previously objected to it.
Alternatively, would you allow for an add-on that, once installed, makes it easier to install a wider range of add-ons? Eg. an add-on that would run an add-on proxy and that would let users easily append new URLs to the
listUrls"
? That would very much be a conscious 'your warranty is now void`act.
That seems like a misuse of add-ons, honestly.
* Would you allow add-ons that don't install binaries but that do install shared libraries? I've looked into it, and Snips could happily be ran as just a binary that I extract from the `.deb` files. But it doesn't seem like I could point it to local shared libraries.
Not if it's still installing system-wide libraries. There are ways to use local shared libraries. That's what the Z-Wave adapter does now, rather than installing libopenzwave on the system.
* Would it be possible to have a process that would allow the community to appeal for the inclusion of certain programs into the (Raspbian) image? E.g. Mosquitto or a default mediaplayer?
By all means. File an issue or open a PR on the gateway and we can discuss.
It seems to me that the Raspberry Pi and a router are very divergent platforms. Not so much in hardware, but in philosophy. Creating a way for users to say "I want to have more choice" on the world's most popular maker platform is something I would like to plead wholeheartedly for. I feel that the WebThings Gateway could support both philosophies, and I hope it will.
I think I've mostly addressed this point in my various statements above. However, I'd also say that yes, the Raspberry Pi is a maker platform, and given that it is, most of its users are accustomed to fiddling around on the command line and such. It shouldn't be much of a stretch for them to modify a single text file.
On another note...
We're trying to figure out our add-on system right now so that the whole "everything is a device" thing can go away. That's why we added notifiers. We're trying to target very specific add-on points, rather than allowing developers to do arbitrary things on the system under the guise of being a "device".
We're not a generic Linux package manager. This voice add-on could easily be packaged as a separate, installable application for Raspbian, which is what we're advocating for with the existing voice add-on. There's really no reason for it to be a gateway add-on. If there are specific things that would make this easier, please file issues.
I now know more about Linux packages than I'd care to, but here it is: a version that doesn't install anything.
https://www.createcandle.com/assistant/voco-0.0.2.tgz 455cf4955cbc14637c773042d341da2bedceb2d93a5b41465ceb20403a18e22e
It does sacrifice some things:
/usr/share
directory. It was hard coded into it, and I didn't know how to fool it. It now uses eSpeak, which is a bit more robotic, but was fully configurable. It has now been replaced by NanoTTS, which is very similar to the original voice that came with Snips.on
or off
, it would ask you to clarify. It's a todo.There are also some upshots.
Added a small fix.
Ok, I'm trying to play with this and noticed a couple things.
2019-08-15 19:37:34.947 INFO : voco: In America/Anchorage the current time is 2019-08-15 18:37:31.516429-08:00
2019-08-15 19:37:34.948 INFO : voco: With your current localization settings, your computer will tell you it is now 2019-08-15 19:37:31.516416
Thanks.
I've just uploaded version 0.0.3. This should actually work. A lot has changed.
snips.tar
. It contains the local version of Mosquitto and Snips.About the time: that first line still required you to take off the -08:00 at the end yourself. So that would make it 10am.
The add-on is no longer starting up, with the new version. See below:
2019-08-15 19:56:55.450 ERROR : voco: chmod: cannot access '/home/pi/.mozilla-iot/addons/voco/speak.sh': No such file or directory
2019-08-15 19:57:03.111 INFO : voco: numid=3,iface=MIXER,name='PCM Playback Route'
2019-08-15 19:57:03.113 INFO : voco: ; type=INTEGER,access=rw------,values=1,min=0,max=2,step=0
2019-08-15 19:57:03.115 INFO : voco: : values=1
2019-08-15 19:57:03.173 ERROR : voco: /home/pi/.mozilla-iot/addons/voco/snips/mosquitto: error while loading shared libraries: libuv.so.1: cannot open shared object file: No such file or directory
About the time: that first line still required you to take off the -08:00 at the end yourself. So that would make it 10am.
That's not how timestamps work. The issue is that:
usernow = self.user_timezone.localize(datetime.utcnow())
should actually be
usernow = datetime.now(tz=self.user_timezone)
I'm uploading a new version (still 0.0.3) with the changes you proposed. It will be available in 2 minutes.
507b9209f45882f9df365e3cb462b9d42fafffddea20b5c4149820aa1c0bd676
Things start up now. New issues below.
2019-08-15 21:00:35.249 INFO : voco: Error settings Snips state: 'VocoAdapter' object has no attribute 'hotword_process'
2019-08-15 21:00:35.250 INFO : voco: property -> update
2019-08-15 21:00:35.251 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"setProperty","data":{"deviceId":"voco","propertyName":"listening","propertyValue":false,"adapterId":"voco-adapter","pluginId":"voco-adapter"}}'
2019-08-15 21:00:47.718 INFO : voco: set_value called for listening
2019-08-15 21:00:47.720 INFO : voco: CHANGING STATE
2019-08-15 21:00:47.721 INFO : voco: stopping. Terminating hotword process.
2019-08-15 21:00:47.722 INFO : voco: Error while terminating hotword process
2019-08-15 21:37:32.475 ERROR : voco: INFO:snips_hotword_hermes : Hotword detected: site_id:default model:hey_snips (94ms ago)
2019-08-15 21:37:32.477 ERROR : voco: INFO:snips_hotword_hermes : Hotword detected: site_id:default model:hey_snips (94ms ago)
2019-08-15 21:37:32.478 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn off Hotword
2019-08-15 21:37:32.482 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: The session has been created because the site "default" is free
2019-08-15 21:37:32.483 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [59766755-f0c1-43a8-80be-c3b9ca5bf5ae]: The session has been dropped because the site "default" is busy
2019-08-15 21:37:32.484 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn off Hotword
2019-08-15 21:37:32.485 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: The session has been created because the site "default" is free
2019-08-15 21:37:32.486 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [85e2beac-4bbb-4123-831a-ba9f23af93d0]: The session has been dropped because the site "default" is busy
2019-08-15 21:37:32.487 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn off Hotword
2019-08-15 21:37:32.488 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [318e265c-8024-430f-a67b-cd0ce5dacf7d]: The session has been created because the site "default" is free
2019-08-15 21:37:32.488 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [0ecdcf2a-1c2f-485d-b348-67719ad8feca]: The session has been dropped because the site "default" is busy
2019-08-15 21:37:32.497 ERROR : voco: Playing WAVE '/home/pi/.mozilla-iot/addons/voco/snips/start_of_input.wav' : Signed 16 bit Little Endian, Rate 22050 Hz, Stereo
2019-08-15 21:37:32.521 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.522 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.523 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.524 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.525 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.526 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default stopped.
2019-08-15 21:37:32.572 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:32.574 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:32.575 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:32.576 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:32.578 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:32.579 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:32.580 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:32.581 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:32.627 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:32.628 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
2019-08-15 21:37:32.629 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:32.635 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
2019-08-15 21:37:32.637 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:32.638 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
2019-08-15 21:37:32.639 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:32.640 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
2019-08-15 21:37:33.079 ERROR : voco: Playing WAVE '/home/pi/.mozilla-iot/addons/voco/snips/start_of_input.wav' : Signed 16 bit Little Endian, Rate 22050 Hz, Stereo
2019-08-15 21:37:35.238 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.610 endpoint detected (rule:4) frame:155 samples:39680 signal_time:2.48 rtf:0.492
2019-08-15 21:37:35.241 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.613 endpoint detected (rule:4) frame:155 samples:39680 signal_time:2.48 rtf:0.483
2019-08-15 21:37:35.243 ERROR : voco: INFO:snips_asr_lib::asr : Source thread stop on push: "Disconnected(..)"
2019-08-15 21:37:35.245 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.619 capture ended
2019-08-15 21:37:35.249 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.622 endpoint detected (rule:4) frame:155 samples:39680 signal_time:2.48 rtf:0.494
2019-08-15 21:37:35.252 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.624 endpoint detected (rule:4) frame:155 samples:39680 signal_time:2.48 rtf:0.509
2019-08-15 21:37:35.255 ERROR : voco: INFO:snips_asr_lib::asr : Source thread stop on push: "Disconnected(..)"
2019-08-15 21:37:35.273 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.626 decoder finalized
2019-08-15 21:37:35.278 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.627 capture ended
2019-08-15 21:37:35.279 ERROR : voco: INFO:snips_asr_lib::asr : Source thread stop on push: "Disconnected(..)"
2019-08-15 21:37:35.281 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.631 capture ended
2019-08-15 21:37:35.284 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.637 decoder finalized
2019-08-15 21:37:35.301 ERROR : voco: INFO:snips_asr_lib::asr : Source thread stop on push: "Disconnected(..)"
2019-08-15 21:37:35.303 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.632 capture ended
2019-08-15 21:37:35.305 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.642 decoder finalized
2019-08-15 21:37:35.306 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.656 lookup and post-processing done
2019-08-15 21:37:35.308 ERROR : voco: INFO:snips_asr_lib::asr : decoded: [Recognition { decoded_string: "what time is it", likelihood: 1.0, tokens: Some([Token { value: "what", confidence: 1.0, time: (0.0, 1.35), range: 0..4 }, Token { value: "time", confidence: 1.0, time: (1.35, 1.5), range: 5..9 }, Token { value: "is", confidence: 1.0, time: (1.5, 1.68), range: 10..12 }, Token { value: "it", confidence: 1.0, time: (1.68, 2.31), range: 13..15 }]) }]
2019-08-15 21:37:35.309 ERROR : voco: INFO:snips_asr_hermes::handler : Publishing the recognition
2019-08-15 21:37:35.321 ERROR : voco: INFO:snips_asr_hermes::handler : Preparing decoder
2019-08-15 21:37:35.323 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.326 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.658 decoder finalized
2019-08-15 21:37:35.327 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.336 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.338 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.349 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.700 lookup and post-processing done
2019-08-15 21:37:35.350 ERROR : voco: INFO:snips_asr_lib::asr : decoded: [Recognition { decoded_string: "what time is it", likelihood: 1.0, tokens: Some([Token { value: "what", confidence: 1.0, time: (0.0, 1.35), range: 0..4 }, Token { value: "time", confidence: 1.0, time: (1.35, 1.5), range: 5..9 }, Token { value: "is", confidence: 1.0, time: (1.5, 1.68), range: 10..12 }, Token { value: "it", confidence: 1.0, time: (1.68, 2.31), range: 13..15 }]) }]
2019-08-15 21:37:35.351 ERROR : voco: INFO:snips_asr_hermes::handler : Publishing the recognition
2019-08-15 21:37:35.352 ERROR : voco: INFO:snips_asr_hermes::handler : Preparing decoder
2019-08-15 21:37:35.353 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.355 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.356 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.358 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.359 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.364 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.718 lookup and post-processing done
2019-08-15 21:37:35.365 ERROR : voco: INFO:snips_asr_lib::asr : decoded: [Recognition { decoded_string: "what time is it", likelihood: 1.0, tokens: Some([Token { value: "what", confidence: 1.0, time: (0.0, 1.35), range: 0..4 }, Token { value: "time", confidence: 1.0, time: (1.35, 1.5), range: 5..9 }, Token { value: "is", confidence: 1.0, time: (1.5, 1.68), range: 10..12 }, Token { value: "it", confidence: 1.0, time: (1.68, 2.31), range: 13..15 }]) }]
2019-08-15 21:37:35.366 ERROR : voco: INFO:snips_asr_hermes::handler : Publishing the recognition
2019-08-15 21:37:35.367 ERROR : voco: INFO:snips_asr_hermes::handler : Preparing decoder
2019-08-15 21:37:35.369 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.370 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.371 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.372 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.373 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.375 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: fails: Event not supported by Timer { anchor: Instant { tv_sec: 2396, tv_nsec: 840650032 }, timeout: 5s }, for response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.376 ERROR : voco:
2019-08-15 21:37:35.377 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: closing session which was active: Error { error: "Event not supported by Timer { anchor: Instant { tv_sec: 2396, tv_nsec: 840650032 }, timeout: 5s }, for response: AsrCapture(AsrPayload { text: \"what time is it\", tokens: Some([AsrTokenPayload { value: \"what\", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: \"time\", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: \"is\", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: \"it\", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })\n" }
2019-08-15 21:37:35.378 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn on Hotword
2019-08-15 21:37:35.381 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.382 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.384 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.385 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.386 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.387 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.389 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: NluIntentParsed(IntentPayload { input: "what time is it", asr_tokens: None, asr_confidence: None, intent: NluIntentClassifierResultPayload { intent_name: "createcandle:get_time", confidence_score: 1.0 }, slots: [] })
2019-08-15 21:37:35.390 ERROR : voco: INFO:snips_asr_lib::asr : T0+2.719 lookup and post-processing done
2019-08-15 21:37:35.391 ERROR : voco: INFO:snips_asr_lib::asr : decoded: [Recognition { decoded_string: "what time is it", likelihood: 1.0, tokens: Some([Token { value: "what", confidence: 1.0, time: (0.0, 1.35), range: 0..4 }, Token { value: "time", confidence: 1.0, time: (1.35, 1.5), range: 5..9 }, Token { value: "is", confidence: 1.0, time: (1.5, 1.68), range: 10..12 }, Token { value: "it", confidence: 1.0, time: (1.68, 2.31), range: 13..15 }]) }]
2019-08-15 21:37:35.392 ERROR : voco: INFO:snips_asr_hermes::handler : Publishing the recognition
2019-08-15 21:37:35.393 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.394 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.395 ERROR : voco: INFO:snips_asr_hermes::handler : Preparing decoder
2019-08-15 21:37:35.398 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: fails: Event not supported by Timer { anchor: Instant { tv_sec: 2396, tv_nsec: 886020954 }, timeout: 5s }, for response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.400 ERROR : voco:
2019-08-15 21:37:35.401 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: closing session which was active: Error { error: "Event not supported by Timer { anchor: Instant { tv_sec: 2396, tv_nsec: 886020954 }, timeout: 5s }, for response: AsrCapture(AsrPayload { text: \"what time is it\", tokens: Some([AsrTokenPayload { value: \"what\", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: \"time\", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: \"is\", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: \"it\", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })\n" }
2019-08-15 21:37:35.402 ERROR : voco: INFO:snips_dialogue_lib::coordinator::coordinator: Site [default]: reset site -> turn off ASR and turn on Hotword
2019-08-15 21:37:35.404 ERROR : voco: WARN:snips_dialogue_lib::coordinator::coordinator: No Session found for this response: AsrCapture(AsrPayload { text: "what time is it", tokens: Some([AsrTokenPayload { value: "what", confidence: 1.0, range_start: 0, range_end: 4, time: AsrDecodingDurationPayload { start: 0.0, end: 1.35 } }, AsrTokenPayload { value: "time", confidence: 1.0, range_start: 5, range_end: 9, time: AsrDecodingDurationPayload { start: 1.35, end: 1.5 } }, AsrTokenPayload { value: "is", confidence: 1.0, range_start: 10, range_end: 12, time: AsrDecodingDurationPayload { start: 1.5, end: 1.68 } }, AsrTokenPayload { value: "it", confidence: 1.0, range_start: 13, range_end: 15, time: AsrDecodingDurationPayload { start: 1.68, end: 2.31 } }]), likelihood: 1.0, seconds: 2.0 })
2019-08-15 21:37:35.420 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default started.
2019-08-15 21:37:35.424 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default started.
2019-08-15 21:37:35.453 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default started.
2019-08-15 21:37:35.458 ERROR : voco: **error: multiple inputs
2019-08-15 21:37:35.465 ERROR : voco:
2019-08-15 21:37:35.466 ERROR : voco: Segmentation fault
2019-08-15 21:37:35.481 ERROR : voco: INFO:snips_hotword_lib::audio : Audio thread for site default started.
2019-08-15 21:37:35.547 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:35.548 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:35.564 ERROR : voco: **error: multiple inputs
2019-08-15 21:37:35.565 ERROR : voco:
2019-08-15 21:37:35.566 ERROR : voco: Segmentation fault
2019-08-15 21:37:35.570 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: session could not be closed because it no longer exists
2019-08-15 21:37:35.572 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: session could not be closed because it no longer exists
2019-08-15 21:37:35.574 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [273c06bc-ddd9-444b-8e8e-faa95cbf1352]: session could not be closed because it no longer exists
2019-08-15 21:37:35.611 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: session could not be closed because it no longer exists
2019-08-15 21:37:35.612 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: session could not be closed because it no longer exists
2019-08-15 21:37:35.614 ERROR : voco: ERROR:snips_dialogue_lib::coordinator::coordinator: Session [4a6ea633-9c5b-4a0d-97ba-bd105c1cd3b6]: session could not be closed because it no longer exists
2019-08-15 21:37:35.646 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:35.647 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
2019-08-15 21:37:35.682 ERROR : voco: INFO:snips_asr_hermes::handler : Listening at site id default
2019-08-15 21:37:35.683 ERROR : voco: INFO:snips_asr_hermes::handler : Listening
2019-08-15 21:37:35.811 ERROR : voco: INFO:snips_asr_lib::asr : T0 entered AsrRunner::run
2019-08-15 21:37:35.812 ERROR : voco: INFO:snips_asr_lib::asr : T0+0.000 capture started
For reference, my hardware setup is:
asound.conf
file. I still have to look into how I can then change the output. For now it's hard wired to the headphone jack. Similarly it's currently not possible to use the ReSpeaker Pi Hat anymore since it would require a driver installation.I'll create a fresh test image and do a test, and then create version 0.0.4.
It looks like you should be able to configure things via the ~/.asoundrc
file, rather than /etc/asound.conf
. While still not ideal, at least it doesn't require root privileges.
There are also some environment variables you can use.
I've found those pages before, and they didn't really get me any further. But I'll have another go.
Could you share your results of the aplay -l
and aplay -L
commands?
In the meantime I've finished version 0.0.4. Please let me know if you run into any issues. It's working well here. Download link is in the first post.
By the way, could it be that the "multiple inputs" error was the result of the add-on being an add-on
and a notifier
at the same time?
I've removed that functionality now, and everything is stable. It could be a coincidence.
Does the Gateway allow add-ons to be both at the same time? If so, what is the proper way of doing that? Creating a thread for the notifier might not be it :-)
Could you share your results of the
aplay -l
andaplay -L
commands?
pi@gateway:~ $ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
Subdevices: 7/7
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 IEC958/HDMI [bcm2835 IEC958/HDMI]
Subdevices: 1/1
Subdevice #0: subdevice #0
card 1: USB [Plantronics .Audio 628 USB], device 0: USB Audio [USB Audio]
Subdevices: 1/1
Subdevice #0: subdevice #0
pi@gateway:~ $ aplay -L
null
Discard all samples (playback) or generate zero samples (capture)
default:CARD=ALSA
bcm2835 ALSA, bcm2835 ALSA
Default Audio Device
sysdefault:CARD=ALSA
bcm2835 ALSA, bcm2835 ALSA
Default Audio Device
dmix:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample mixing device
dmix:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample mixing device
dsnoop:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct sample snooping device
dsnoop:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct sample snooping device
hw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Direct hardware device without any conversions
hw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Direct hardware device without any conversions
plughw:CARD=ALSA,DEV=0
bcm2835 ALSA, bcm2835 ALSA
Hardware device with all software conversions
plughw:CARD=ALSA,DEV=1
bcm2835 ALSA, bcm2835 IEC958/HDMI
Hardware device with all software conversions
default:CARD=USB
Plantronics .Audio 628 USB, USB Audio
Default Audio Device
sysdefault:CARD=USB
Plantronics .Audio 628 USB, USB Audio
Default Audio Device
front:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
Front speakers
surround21:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
2.1 Surround output to Front and Subwoofer speakers
surround40:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
4.0 Surround output to Front and Rear speakers
surround41:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
4.1 Surround output to Front, Rear and Subwoofer speakers
surround50:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
5.0 Surround output to Front, Center and Rear speakers
surround51:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
5.1 Surround output to Front, Center, Rear and Subwoofer speakers
surround71:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
7.1 Surround output to Front, Center, Side, Rear and Woofer speakers
iec958:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
IEC958 (S/PDIF) Digital Audio Output
dmix:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
Direct sample mixing device
dsnoop:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
Direct sample snooping device
hw:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
Direct hardware device without any conversions
plughw:CARD=USB,DEV=0
Plantronics .Audio 628 USB, USB Audio
Hardware device with all software conversions
Does the Gateway allow add-ons to be both at the same time? If so, what is the proper way of doing that? Creating a thread for the notifier might not be it :-)
Yes, you can do both. What I tend to do is create a device from the adapter, and then attach the notifier to the device. Here's an example (in JS): https://github.com/mozilla-iot/lg-tv-adapter/commit/e17be80bf0ffafeda16142ffa387d2efdb6e396f
I'll get back to testing this on Monday. Thanks for following up on all of this!
Could the notifier be attached to the adapter instead of the device?
Absolutely. It just made sense to do it the other way in my case.
I've created a new version that in theory should work with USB playback devices. I cannot test it here. If it works for you, then by all means consider this the version being proposed for the add-on list.
https://www.createcandle.com/assistant/voco-0.0.5.tgz d5ee6b0cb32ee934a2dbca80c474caae303d447deea5f172a60ed1f1723f77f3
This version also has re-enabled support for a speech notifier.
// I've cleaned up the code and was able to remove a dependency on Hermes. I've also tested it on a fresh image. I noticed it wouldn't cleanly shut down. That should be fixed now.
That's partially working now! A couple things to address:
However, I am unable to get rid off the rebooting effect. I had actually wanted to ask why the add-on seemed to get restarted after I disabled the add-on. It's definitely gateway related, in the sense that if I restart the gateway the add-on does finish. I suspect/hope it's some kind of internal feature to reboot the add-ons when they accidentally fail? And the fact that the add-on didn't close all it's children properly triggered that feature somehow?
The logs seem to indicate something like that:
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.787 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadAdapter","data":{"adapterId":"voco-adapter","pluginId":"voco-adapter"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.790 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadNotifier","data":{"notifierId":"voco-notifier","pluginId":"voco-notifier"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.791 INFO : voco: Shutting down Voco. Talk to you soon!#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.792 INFO : voco: Notifier: voco-notifier unloaded#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.794 ERROR : voco: 1566262564: Socket error on client snips-hotword|19924-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.795 ERROR : voco: 1566262564: Socket error on client snips-tts|19902-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.796 INFO : voco: Terminated the hotword#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.798 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.799 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadPlugin","data":{"pluginId":"voco-notifier"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.802 ERROR : voco: 1566262564: Socket error on client snips-audio-server|19904-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.803 ERROR : voco: 1566262564: Socket error on client snips-dialogue|19908-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.804 ERROR : voco: 1566262564: Socket error on client snips-asr|19912-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.811 ERROR : voco: 1566262564: Socket error on client snips-nlu|19916-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.813 ERROR : voco: 1566262564: Socket error on client snips-injection|19918-gateway-1, disconnecting.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.816 ERROR : voco: 1566262566: Client voco_mqtt_snips_client disconnected.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.817 ERROR : voco: 1566262566: Error in poll: Interrupted system call.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.818 ERROR : voco: 1566262566: mosquitto version 1.4.10 terminating#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.823 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.824 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.825 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.826 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.827 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.828 INFO : voco: Terminated mosquitto#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.829 INFO : voco: AddonManagerProxy: Error receiving message from socket: Bad file descriptor#033[22m#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:12.151 ERROR : voco: /home/pi/.mozilla-iot/addons/voco/lib/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:12.152 ERROR : voco: warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:12.249 INFO : Plugin: voco died, code = 0 restarting after 0#033[22m#033[39m
The tail log is a bit more clear about the order of things:
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.787 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadAdapter","data":{"adapterId":"voco-adapter","pluginId":"voco-adapter"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.790 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadNotifier","data":{"notifierId":"voco-notifier","pluginId":"voco-notifier"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.791 INFO : voco: Shutting down Voco. Talk to you soon!#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.792 INFO : voco: Notifier: voco-notifier unloaded#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.794 ERROR : voco: 1566262564: Socket error on client snips-hotword|19924-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.795 ERROR : voco: 1566262564: Socket error on client snips-tts|19902-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.796 INFO : voco: Terminated the hotword#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.798 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:04.799 INFO : voco: AddonMangerProxy: recv: b'{"messageType":"unloadPlugin","data":{"pluginId":"voco-notifier"}}'#033[22m#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.802 ERROR : voco: 1566262564: Socket error on client snips-audio-server|19904-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.803 ERROR : voco: 1566262564: Socket error on client snips-dialogue|19908-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.804 ERROR : voco: 1566262564: Socket error on client snips-asr|19912-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.811 ERROR : voco: 1566262564: Socket error on client snips-nlu|19916-gateway-1, disconnecting.#033[39m
Aug 20 01:56:04 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:04.813 ERROR : voco: 1566262564: Socket error on client snips-injection|19918-gateway-1, disconnecting.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.816 ERROR : voco: 1566262566: Client voco_mqtt_snips_client disconnected.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.817 ERROR : voco: 1566262566: Error in poll: Interrupted system call.#033[39m
Aug 20 01:56:06 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:06.818 ERROR : voco: 1566262566: mosquitto version 1.4.10 terminating#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.823 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.824 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.825 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.826 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.827 INFO : voco: Terminated Snips process#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.828 INFO : voco: Terminated mosquitto#033[22m#033[39m
Aug 20 01:56:10 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:10.829 INFO : voco: AddonManagerProxy: Error receiving message from socket: Bad file descriptor#033[22m#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:12.151 ERROR : voco: /home/pi/.mozilla-iot/addons/voco/lib/fuzzywuzzy/fuzz.py:11: UserWarning: Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[31m2019-08-20 01:56:12.152 ERROR : voco: warnings.warn('Using slow pure-python SequenceMatcher. Install python-Levenshtein to remove this warning')#033[39m
Aug 20 01:56:12 gateway run-app.sh[19442]: #033[37m#033[2m2019-08-20 01:56:12.249 INFO : Plugin: voco died, code = 0 restarting after 0#033[22m#033[39m
The gateway somehow interprets the shutting down as a crash, and restarts it. How can I avoid that?
self.close_proxy()
?os.setpgrp()
, or something else?Here's the code that tries to cleanly shut down all the started processes:
try:
self.hotword_process.terminate()
self.hotword_process.wait()
print("Terminated the hotword")
except Exception as ex:
print("Error terminating the hotword process: " + str(ex))
try:
for process in self.external_processes:
process.terminate()
process.wait()
print("Terminated Snips process")
except Exception as ex:
print("Error terminating the hotword process: " + str(ex))
sleep(2)
try:
self.mqtt_client.disconnect() # disconnect
self.mqtt_client.loop_stop() #Stop loop
#self.mqtt_client.loop.stop()
except Exception as ex:
print("Error cleanly closing Paho MQTT client: " + str(ex))
try:
self.mosquitto_process.terminate()
self.mosquitto_process.wait()
print("Terminated mosquitto")
except Exception as ex:
print("Error terminating the mosquitto process: " + str(ex))
I've tried many variations. With and without sleeps, with sys.exit
, and so forth.
Version 0.0.6: https://www.createcandle.com/assistant/voco-0.0.6.tgz 43d3d3c66ac7c66edf662f19c1999075f98a5b9c7e7e61b94a224ce2366d0b3b
Ok, the audio is working better now, but the process still isn't being killed off.
Could you update your git repo to the 0.0.6 version and I'll take a look?
Great!
I've uploaded 0.0.6 to Github: https://github.com/createcandle/voco/blob/fc607e0b5109f7d12de5e4cf02efefba300739df/pkg/voco_adapter.py#L1071
What do you mean by "the audio is working better"? Are there still issues?
Thank you so much!
From looking at your changes, was the adapter name `voco-adapter' to blame?
I'm impressed you spotted the type in speak.sh too, wow.
Version 0.0.7 is now ready. I have updated the first post.
From looking at your changes, was the adapter name `voco-adapter' to blame?
Yes.
I'll give this another look tomorrow and see about merging things.
Fantastic! Thank you for your help on fixing that last bug. I'm very happy to see that all the labour that went into this bore fruit!
@createcandle , Hope you can help me out. We're setting up a test environment with CandleSmartHome. Snips is able to answer questions like "What time is it" fine. However, when asking to turn a device on or off, I'm getting the response "Sorry, I couldn't do that.". Looking in the code, it looks like it cannot find properties (any specific requirments on how to build the MySensor devices?).
I've enabled debugging for voco, and get a 403 Forbidden response:
2019-11-19 09:47:01.309 INFO : voco: >> Hotword detected
2019-11-19 09:47:01.310 INFO : voco: MQTT msg.topic = hermes/hotword/toggleOff
2019-11-19 09:47:01.310 INFO : voco: MQTT msg.topic = hermes/intent/createcandle:set_state
2019-11-19 09:47:01.311 INFO : voco: -----------------------------------
2019-11-19 09:47:01.311 INFO : voco: >> other message.
2019-11-19 09:47:01.312 INFO : voco: message received {"sessionId":"7b322fe9-a80c-47a6-a01a-9eb3d3840a9a","customData":null,"siteId":"default","input":"turn on the fireplace","asrTokens":[[{"value":"turn","confidence":0.76295245,"rangeStart":0,"rangeEnd":4,"time":{"start":0.0,"end":1.3271115}},{"value":"on","confidence":1.0,"rangeStart":5,"rangeEnd":7,"time":{"start":1.3271115,"end":1.41}},{"value":"the","confidence":1.0,"rangeStart":8,"rangeEnd":11,"time":{"start":1.41,"end":1.77}},{"value":"fireplace","confidence":1.0,"rangeStart":12,"rangeEnd":21,"time":{"start":1.77,"end":2.9399998}}]],"asrConfidence":0.93459696,"intent":{"intentName":"createcandle:set_state","confidenceScore":1.0},"slots":[{"rawValue":"turn on","value":{"kind":"Custom","value":"on"},"alternatives":[],"range":{"start":0,"end":7},"entity":"boolean","slotName":"boolean","confidenceScore":0.8734715},{"rawValue":"fireplace","value":{"kind":"Custom","value":"fireplace"},"alternatives":[],"range":{"start":12,"end":21},"entity":"Thing","slotName":"thing","confidenceScore":1.0}],"alternatives":[{"intentName":"createcandle:get_boolean","confidenceScore":0.23128021,"slots":[{"rawValue":"fireplace","value":{"kind":"Custom","value":"fireplace"},"alternatives":[],"range":{"start":12,"end":21},"entity":"Thing","slotName":"thing","confidenceScore":1.0}]},{"intentName":"createcandle:set_value","confidenceScore":0.03733796,"slots":[{"rawValue":"fireplace","value":{"kind":"Custom","value":"fireplace"},"alternatives":[],"range":{"start":12,"end":21},"entity":"Thing","slotName":"thing","confidenceScore":1.0}]}]}
2019-11-19 09:47:01.313 INFO : voco: message topic = hermes/intent/createcandle:set_state
2019-11-19 09:47:01.314 INFO : voco:
2019-11-19 09:47:01.315 INFO : voco:
2019-11-19 09:47:01.316 INFO : voco: >>
2019-11-19 09:47:01.316 INFO : voco: >> incoming intent : createcandle:set_state
2019-11-19 09:47:01.317 INFO : voco: >> intent_message : turn on the fireplace
2019-11-19 09:47:01.317 INFO : voco: >> session ID : 7b322fe9-a80c-47a6-a01a-9eb3d3840a9a
2019-11-19 09:47:01.318 INFO : voco: >>
2019-11-19 09:47:01.318 INFO : voco: INCOMING SLOTS = {'sentence': 'turn on the fireplace', 'thing': 'fireplace', 'property': None, 'space': None, 'boolean': 'on', 'number': None, 'percentage': None, 'string': None, 'time_string': None, 'color': None, 'start_time': None, 'end_time': None, 'special_time': None, 'duration': None, 'period': None, 'timer_type': None, 'timer_last': None}
2019-11-19 09:47:01.319 INFO : voco: GET PATH = /things
2019-11-19 09:47:01.319 INFO : voco: API GET: 403, Forbidden
2019-11-19 09:47:01.320 INFO : voco: SCANNING THINGS
2019-11-19 09:47:01.320 INFO : voco: -> target thing title is: fireplace
2019-11-19 09:47:01.321 INFO : voco: -> No property title provided. Will try to get relevant properties.
2019-11-19 09:47:01.321 INFO : voco: Notice: thing had no title
2019-11-19 09:47:01.321 INFO : voco: Warning: thing had no name either. Skipping it.
2019-11-19 09:47:01.322 INFO : voco:
2019-11-19 09:47:01.322 INFO : voco: found 0 properties: []
I've looked (after enabling SSH) at the gateway.js (node), and found the place where the 403 is probably returned - is it correct that the request is done over HTTP and then redirected to HTTPS?
Anyway, I can't get any further - do you have any suggestions?
Thanks,
Jaap Holweg
It seems Voco is working ok, in that it correctly analyses your request and it it received by the add-on.
So the logical conclusion is that the add-on cannot see any device named "fireplace". It believes it has a list of devices, but when it goes over the list, a device doesn't have a title or a name, which is very odd.
It could be that there's an edge case where it tried to get the list, it failed, and somehow created a list object anyway. I can't really tell.
@flatsiedatsie : thanks for the suggestions - now it works! (and yes, I did create the authorization token). I re-installed from the mozilla image, and this time I did NOT create an external domain name (first time I did). Now, everything works like a charm. Apparently, something goes wrong when you enter an external domain name. @createcandle Maybe worth mentioning on the CandleSmartHome site (https://www.candlesmarthome.com/controller and https://www.candlesmarthome.com/voco-privacy-friendly-voice-control) that at this time you shouldn't fill in an external domain name.
Any suggestions on how to get this working WITH an external domain name filled in?
@mrstegeman How should an internal API request be changed if the external access option is enabled? Should it use HTTPS instead?
If so, do you have a suggestion for an easy way for an add-on to find out whether external access is enabled?
@flatsiedatsie You should be able to always make your requests to http://127.0.0.1:8080
. If external access is set up, you'll be redirected to https://127.0.0.1:4443
. At that point, you'll just need to accept the invalid certificate.
At that point, you'll just need to accept the invalid certificate.
Ah, is that something I have to do with the request
module?
Yes, you’ll pass verify=False
to your requests. See here: https://2.python-requests.org/en/master/user/advanced/#ssl-cert-verification
Interesting. From what I found online I'd have to actually link to a local cert file? https://stackoverflow.com/questions/30405867/how-to-get-python-requests-to-trust-a-self-signed-ssl-certificate
From that post:
I know that I can pass False to the verify parameter, like this:
r = requests.post(url, data=data, verify=False)
Hmm, it seems it's already in the current Voco code.
Also, http://127.0.0.1:8080
is the address used, so that also correct.
The add-on currently uses get
. Should that be post
instead?
That depends entirely on the request you're making. What's the request path, what are you trying to do? The method you linked to is pretty generic.
But could the current method generate a 403 error somehow? From what I've been reading it can sometimes happen if not all required things are inside the header.
If it looks OK to you, then what else could be causing the 403 error?
Huh, I found some code in the gateway I didn't know about! https://github.com/mozilla-iot/gateway/blob/master/src/app.js#L304-L311
So, given that, what I'd do is the following: When the add-on starts up, try making a request to http://127.0.0.1:8080/things
. If you get a 403 status, switch to HTTPS on 4443 (and make sure it works). If that still doesn't work, then do the following inside your adapter:
self.manager_proxy.send_error('some informative message')
sys.exit(1)
Interesting :-) If I understand correctly, that code basically prohibits get requests?
Will that code be removed from the next version of the Gateway?
self.manager_proxy.send_error('some informative message')
sys.exit(1)
So basically shut down the add-on if it can't connect to the API?
Interesting :-) If I understand correctly, that code basically prohibits get requests?
No, it prohibits non-GET requests, as well as any request with an Authorization
header. Essentially, it will allow a GET of the web UI.
Will that code be removed from the next version of the Gateway?
No, I see no immediate reason to. What would be better would be to send the port/SSL info to add-ons.
self.manager_proxy.send_error('some informative message') sys.exit(1)
So basically shut down the add-on if it can't connect to the API?
Yes. The error message gets propagated to the gateway log and the UI.
The Voco Add-on allows users to control their smart home with their voice. It builds on the adapter/ideas from Andre Natal (but at the same time changes quite a bit).
The full capabilities are described at: https://github.com/createcandle/voco
It has a built-in notifier, you may want to look at how that's been implemented.
Some questions:
"type": "adapter",
) assumes a creation can only be one type at a time? If it's useful, I could open an issue to better allow hybrids adapter+notifiers?0.9
.The
voco-X.X.X.tgz
file is hosted on the domain of the project for which this add-on was developed. Github doesn't allow me to upload files that are bigger than 25Mb.Designed for - and only tested on - a Raspberry Pi 4.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme, under grant agreement No 786641.