search

antmicro-labs / voice-assistant

Apache License 2.0
0 stars 1 forks source link

Make use of the "Micro speech" example #2

Closed plonajakub closed 4 years ago

plonajakub commented 4 years ago

Standalone TF example "Micro speech" compiled for the board will have no real use case. There is a need for feeding the example with audio data, taking outputs from the TF model and visualizing them on the board - for example by setting the LEDs.

The goal is to write a small app capable of driving the "Micro speech" TF model.

Investigation of this topic is delegated to @Marzecki and @FilipKadziolka .

Reference: TF Lite for microcontrollers

Marzecki commented 4 years ago

Added a new branch with command_responder changed so that it would turn on LEDs corresponding to the command that has been recognized.

kgugala commented 4 years ago

You don't have to keep everything in a single repository. We can create more here, just let us know what you need.

Marzecki commented 4 years ago

Is there any example of controlling LEDs on the Arty A7-35T board? Because I can't find any.

kgugala commented 4 years ago

You'd need to extend the design with a GPIO controller and connect the outputs to LED's

szymon-filipiak commented 4 years ago

Ok, so we did some research and could not find exactly how to reference LEDs on the board.

We've looked at how you do it in the accelerometer example in this file. You are trying to get the sensor by calling sensor = device_get_binding("accel-0");. We are wondering where to find a list of device names (like "accel-0"; we assume this to be an accelerometer sensor alias).

Will calling the device_get_binding() method with proper arguments be enough to reference LEDs and GPIO?

Another thing we've found is some names of components in litex arty.py. Are those possibly the names of the devices that we are looking for?

We want to adopt this file. LED0 macro value is by default equal "led0".

@kgugala or @rw1nkler Can you help us with this and let us know if our reasoning is correct?

Thank you in advance.

kgugala commented 4 years ago

Can we schedule a quick meeting (telco, given the circumstances)? It will be way faster if we discuss this.

Please propose the slots when we can do it. It doesn't have to be with the whole team (this way it will be easier to find a slot)

szymon-filipiak commented 4 years ago

Sure, it's possible. I and @plonajakub have time tomorrow after 3 p.m. until 6 p.m. Maby some of my colleagues have time earlier, but I can't reach them at the moment. Does this timebox suit you?

kgugala commented 4 years ago

I scheduled a telco for 4:30 PM. You should get the invitations. Please add any other team members that should be on the call

szymon-filipiak commented 4 years ago

Some of us have executed the steps in the litex-buildenv tutorial. Can we schedule the next meeting, as discussed yesterday? We are available in the evening. @kgugala

kgugala commented 4 years ago

I missed your yesterday's update. We can meet today in the afternoon, is that OK?

szymon-filipiak commented 4 years ago

Sure, can we meet after 17.00?

kgugala commented 4 years ago

ok, I'll send you the invitations

szymon-filipiak commented 4 years ago

I've successfully booted a custom app that switches LEDs state in the interval of 1s onto the board. I will take over the issue from now on. @FilipKadziolka @Marzecki Your assistance will be appreciated. ^^

szymon-filipiak commented 4 years ago

After some trouble, we have built and run TensorFlow microspeech on the board. 🥳

Great thanks for everyone involved!

The app is running with some troubles (time measured by TF is weird, needs further investigation) but the board responds to prepared voice commands.