androidthings / sample-googleassistant

Google Assistant API sample for Android Things
Apache License 2.0
467 stars 147 forks source link
aiyprojects android-things google-assistant-api

Google Assistant SDK for devices - Android Things

This sample shows how to call the Google Assistant Service from Android Things using gRPC. It records a spoken request from the connected microphones, sends it to the Google Assistant API and plays back the Assistant's spoken response on the connected speaker.

Note: The Android Things Console will be turned down for non-commercial use on January 5, 2022. For more details, see the FAQ page.

Pre-requisites

Run the sample

  1. Create or open a project in the Actions Console
  2. Follow the instructions to register a device model
    1. Download client_secret_XXXX.json
    2. Configure the OAuth consent screen for your project
  3. Install the google-oauthlib-tool in a Python 3 virtual environment:
python3 -m venv env
env/bin/python -m pip install --upgrade pip setuptools
env/bin/pip install --upgrade google-auth-oauthlib[tool]
source env/bin/activate
google-oauthlib-tool --client-secrets client_secret_XXXX.json \
                     --credentials app/src/main/res/raw/credentials.json \
                     --scope https://www.googleapis.com/auth/assistant-sdk-prototype \
                     --save
./gradlew assembleDebug
adb install -g app/build/outputs/apk/debug/app-debug.apk
./gradlew installDebug
adb shell am start com.example.androidthings.assistant/.AssistantActivity

Audio Configuration

By default the sample routes audio to the I2S Voice Hat on Raspberry Pi 3 and default audio on other boards (on-board Line out or HDMI/USB if connected).

You can change those mappings by changing the USE_VOICEHAT_I2S_DAC constant or replacing the audio configuration in the onCreate method of AssistantActivity with one of the following:

// Force using on-board Line out:
audioInputDevice = findAudioDevice(AudioManager.GET_DEVICES_INPUTS, AudioDeviceInfo.TYPE_BUILTIN_MIC);
audioOutputDevice = findAudioDevice(AudioManager.GET_DEVICES_OUTPUTS, AudioDeviceInfo.TYPE_BUILTIN_SPEAKER);

// Force using USB:
audioInputDevice = findAudioDevice(AudioManager.GET_DEVICES_INPUTS, AudioDeviceInfo.TYPE_USB_DEVICE);
audioOutputDevice = findAudioDevice(AudioManager.GET_DEVICES_OUTPUTS, AudioDeviceInfo.TYPE_USB_DEVICE);

// Force using I2S:
audioInputDevice = findAudioDevice(AudioManager.GET_DEVICES_INPUTS, AudioDeviceInfo.TYPE_BUS);
audioOutputDevice = findAudioDevice(AudioManager.GET_DEVICES_OUTPUTS, AudioDeviceInfo.TYPE_BUS);

Device Actions

With Device Actions, you can control hardware connected to your device. In this sample, you can turn on and off the LED attached to your Android Things board.

Follow the guide here to learn how to register your device.

private static final String DEVICE_MODEL_ID = "my-device-model-id";
private static final String DEVICE_INSTANCE_ID = "my-device-instance-id";
mEmbeddedAssistant = new EmbeddedAssistant.Builder()
    ...
    .setConversationCallback(new ConversationCallback() {
        ...
        @Override
        public void onDeviceAction(String intentName, JSONObject parameters) {
            // Check the type of command
            if (intentName.equals("action.devices.commands.OnOff")) {
                try {
                    boolean turnOn = parameters.getBoolean("on");
                    mLed.setValue(turnOn);
                } catch (JSONException e) {
                    Log.e(TAG, "Cannot get value of command", e);
                } catch (IOException e) {
                    Log.e(TAG, "Cannot set value of LED", e);
                }
            }
        }
    }
    ...
...

Try it:

The LED should change states based on your command.

Enable auto-launch behavior

This sample app is currently configured to launch only when deployed from your development machine. To enable the main activity to launch automatically on boot, add the following intent-filter to the app's manifest file:

<activity ...>

    <intent-filter>
        <action android:name="android.intent.action.MAIN"/>
        <category android:name="android.intent.category.HOME"/>
        <category android:name="android.intent.category.DEFAULT"/>
    </intent-filter>

</activity>

License

Copyright 2017 The Android Open Source Project, Inc.

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to you under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.