MQTT as Endpoint

[longunmin]

Would it be possible to have an MQTT endpoint be an option, instead of Home Assistant or REST?

[kristiankielhofner]

It's not impossible. Can you provide an idea of what you would be looking for?

I assume it would be something like publishing the STT transcript to an MQTT topic or similar?

[longunmin]

Exactly, basically I want to pull it into Node Red and then be able to do what I want. Assist takes alot of work to set up simple functionality (I.e. timers or the like) that could be done quickly and simply with json/filter nodes. I know HA plans on making a lot of this stuff boiler plate in the future, but that's just one example. Once I'm able to get the stt into an engine of my choosing (mqtt is pretty universal) I can move alot quicker.

Setting the payload as whatever speech Willow hears, I think opens up a world of possibilities

[hamishcunningham]

If you don't mind a bit of python fiddling, then if you configure the exiting REST endpoint to point at a flask server something like this will pick out the STT results so you can post them to your MQTT broker:

from flask import Flask, request
import asyncio, websockets, ssl, json, os
app = Flask(__name__)

@app.route('/mqtt4willow', methods=['POST', 'GET'])
def rest4willow():
    print(f"request: {request}")
    jdata = request.json
    text = jdata.get('text')
    # post to MQTT here

[longunmin]

If you don't mind a bit of python fiddling, then if you configure the exiting REST endpoint to point at a flask server something like this will pick out the STT results so you can post them to your MQTT broker:

from flask import Flask, request
import asyncio, websockets, ssl, json, os
app = Flask(__name__)

@app.route('/mqtt4willow', methods=['POST', 'GET'])
def rest4willow():
    print(f"request: {request}")
    jdata = request.json
    text = jdata.get('text')
    # post to MQTT here

I tried the above code and I see the post come in on flask, but I'm getting an error message from willow

[hamishcunningham]

Not sure what your error is but it might be that the method isn't finished and needs to return a value, eg the TTS string. You also need to fill in the mqtt post.

[longunmin]

Not sure what your error is but it might be that the method isn't finished and needs to return a value, eg the TTS string. You also need to fill in the mqtt post.

I think its getting an error (Response invalid json data) because its not getting a proper response, but also, do you know how I can view the debugger and see what willow is sending? Then I can update flask to update the payload with text

[longunmin]

So if I send the following command to my flask app, it sends the text via mqtt. But if I try via willow, i get an error in willow and the message doesn't come through. So I must have the wrong json format, but can't find the proper format

curl -X POST -H "Content-Type: application/json" -d '{ "input": { "text": "turn on basement table lamp" }, }' http://192.x.x.x:5000/publish

[nikito]

What error are you seeing in willow?

[longunmin]

What error are you seeing in willow?

the only error i get from willow that i can see is on my esp box, and its just Response invalid json data

[nikito]

I think that's the response from the server, maybe the server doesn't like what willow is sending?

[longunmin]

I think that's the response from the server, maybe the server doesn't like what willow is sending?

Do you know how I can see what willow is sending? I'm more concerned with that than the feedback from willow at this point

[nikito]

You could hook the esp up to the serial monitor as described here: https://heywillow.io/development/#set-serial-port and here: https://heywillow.io/development/#start-serial-monitor Alternatively you could check the logs at the server side to see what it received, or add logging to show what it didn't like?

[longunmin]

If you don't mind a bit of python fiddling, then if you configure the exiting REST endpoint to point at a flask server something like this will pick out the STT results so you can post them to your MQTT broker:

from flask import Flask, request
import asyncio, websockets, ssl, json, os
app = Flask(__name__)

@app.route('/mqtt4willow', methods=['POST', 'GET'])
def rest4willow():
    print(f"request: {request}")
    jdata = request.json
    text = jdata.get('text')
    # post to MQTT here

FYI, here is the eventual flask app that got me working, in case anyone wants to go this route. I still think an MQTT endpoint option would be good, but this works as a workaround for now. Thanks everyone!

from flask import Flask, request, jsonify
import paho.mqtt.publish as publish

app = Flask(__name__)

# Define your MQTT broker information
MQTT_BROKER = "x.x.x.x"  # Replace with your MQTT broker address
MQTT_PORT = 1883  # Replace with your MQTT broker's port

@app.route('/publish', methods=['POST'])
def publish_to_mqtt():

    print(request.get_json())
    try:
        data = request.get_json()

        if not data or 'text' not in data:
            return jsonify({'error': 'Invalid JSON data or missing "text" field'}), 400

        text = data['text']
        if not text:
            return jsonify({'error': 'Missing "text" field in the "input" object'}), 400

        topic = "/willow"  # You can modify this to set a default topic
        message = text

        # Debugging: Print the incoming message
        print(f"Incoming Message: {message}")

        publish.single(topic, message, hostname=MQTT_BROKER, port=MQTT_PORT)

        response_data = {'Okay': 'Command Received'}
        #response_data = {'Command Received'}
        return jsonify(response_data), 200

    except Exception as e:
        return jsonify({'error': str(e)}), 500

if __name__ == '__main__':
   app.run(host='x.x.x.x', port=5000)

[hamishcunningham]

Glad you got it working! I think the error you were seeing was to do with the flask server route method not returning anything, so now your "return jsonify..." call fixes it I guess. To add this type of thing as a built-in facility would mean something like:

• adding willow/main/endpoint/mqtt.c with roughly what's in rest.c plus the equivalent of your "publish.single" call
• adding config options to allow WAS to maintain the MQTT broker and port data

Very doable and would be useful :)

[stintel]

FYI, as of toverainc/willow-application-server@20f69d7822fa9537ece12f9df983b186d2944463 we started working on command endpoint support in WAS. This is included in the WAS 0.1.0-rc.1 release, and though currently still marked experimental, it is unlikely we will add new endpoints to Willow, or accept PRs to do so. You might want to have a look at adding an MQTT endpoint to WAS, instead of doing it in the Willow C code.

[jerome83136]

Would it be possible to have an MQTT endpoint be an option, instead of Home Assistant or REST?

Hello, I also would love having MQTT endpoint implemented in Willow. And the reply from @stintel seems to confirm if such implementation occurs; it will be on WAS component.

In the meantime; maybe could you address your use case by making Willow talking to an MQTT implementation that natively supports REST API endpoints ? Maybe this one could be interesting: https://docs.emqx.com/en/cloud/latest/api/publish_v5.html

Hope this will be helpful :) Best regards

[stintel]

So I started on an MQTT endpoint for WAS: toverainc/willow-application-server@63fa17c5f21dc32a196259bde68120b42001e92d

If you want to test it you could use this container image: ghcr.io/toverainc/willow-application-server:feature-mqtt_endpoint. As this does not include the UI, you'll have to run it yourself.

[stintel]

Moved the issue as we will implement this in WAS.

[longunmin]

So I started on an MQTT endpoint for WAS: 63fa17c

If you want to test it you could use this container image: ghcr.io/toverainc/willow-application-server:feature-mqtt_endpoint. As this does not include the UI, you'll have to run it yourself.

Thanks! This is amazing! Great work