It does look like you can do a lot more via just the UART compared to the uart and the GPIOs to simulate button presses but i'm not sure if the additional features are worth re-factoring the Jarvis firmware to use just the uart or not.
Wow! I'll definitely take a look. It's already in the form of an ESPHome custom component, so that should make it pretty easy to integrate with this project.
AWESOME WORK!
I did something similar but the code is much less professional looking!
Not too long after buttoning everything up, I stumbled on the work of @maraid who managed to figure out how to send the "up/down/m{1,4}" messages over uart: https://github.com/maraid/Jarvis#technical-notes
It does look like you can do a lot more via just the UART compared to the uart and the GPIOs to simulate button presses but i'm not sure if the additional features are worth re-factoring the Jarvis firmware to use just the uart or not.
In any case, this looks fantastic!