The future of BLE, Bluetooth, and USB

[laurensvalk]

Introduction

This issue gathers some ideas for Bluetooth and USB to help us keep track of the big picture while we work on low-level details.

Note that most of this won't be done any time soon (if ever). This is mainly intended to avoid implementation choices now which might come back to hurt us later. If you want us to prioritize this, please consider becoming a sponsor :rocket: .

Nomenclature

• The code in boxes is pseudocode for commands in user scripts.
• Arrows: wireless or wired communication.
• Each color is a different service/characteristic which may or may not share a common connection.

1. Single-hub coding scenario

This is what we are working on today. We are working on a Pybricks BLE service to handle things like starting and stopping programs in a clean way. Standard I/O (print/input) may be a separate service or characteristic on the same connection.

Until now, both were handled on one and the same characteristic, which made it easy to mess up the connection by sending the wrong characters.

Cleaning this up and documenting it also paves the way for other editors or extensions to support Pybricks.

This is the only use case we will target for the upcoming v3.0 release. That is likely also the last release for Move Hub; it won't have enough space for the other features listed below.

2. Multi-hub coding scenario

This might work as above, but there are many elements still missing such as:

• Distinguishing hubs (e.g. by name or color)
• Multiple Pybricks Code instances (e.g. tabs or windows)

3a. Multi-hub communication scenario

This is still to be explored. At the basic level, there might be serial i/o streams between hubs that could be used with standard MicroPython tools. Higher level abstractions such as mailboxes could be added on top of these later.

3b. Other BLE UART devices

This is really the same as above, but worth showing in a separate diagram because this really opens up interesting possibilities.

4. BLE HID Peripherals

We may support connecting to generic HID devices over BLE (and possibly classic). Users could build on this to add specific mappings for certain devices like popular gaming consoles.

5. Bluetooth classic scenario

This is only available on Prime Hub and Inventor Hub. Therefore, we will not use Bluetooth classic for any system purposes. So, you won't be able to download and run programs with it.

Rather, it might be used within end-user scripts. For example, you might set up a generic RFCOMM connection with a server or client. This could be another Prime Hub, an EV3 Brick, a laptop, and so on.

6. LEGO Powered Up Remote

See #186.

7. USB scenario

This is only available on Prime Hub and Inventor Hub. Therefore, we will probably not use USB for system purposes. So, you probably won't be able to download and run programs with it. USB will be used primarily for charging the hub. We may re-enable REPL over USB if users want this. Since REPL is treated as a regular program, we'd have to define a way to start and stop it. Perhaps USB can also be used for fast(er) file transfer.

[TechnicBRICKs]

This would be really cool!! My priorities would be, first scenario # 3 (although this also implies scenario # 2) and then scenario # 4.

My hope is that basic functionality may be feasible with ubluetooth module (# 3), even before you are able to prioritize it, as we start to see examples making use of it. It would be a matter of finding someone willing to share his work and code examples, or going the hard way and figure it out yourself.

[laurensvalk]

Partly inspired by the existing messaging module, here is one idea for what communication could look like.

Peripherals:

from pybricks.messaging import PeripheralHub

# Can make only one of these
peripheral = PeripheralHub()

# Starts advertising. 
peripheral.advertise(True)

# Waits for incoming connection. On success, stops advertising and returns a stream object, else raises TimeOutException.
hub_connection = peripheral.listen(timeout=1000)

# Now you can send and receive data using hub_connection.read and hub_connection.write

The central:

from pybricks.messaging import CentralHub

# Can make only one of these
central = CentralHub()

# Starts scanning for hubs that are advertising.
central.scan(True)

# Results will be stored in central.scan_results, a dict of  name:address pairs
# So you could wait until some devices are found, or look for a specific one:
while 'my_other_hub' not in central.scan_results:
     pass

# Found it!
address = central.scan_results['my_other_hub']

# Stop scanning
central.scan(False)

# Connect. Returns a stream object on success, else raises TimeOutException.
hub_connection = central.connect(address, timeout=1000)

# Now you can send and receive data using hub_connection.read and hub_connection.write

Peripherals (simplified API idea)

from pybricks.messaging import ble_wait_for_connection

# Waits for incoming connection. On success, stops advertising and returns a stream object, else raises TimeOutException.
stream = ble_wait_for_connection(timeout=1000)

# Now you can send and receive data using stream.read and stream.write

The central (simplified API idea)

from pybricks.messaging import ble_connect

# Connect. Returns a stream object on success, else raises TimeOutException.
stream = ble_connect(address_or_name, timeout=1000)

# Now you can send and receive data using stream.read and stream.write

[mwinkler]

Really like to see this roadmap! As TechnicBRICKs already mention, I hope that ubluetooth where available, so that we can experiment and build exciting use cases.

[TechnicBRICKs]

Partly inspired by the existing messaging module, here is one idea for what communication could look like.

Peripherals:

from pybricks.messaging import PeripheralHub

# Can make only one of these
peripheral = PeripheralHub()

# Starts advertising. 
peripheral.advertise(True)

# Waits for incoming connection. On success, stops advertising and returns a stream object, else raises TimeOutException.
hub_connection = peripheral.listen(timeout=1000)

# Now you can send and receive data using hub_connection.read and hub_connection.write

So it looks halfway done already! :D

Meanwhile maybe playing around these examples could lead us somewhere. https://github.com/micropython/micropython/tree/master/examples/bluetooth

[laurensvalk]

The examples for ubluetooth already work just fine on the official firmware.

Our objectives are just a little different. Compare those examples with the suggested code snippets above. We're aiming to let you do it with two lines of code and about 10x fewer system resources used. ubluetooth is great for communicating with everything including the kitchen sink, but we don't need all of that just to communicate between hubs.

[TechnicBRICKs]

Fully understood your approach! Just thought that for the time being, this could lead to an intermediate solution.

To get it working guess one needs to add the ubluetooth library into your firmware zipfile, correct!?

[laurensvalk]

No, it doesn't work that way. ubluetooth is written in C, not Python.

[mwinkler]

The examples for ubluetooth already work just fine on the official firmware.

Your referring to the official spike prime firmware, correct? Because there is some stuff missing (pair function for example).

[laurensvalk]

Ah, that's interesting. I haven't tried all of the examples. The ubluetooth module seems to be actively developed still, so perhaps newer spike/inventor firmwares will get the latest versions in future updates too.

@mwinkler, as you've expressed interest in having ubluetooth in the Pybricks firmware too, could you share a few use cases that really need it, which aren't already covered above?

[mwinkler]

With the described use cases, is all covered what I have currently in mind.

My idea is a track based racing game, where every player controls his own vehicle via game pad (or pu remote). The cars are also connected to a central unit/hub. The central unit then can send commands and receive events from the cars, like increase speed (give a boost), count a marker/checkpoint (to track lap count/time), stop all cars, and so on...

So in the end, if all is implemented in pybricks, I don't need the ubluetooth module. The only reason I'am asking for the module is, because I have started experimenting with it (HID over GATT with a BLE pad) on spike prime and stuck now because of the missing functions in the lego firmware. But I see, that providing these modules have impact on memory usage and needs to be carefully selected.

Btw, thank you for the great work you do (you both) with this project.

[NStrijbosch]

Interesting dicussion!

I have been playing around with the ubluetooth from the official SPIKE Prime firmware to get Scenario 3.A/B working.

I second @laurensvalk statement on the complexity of the code necessary to get everything setup and the resources the ubluetooth module requires. So, once a more resource efficient implementation is available in Pybricks that would be my goto firmware for multi hub projects.

Just a few questions that came to my mind on the implementation of Scenario 3A:

• what would be the maximum number of pheripherals that can connect to a single central?
• can a hub be both a central and peripheral at the same time? (e.g., to get a network of three hubs that each have a direct connection to the other two)
• what happens when a connection is lost? Would that require a complete restart of the programs to get the connection back? or does it trigger a peripheral to start advertising and the central to start scanning automatically?

[laurensvalk]

Thanks @NStrijbosch -- some good points there. We're not sure yet about the connection limits.

I think we could enable the user to monitor the connection status so they can reconnect if they want to. But hopefully it should be stable enough within reasonable distances to the point where most users might not need to do this.

How is your experience with connectivity and stability, since you've worked with 9 hubs at once?

[NStrijbosch]

From my experience the connection limit of the Robot Inventor/Spike Prime hub:

• one hub is able to connect to 4 other hubs (3 peripherals and 1 central in my case) But I am not knowledgeable enough to know if this is: a hardware limitation; a software limitation in the firmware; or my own implementation. To get 9 hubs to communicate I implemented a BLE Tree network structure, with a few hubs acting as both central and peripheral.

The stability has some issues from time to time, once again not sure if this is hardware/firmware/my software. But some of my observations:

• For small scale projects (<4 hubs) connection is fine. Although it seems that it it is important to not flood the network at the central side with messages: this leads to an out of memory error. I have not managed to prevent this when implementing the Nordic UART protocol, so I opted for a messaging protocol that prevents this flooding.
• for bigger projects establishing a connection fails once in a while (at the hub that acts as both a central and peripheral). If this happens an already established connection fails at the moment of connecting a new hub to the same central. Not a big issue if it would have been possible to reestablish the lost connection. But for now the only option I have is restarting all hubs... (which can be a pain with 9 hubs). When everything is connected and the network is not flooded with messages it works fine.

[laurensvalk]

Thanks a lot for sharing this. This will set the minimal benchmark for us :wink:

We would certainly be interested in your findings once you test our stuff. We welcome contributions to the code as well.

And good to know that a hub can be both a central and a peripheral, too. We need that, since we use BLE for programming as well, where it is the peripheral to the PC.

[giadefa]

Going back to a previous issue which is now referring to this issue for the solution: https://github.com/pybricks/support/issues/164

How do I communicate few floats back and forward between a normal python program running in my PC and the hub micropython? I don't quite see this simple scheme in the description above.

Thanks

[dlech]

For EV3, we have https://pybricks.github.io/ev3-micropython/messaging.html

For Powered Up hubs, this is not implemented yet.

[laurensvalk]

How do I communicate few floats back and forward between a normal python program running in my PC and the hub micropython? I don't quite see this simple scheme in the description above.

That would be use case 3b. This pictures a phone as an example, but it could be a PC as well. The PC could use any language with BLE support, such as Python with the bleak library, or JS with WebBLE, etc.

And indeed, this is all still in the concept stage.

[dlech]

One thing I have been thinking about that is specific to BLE UART is that we have write with response or write without response in one direction and indicate (with response) or notify (without response) in the other direction. In the with response case, there should be no packets dropped, but throughput will be lower since there is the extra overhead of a response sent for each packet. In the without response case, the transmitter has no way of knowing if the receiver actually received the data and the receiver has no way of knowing if they missed something (unless it is handled at the protocol level).

So I'm thinking we probably want to default to the slower, safer option. But we might want to add an option for the faster lossy option if there are use cases that require high bandwitdh.

[laurensvalk]

At the moment, we have pybricks.bluetooth for lower-level bluetooth classic and pybricks.messaging with higher level messaging tools. Along these lines, perhaps we can have pybricks.ble for lower level BLE stuff.

I've been brainstorming what that absolute minimum might be, which will still support all of the communication use cases above. It might all boil down to just this:

# SPDX-License-Identifier: MIT
# Copyright (c) 2018-2020 The Pybricks Authors

"""
pybricks.ble: Bluetooth Low Energy communication tools.
"""

class BLEDevice:

    address = "AA:BB"
    """Address of the connected device"""

    name = "my_city_hub"
    """Name of the connected device"""

    mtu = 23
    """Maximum transmission unit."""

    # Add methods and/or attributes such as to configure write with/without response, etc

class NUSDevice(BLEDevice, PybricksIOBuffer):
    pass

def nus_connect(address, timeout=3000) -> NUSDevice:
    """Connects to a peripheral that is currently accepting a connection.

    This hub will act as the central. It can connect to any other peripheral
    that advertises the Nordic UART Service (NUS).

    Arguments:
        address: Bluetooth address or name of the device you wish to connect to.
        timeout: How long to scan for the device before giving up, or
            ``None`` to wait indefinitely.

    Raises:
        TimeoutException: If the connection could not be made within the
            given timeout.

    Returns:
        Stream object representing the connection to the peripheral.
    """
    pass

def nus_wait_for_connection(timeout=None) -> NUSDevice:
    """Accepts an incoming connection from a central.

    This hub will act as a peripheral and advertise the Nordic UART Service
    until a connection is made or the timeout is reached.

    Arguments:
        timeout: How long to wait for a connection before giving up, or
            ``None`` to wait indefinitely.

    Raises:
        TimeoutException: If the connection could not be made within the
            given timeout.

    Returns:
        stream object representing the connection to the central.
    """
    pass

# Peripheral 

from pybricks.ble import nus_wait_for_connection

prime_hub_stream = nus_wait_for_connection()

prime_hub_stream.write("Hello!")

# Central

from pybricks.ble import nus_connect

city_stream = nus_connect("my_city_hub")
technic_stream = nus_connect("AA:BB:CC")

class LWP3Device(BLEDevice):
    pass

def lwp_connect("....") -> LWP3Device
    pass

class pybricks.pupdevices.RemoteControl(LWP3Device):
    pass

my_duplo_hub = lwp_connect("Duplo Hub")

So I'm thinking we probably want to default to the slower, safer option. But we might want to add an option for the faster lossy option if there are use cases that require high bandwitdh.

Could we add this as a parameter above? Or is this a hub-wide setting that will affect all subsequently made connections?

[NStrijbosch]

Is this the lowest level possible? I am thinking beyond Nordic UART, e.g., LWP ;)

My main question: what are your thoughts on support for LWP, i.e., either by allowing specific characteristics in this low-level BLE to communicate with LWP devices, or by extending scenario 6 beyond the PoweredUp Remote?

I have been diving into the LWP docs lately (that would be extending scenario 6). For myself I see a few advantages (and of course also disadvantages) in using this protocol to communicate between a SPIKE Prime/Robot Inventor hub to any other PU hub supporting LWP. The advantages I enjoy at the moment:

• the program runs from a single hub, this makes coding/debuging way more efficient compared to updating the program on each of the hubs.
• No third party firmware necessary on the peripheral hubs.

Of course the disadvantages are numerous:

• You rely on the computation power of a single hub
• The update frequency of everything on the peripheral hubs is limited by BLE
• ...

[laurensvalk]

Thanks for your quick response.

Is this the lowest level possible?

No, this is mainly the low level for the NUS communication stuff, hence the nus_ prefixes. And despite being "low-level", this is really quite usesable in end-user scripts already, since you get a stream object that is fairly easy to use. Higher level pybricks.messaging is optional and maybe we don't even need it.

My main question: what are your thoughts on support for LWP, i.e., either by allowing specific characteristics in this low-level BLE to communicate with LWP devices, or by extending scenario 6 beyond the PoweredUp Remote?

In other words, this leaves room for other functions in pybricks.ble that start with lwp_ :wink:

[michieldetailleur]

I hope this is the best issue to post this under...

I applaud the good progress on making the official Lego controller work with PyBricks #186

Allthough one thing the Lego controller seems to lack is analog input. I love to control my Powered Up sets with my PS4 controller via BrickController2. And I would love it even more if I could connect the PS4 controller directly to the Powered Up hub (https://github.com/pybricks/support/discussions/156#discussioncomment-802999). Unfortunately that controller is not BLE capable, neither is the PS5 controller.

However: the latest Xbox controller seems to support BLE! https://en.wikipedia.org/wiki/Xbox_Wireless_Controller#Xbox_Series_X/S_launch_model

Seeing that:

• probably many people might already or will in the future own this controller
• this controller will be available for sale for probably a long time
• it's a high quality controller for an affordable price (around €70)
• you can also use this controller on your PC (so it's multipurpose, even if you don't have an Xbox)

Would anyone else be interested in PyBricks support for this controller? It seems to be technically possible at least.

I would be very willing to donate to the devs so they can buy one of these controllers and start playing with it/developing support for it. (and buy one for myself once there is some code to test) Anyone else wanting to chip in? What do the devs think? Is there room to put this on the roadmap in the near future?

[laurensvalk]

I applaud the good progress on making the official Lego controller work with PyBricks

:+1:

However: the latest Xbox controller seems to support BLE!

This is good to know. It would be even better if it uses a standard protocol similar to other input devices. If it does, this increases the chances we could ever do it, since space on the hubs is limited.

A good place to start would be to make (or search for) a regular Python script (so not Pybricks) that runs on your PC which reads the controller button state. Then we'd know what's theoretically required to be done on the hub firmware side.

What do the devs think? Is there room to put this on the roadmap in the near future?

So if everything above checks out, it might be part of category 4 of the first post in this thread, though that's perhaps not for the near future.

I would be very willing to donate to the devs

Thank you. We accept sponsors for the project. However, while we highly appreciate all contributions, we can't make promises as to which features get done first, if ever, since we do this in our free time.

[mwinkler]

The latest Xbox controller with BLE, supports HID over GATT, so its compatible to defined standards. Possibly for some features like the rumble motors are special GATT characteristics needed.

[NStrijbosch]

class LWP3Device(BLEDevice):
    pass

def lwp_connect("....") -> LWP3Device
    pass

class pybricks.pupdevices.RemoteControl(LWP3Device):
    pass

my_duplo_hub = lwp_connect("Duplo Hub")

Now that I am thinking about this: Will the my_duplo_hub be a stream object similar to nus, that collects all notifications received from a LWP3Device? (I think this will mean that notifications cannot be handled parallel to your program, or could this be possible with generator functions?)

Since we are talking about the absolute minimum could it be an option to define your own notification handler to obtain similar behaviour as a remote, i.e., all notifications handled in the background of your program. Something like this:

from pybricks.ble import lwp_connect
from pybricks.tools import wait

# custom notification handler
def notification_handler(message,state):
    if message[2] == 0x45: 
         if message[3] == 0x00:
               if message[4]==0x01:
                     state.left.plus = true
               elif message[4] == 0xFF:
                     state.left.min = true
         if message[3] == 0x01:
             .... 

#setup connection
Remote = lwp_connect("Remote",notification_handler)

# subscribe to left remote button
Remote.write([0x0A, 0x00, 0x41, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x01]) 

# main program
while true:
      print(Remote.state.left.plus)
      wait(100)

[laurensvalk]

Instead of a notification handler, we were thinking to keep incoming LWP3 messages in a finite queue that you can process at your own convenience.

[Tcm0]

This is good to know. It would be even better if it uses a standard protocol similar to other input devices. If it does, this increases the chances we could ever do it, since space on the hubs is limited.

https://github.com/ndeadly/MissionControl/issues?q=is%3Aissue+is%3Aopen+label%3A%22BLE+controller%22 lists some BLE controllers and some of them have documentation/logs linked.

[giadefa]

HI, sorry if the question has been answered already somewhere. I asked several months ago if it was possible to communicate between the hub and a python program on my PC in input and output. I practically need to send commands to the hub and read sensors back, i.e. communicate a bunch of floats back and forward. The answer was back then that some new feature would allow this. Anything new?

[laurensvalk]

While the simplified version discussed here is not yet implemented, it's certainly already possible to send data back and forth to a PC.

Could you open a separate issue to describe your use case? It would be easier to help that way. It would be good to know what kind of code is running on the PC side (Python or something else.)

[darvin]

Subscribing to get notified when it's done. Need it for finishing my Lego assembly robot replicator (it's like a 3d printer but it assembles Lego models from Lego instead of printing. Can assemble itself - its replicator!)

[antonvh]

The discussion is now mainly focussed on BLE, while the original issue also mentioned USB. I love all the BLE stuff and I think it is more important than USB. But what about this?

ser = UARTDevice(Port.USB, 115200, timeout=None) and then: os.dupterm(ser)

I can see use cases where even one of the hardware ports of a hub is connected to third party pyboards or uart devices. Then it would make sense to even do dupterm over one of these ports: ser_a = UARTDevice(Port.A, 115200, timeout=None) os.dupterm(ser_a)

[laurensvalk]

For UART, see also #220. USB could be an interesting addition there for sure.

[SpudGunMan]

the pybricks.messaging being able to interface nicely with a pybricksdev class would be cool

it would be used like.. pybricksdev.messaging_transmit('ABC', '123')

[JeanRev]

Scenario 3 would be the most interesting for me in the short term. It would open the door to a large number of interactions with external peripherals (game controllers for example, but also others non-lego hardware) using a PC as a gateway.

The dream would be to have an API on top of it allowing also to manage sensors and motors remotely (in the same way as the official lego firmware), without having to write a communication protocol oneself. We would have the best of both worlds in some way.

Scenario 4 also looks exciting. However, I managed to connect a recent Xbox BLE controller with a Rapsberry Pi, and that was already a challenge. I had to update the controller's firmware, disable DRM on the linux side, fine-tune the Bluetooth settings to successfully pair, and finally use a different USB dongle than the Raspberry's because of some obscure bug in its Bluetooth implementation. And even then, pairing only works 1 time out of 5. I can imagine that successfully pairing a lego hub with a wide range of devices reliably will not be without difficulty... But I'm no expert :-)

[JeanRev]

BTW, regarding Scenario 1

This is the only use case we will target for the upcoming v3.0 release. That is likely also the last release for Move Hub; it won't have enough space for the other features listed below.

It would be cool, if there was some kind of graceful degradation for older hubs (so they may still works with newer PyBricks version but with the corresponding features disabled).

[pbochynski]

I have some questions about current status (the issue wasn't updated recently).

1. What options are selected for development? What is the current priority?
2. I've seen Nordic UART service can be used for PC <-> HUB communication. What is missing to make it possible in HUB<->HUB scenario?
3. Do you need any help (testing, writing examples, docs, etc) to speed it up?

PS I love this project!

[laurensvalk]

What is the current priority?

Our current priority is to get the beta version finished and tested and released on the stable channel.

While this technically does not include new Bluetooth communication features, we did indeed push out a tutorial for Hub <--> device communication.

What is missing to make it possible in HUB<->HUB scenario?

Various bits and pieces are there to make it work, but we need to have a good look to make it simple and consistent across all the hubs. If you want to try something experimental, check out https://github.com/pybricks/pybricks-micropython/pull/80

I love this project!

Thanks! (And thanks for becoming a sponsor!)

[pbochynski]

Thanks for the update. What did you use to prepare this video? https://www.youtube.com/watch?v=4pbSTq4WUvg Is it based on this PR?

[laurensvalk]

Yes. In the short term, what I'm planning to do is make a tutorial to explain how to use this.

Because our main app now has an easy way to upload any firmware version, this is all quite doable for non-hackers as well :)

[laurensvalk]

Anyone looking for basic hub-to-hub communication can try this experimental feature: https://pybricks.com/projects/tutorials/wireless/hub-to-hub/broadcast/

Thanks @NStrijbosch !

[SpudGunMan]

woah cool - yes thanks.

[pbochynski]

With broadcast implementation, you opened a lot more possibilities than described in the initial comment on this issue. The publish-subscribe model is much better for integrating many hubs than a direct connection. I tried already simple scenario of sharing sensor data between hubs, and now I have a few other ideas, but wanted first to double check with you if it makes sense and how hybrids project can support it.

1. I started with reading advertisement data from hubs to see the messages published by Broadcast.send on my PC using the bleak library you introduced in pc-communication example. I now analyze the code in the PR to decode the data, but as both BLE and micropython code are new to me it will take some time ;). Maybe you already have some code snippet that you can share? Something like python/bleak implementation of Broadcast receive.
2. Receiving should be easy, but sending probably requires something more (bleak is just the client). What about sending data from non-lego devices in the same format (using broadcast)? I was thinking about using ESP32 with different sensors (like this ultrasonic sensor) to communicate with lego hub. I could build a 4-direction ultra-precise wireless distance sensor for less than $5 and use it instead overpriced Lego 45604 ($45).

What do you think?

[laurensvalk]

Those are good ideas :+1:. Could you perhaps start a discussion for the specifics? That way we can leave this issue for the generic overview/plan. Thanks!

[SpudGunMan]

I noticed movement over here. And comment for solicitation. https://github.com/LEGO/lego-ble-wireless-protocol-docs/pull/30

[dlech]

Are you saying that hubs running Pybricks should be able to connect to other hubs running official LEGO firmware using LWP3 (they already can)? Or are you saying that hubs running Pybricks should implement the full or partial LWP3 themselves? What would be the benefit of this compared to the other alternatives?

[SpudGunMan]

Are you saying that hubs running Pybricks should be able to connect to other hubs running official LEGO firmware using LWP3 (they already can)? Or are you saying that hubs running Pybricks should implement the full or partial LWP3 themselves? What would be the benefit of this compared to the other alternatives?

If @ me, it was just FYI of activity on the community level, topically related, but no requests. Likely better placed in I/M.

[westurner]

4. BLE HID Peripherals

We may support connecting to generic HID devices over BLE (and possibly classic). Users could build on this to add specific mappings for certain devices like popular gaming consoles.

From https://github.com/pybricks/support/issues/191#issuecomment-1493256816 :

• From "Connecting to an Xbox one controller" (over BLE w/ ubluetooth) https://gist.github.com/CubeLegend/f77c6f0d8b2aaee78a30d2c6508cdaf2 :

import binascii

import ubluetooth

from micropython import const

_IRQ_CENTRAL_CONNECT = const(1)
_IRQ_CENTRAL_DISCONNECT = const(2)
_IRQ_GATTS_WRITE = const(3)
_IRQ_GATTS_READ_REQUEST = const(4)
_IRQ_SCAN_RESULT = const(5)
_IRQ_SCAN_DONE = const(6)
_IRQ_PERIPHERAL_CONNECT = const(7)
_IRQ_PERIPHERAL_DISCONNECT = const(8)
_IRQ_GATTC_SERVICE_RESULT = const(9)
_IRQ_GATTC_SERVICE_DONE = const(10)
_IRQ_GATTC_CHARACTERISTIC_RESULT = const(11)
_IRQ_GATTC_CHARACTERISTIC_DONE = const(12)
_IRQ_GATTC_DESCRIPTOR_RESULT = const(13)
_IRQ_GATTC_DESCRIPTOR_DONE = const(14)
_IRQ_GATTC_READ_RESULT = const(15)
_IRQ_GATTC_READ_DONE = const(16)
_IRQ_GATTC_WRITE_DONE = const(17)
_IRQ_GATTC_NOTIFY = const(18)
_IRQ_GATTC_INDICATE = const(19)
_IRQ_GATTS_INDICATE_DONE = const(20)
_IRQ_MTU_EXCHANGED = const(21)
_IRQ_L2CAP_ACCEPT = const(22)
_IRQ_L2CAP_CONNECT = const(23)
_IRQ_L2CAP_DISCONNECT = const(24)
_IRQ_L2CAP_RECV = const(25)
_IRQ_L2CAP_SEND_READY = const(26)
_IRQ_CONNECTION_UPDATE = const(27)
_IRQ_ENCRYPTION_UPDATE = const(28)
_IRQ_GET_SECRET = const(29)
_IRQ_SET_SECRET = const(30)

_ADV_IND = const(0x00)
_ADV_DIRECT_IND = const(0x01)
_ADV_SCAN_IND = const(0x02)
_ADV_NONCONN_IND = const(0x03)

_UART_SERVICE_UUID = ubluetooth.UUID("6E400001-B5A3-F393-E0A9-E50E24DCCA9E")
_UART_RX_CHAR_UUID = ubluetooth.UUID("6E400002-B5A3-F393-E0A9-E50E24DCCA9E")
_UART_TX_CHAR_UUID = ubluetooth.UUID("6E400003-B5A3-F393-E0A9-E50E24DCCA9E")

class BLESimpleCentral:
    def __init__(self, ble):
        self.gamepad_name = None
        self.gamepad_addr_type = None
        self.gamepad_addr = None

        self._ble = ble
        self._ble.active(True)
        self._ble.irq(self.bt_irq)

    def bt_irq(self, event, data):
        if event == _IRQ_SCAN_RESULT:
            # A single scan result.
            addr_type, addr, adv_type, rssi, adv_data = data
            if find_adv_name(adv_data) is None:
                return

            print('Address-Type: ', addr_type)
            print('Address: ', [hex(n) for n in addr])
            print('ADV-Type: ', adv_type)
            print('RSSI in dBm: ', rssi)
            print('ADV-Data: ', [hex(n) for n in adv_data])
            print('ADV-Data (Name): ', find_adv_name(adv_data))  # Xbox Wireless Controller
            print()

            if find_adv_name(adv_data) == "Xbox Wireless Controller":
                self.gamepad_name = find_adv_name(adv_data)
                self.gamepad_addr_type = addr_type
                self.gamepad_addr = addr
                self._ble.gap_scan(None)
                self.connect()
        elif event == _IRQ_SCAN_DONE:
            # Scan duration finished or manually stopped.
            pass
        elif event == _IRQ_PERIPHERAL_CONNECT:
            # A successful gap_connect().
            conn_handle, addr_type, addr = data
            print("connect")
            print(data)
            print('Address-Type: ', addr_type)
            print('Address: ', [hex(n) for n in addr])
            print()
        elif event == _IRQ_PERIPHERAL_DISCONNECT:
            # Connected peripheral has disconnected.
            conn_handle, addr_type, addr = data
            print("disconnect")
            print(data)
            print('Address-Type: ', addr_type)
            print('Address: ', [hex(n) for n in addr])
            print()

    # Find a device advertising the environmental sensor service.
    def scan(self):
        self._ble.gap_scan(5000, 100000, 25000, True)

    def connect(self):
        print(self.gamepad_addr)
        self._ble.gap_connect(self.gamepad_addr_type, self.gamepad_addr)

def find_adv(adv_type, data):
    i = 0
    while i + 1 < len(data):
        ad_structure_len = data[i]
        ad_structure_type = data[i + 1]
        ad_structure_payload = data[i + 2: i + ad_structure_len + 1]
        if ad_structure_type == adv_type:
            return ad_structure_payload
        i += ad_structure_len + 1
    return None

def find_adv_name(data):
    n = find_adv(9, data)
    if n:
        return str(n, 'UTF-8')  # Text
    return None

def demo():
    ble = ubluetooth.BLE()
    central = BLESimpleCentral(ble)

    central.scan()

def demo_connect():
    ble = ubluetooth.BLE()
    central = BLESimpleCentral(ble)

    central.connect()

if __name__ == "__main__":
    demo()

@all @mwinkler @cubelegend ( https://github.com/pybricks/support/issues/140#issuecomment-782938904)

• Does this work with an Xbox controller?
• Does this work with a Stadia controller?
• Does this work with a PS4 controller?
• Is there an / is this the HID Gamepad + Joystick(s) standard, or does there need to be configurable button remapping from device ids sort of like Steam PC?
  • https://www.google.com/search?q=steam+controller+remap
  • https://partner.steamgames.com/doc/features/steam_controller/getting_started_for_players

[Tcm0]

@westurner This seems to be especially for XBOX ONE controllers. I'm not 100% sure about the low level stuff, but as far as I know, XBOX controllers use XINPUT while most other controllers use DINPUT (DINPUT beeing HID based?). It won't work with the stadia controller as that one uses DINPUT and it won't work with a PS4 controller because those use BTC and not BLE. I think that there is a button standard for XINPUT but not really for DINPUT. Of course, one could focus on the most popular gamepads. I don't think that there are many gamepads out there that use BLE.

[mwinkler]

@westurner Here is the output of the script (running on stock mindstorm firmware, ble xbox controller): So connection seems to work

[westurner]

• [ ] https://github.com/pybricks/support/issues/995
  • [ ] ENH: pybricks-code:
  • Get the BLE Controller device ID
  • Show which buttons are being activated (~ https://gamepad-tester.com/ https://www.google.com/search?q=gamepad-tester+github https://opengameart.org/content/free-keyboard-and-controllers-prompts-pack )
  • (Remap buttons)
  • Generate the Python code block and or ideally just a config dict (?) for the LEGO hub
    • device ID
    • HID ~profile
    • button map
• https://en.wikipedia.org/wiki/PlayStation_4_technical_specifications#Input_and_output
  • PS4: Bluetooth 2.1
  • PS4 Slim: Bluetooth 4.0 (BLE)
• https://en.wikipedia.org/wiki/PlayStation_5#Hardware
  • PS5: Bluetooth 5.1 (BLE)
• https://en.wikipedia.org/wiki/Xbox_Wireless_Controller#Third_revision_(2020)
  • XboxOne Controller rev 1: No Bluetooth
  • XboxOne Controller rev 2: Bluetooth 4.0
  • XboxOne Controller rev 3 (2019-): Bluetooth (BLE)

    The controller also supports Bluetooth Low Energy, and can be paired to a Bluetooth device and an Xbox device simultaneously.

  • XboxOne Adaptive Controller: Bluetooth 4.1 (BLE) https://en.wikipedia.org/wiki/Xbox_Adaptive_Controller
• https://en.wikipedia.org/wiki/Google_Stadia#Controller
  • Stadia: Bluetooth 5.0 (BLE) [2019-2023]

    DirectInput (DINPUT), XInput (XINPUT), xlib

• https://en.wikipedia.org/wiki/DirectInput#DirectInput_vs_XInput
• https://www.pcgamingwiki.com/wiki/Glossary:Controller#Windows

  XInput XInput is Microsoft's API for Xbox 360-compatible controllers such as the official Xbox 360 Controller.

• XInput wrapping:
  • https://www.pcgamingwiki.com/wiki/Controller:Stadia_Controller#XInput_wrapping
  • If necessary for a particular game, you can convert DirectInput to XInput with various apps including Steam Link on Linux (flatpak)
• xlib (X11; Linux) https://python-xlib.github.io/python-xlib_11.html#Selecting-Events
• https://github.com/r4dian/Xbox-Controller-for-Python/blob/master/xinput.py
• https://wiki.archlinux.org/title/Gamepad#Mimic_Xbox_360_controller
  • xboxdrv
• https://levelup.gitconnected.com/build-an-xbox-controller-abstraction-layer-in-python-using-xinput-api-aaf0f8d05ac2
  • https://github.com/izdwuut/xbox-mapper-tutorial/blob/main/xinput.py
• https://wiki.archlinux.org/title/Gamepad#Connect_Xbox_Wireless_Controller_with_Bluetooth
  • https://github.com/atar-axis/xpadneo/#connection
  • how to bluetoothctl scan
  • https://github.com/atar-axis/xpadneo/blob/master/hid-xpadneo/src/hid-xpadneo.c

    Force feedback support for XBOX ONE S and X gamepads via Bluetooth

• https://wiki.archlinux.org/title/Gamepad#Device_permissions
  • https://codeberg.org/fabiscafe/game-devices-udev
  • https://pkgs.rpmfusion.org/cgit/nonfree/steam.git/tree/60-steam-input.rules (rpmfusion steam-devices
  • https://github.com/ValveSoftware/steam-devices/blob/master/60-steam-input.rules