search

apache / nuttx

Apache NuttX is a mature, real-time embedded operating system (RTOS)
https://nuttx.apache.org/
Apache License 2.0
2.92k stars 1.19k forks source link

BLE Link Layer and nRF52 implementation (WIP) #1474

Closed protobits closed 3 years ago

protobits commented 4 years ago

I'm creating an issue to track my progress on this effort mainly to let others know what I've achieved so far and what is currently being worked on. Since this work will also include various contributions to nRF52 arch I want to avoid duplicate effort in case someone starts working on these areas in the meantime. I will create a PR (will probably be various PRs in the end) when changes settle a bit more.

These is the status of this work (I will update over time):

nRF52 arch:

Link Layer:

protobits commented 4 years ago

A teaser of things working so far. This is scanning:

Peek 2020-07-24 21-44

This is advertising (you can see the "btsak" string which is part of the advertisement data): Peek 2020-07-27 16-55

acassis commented 4 years ago

wow! Kudos Matias! You are so fast!

protobits commented 4 years ago

Just wanted to give an update, managed to establish and hold a connection. Next I will start by passing data packets back and forth to the HCI layer. This should be enough to play with btsak handling of GATT characteristics.

Peek 2020-08-22 11-24

BTW, raccoon is a nice open-source BLE sniffer. Also, I'm now using MakerDiary NRF52832 MDK board, which is quite nice for development (has embedded daplink which provides SWD+UART over USB). I have added support for it on my fork.

protobits commented 4 years ago

I went back to work with this now that I have nimBLE working on NuttX. This helped provide meaningful traffic to test the link-layer. I managed to pair my phone and start exchanging some packets.

However, this is becoming a really big task and I'm not sure I will be able to get it to 100% on my own as it takes a lot of work to do so. Is there someone else willing to take on the task and continue the work? It would be great if someone with prior experience in low-level bluetooth understanding but I had zero prior knowledge so it can be learned by reading the spec and some books. If there's someone willing, I can assist and help reviewing and maybe even testing.

Regarding what works right now:

I can polish off the implementation a bit more and leave scanning and advertising (as described above) well tested, but the whole connection handling requires much more work. It has various critical paths which I attempted to addressed using PPI peripheral but it becomes complex very fast (so that may need some rework).

Let me know if anyone is willing to work on this and I will open a draft PR.

patacongo commented 4 years ago

Sorry, I am not the Bluetooth expert you are looking for.

But I wonder if we should not try to get your changes pushed upstream in MyNewt.  There is a very high probability that they would be agree to that since we are Apache brothers in arms.

I also have no problem supporting both stacks for a period of time.  The stacks could be selected with a configuration option.  The nimBLE stack could depend on CONFIG_EXPERIMENTAL until it is rock solid, then we could remove the Intel stack.

Having the code available (although disabled via CONFIG_EXPERIMETAL) would encourage contribution.

Greg

On 9/26/2020 5:24 PM, Matias N. wrote:

I went back to work with this now that I have nimBLE working on NuttX. This helped provide meaningful traffic to test the link-layer. I managed to pair my phone and start exchanging some packets.

However, this is becoming a really big task and I'm not sure I will be able to get it to 100% on my own as it takes a lot of work to do so. Is there someone else willing to take on the task and continue the work? It would be great if someone with prior experience in low-level bluetooth understanding but I had zero prior knowledge so it can be learned by reading the spec and some books. If there's someone willing, I can assist and help reviewing and maybe even testing.

Regarding what works right now:

  • I focused on the slave role, the master role is much more complex as it requires scheduling all current connections
  • Scanning works, but only passive mode
  • Advertising works in all modes, sends scan responses, handles initiating a connection (this last part requires some work)
  • Partially handles maintaining a connection and pushes received data packets to the host layer but there's no support yet for sending data packets received from host layer
  • Some tricky link-layer control commands are not yet supported (updating connection parameters, channel map) and some not very critical features requested from host are not yet implemented (or dummy values sent)

I can polish off the implementation a bit more and leave scanning and advertising (as described above) well tested, but the whole connection handling requires much more work. It has various critical paths which I attempted to addressed using PPI peripheral but it becomes complex very fast (so that may need some rework).

Let me know if anyone is willing to work on this and I will open a draft PR.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-nuttx/issues/1474#issuecomment-699560254, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFUG6VCS7JQVWBCIF7ERKTSHZZ33ANCNFSM4PKY53KA.

protobits commented 4 years ago

Hi Greg, the work I'm referring to is independent of the support for nimBLE. To support nimBLE I've added support for RAW HCI sockets on top of Brennan's work. We're discussing this here: https://github.com/btashton/incubator-nuttx/pull/9 As I mentioned there, my goal is to have this RAW HCI support merged in NuttX (after 10.0 release) and once that is there, I can open a PR to nimBLE repo and have them support NuttX. Right now nimBLE works very well.

Anyway, I also mentioned there that we should soon discuss if we indeed want to adopt nimBLE or it will just be there as an option. This determines some of the future steps.

Going back to this issue regarding the work on nRF52 link-layer, it would be also very good to have this in NuttX, as it is such a widely used platform and a very nice chip family with a documented RADIO peripheral. So, hopefully, someone is interested in continuing this work I started.

acassis commented 4 years ago

Hi Matias, it is sad to see all these efforts you did dying again. I'm also not an expert on BLE, but I could help with tests to get the current implementation integrated on NuttX. I got the nRF52832-MDK board, so I can use it for tests. I agree that nimBLE is widely used and this is also a concern, not because the "Not Invented Here (NIH) syndrome", but because a fault discovered on it could affect all RTOSes using it. But I think it is interesting to have it on NuttX as an option, but not replacing the original stack.

protobits commented 4 years ago

Well, I certainly hope that someone else is interested in such a feature and feels motivated to help. I certainly didn't have any BLE knowledge to begin with and learned everything by reading the standard and a few books. I will see if I feel motivated again to continue this to a point a minimum usability.

Regarding nimBLE, I personally think it is a very good option and worked very well out of the box. It is certainly much more complete than our current stack. If we're discussing how to finish this link-layer, just think about all the work that would be needed to get current NuttX's stack up to date and with full features (such as security layer). But again, let's discuss this separately, it is really unrelated to this issue.

btashton commented 4 years ago

@v01d If you have a PR on your fork or here I'll start looking as I can. Now that I have a fair bit of nRF52 hardware I would be happy to do some testing. I have not used the PPI so that might take some digging as well.

protobits commented 4 years ago

I have rebased my local branch on top of my hci-socket-raw branch (the PR on top of your branch) so that I could try it using nimBLE. I'm keeping everything on a single commit for now, so maybe you can simply look at this: https://github.com/v01d/incubator-nuttx/commit/68fed7a01576542ab7cde9af7a2de02ef4779771 If we later merge the HCI RAW socket support I can open a draft PR to nuttx repo

protobits commented 4 years ago

Ok, I have worked hard on this the last few days and managed to get it closer to a working solution. I have to complete the implementation of some necessary control commands but it looks like I'm close to get the nRF app speak GATT with nimBLE on nRF52. There will be a lot to improve on the link layer after this, but I'm happy to get this working finally. Once I pass this final test I can open a draft PR. I also have various little changes to NuttX that I introducing during this work so I could start upstreaming those.

acassis commented 4 years ago

Hi @v01d nice to hear you are getting it to work. If you want me to test it please let me know, probably we will need it to add support to ESP32 BLE on NuttX. There is a nimBLE port to ESP32: https://github.com/espressif/esp-nimble so I think it will be a matter of fusing both works to get ESP32 BLE working on NuttX.

btashton commented 4 years ago

Alan this is the low level link layer which is device specific. The to integrate the ESP you will need to create the HCI abstraction. This i believe is implemented in the IDF ask from what I remember a few years back.

protobits commented 4 years ago

@acassis this will probably only usable with nimBLE so to test it will probably better to wait we can get this in as well

From what I see in that fork on nimBLE, espressif added a porting layer which is based on freeRTOS and gluing their custom HCI api to handle HCI commands from nimBLE. If you want to support ESP32 BLE on NuttX, you will have to write a very simple driver that does the same but for NuttX (like bt_uart.c and similar, but calling ESP32 API functions). With this, ESP32 BLE could be used with any BLE host stack that NuttX supports.

protobits commented 4 years ago

From what I see in that fork on nimBLE, espressif added a porting layer which is based on freeRTOS and gluing their custom HCI api to handle HCI commands from nimBLE. If you want to support ESP32 BLE on NuttX, you will have to write a very simple driver that does the same but for NuttX (like bt_uart.c and similar, but calling ESP32 API functions). With this, ESP32 BLE could be used with any BLE host stack that NuttX supports.

Actually, I'm not finding where this code interfaces with ESP32 functions. Maybe you know more about how BLE is exposed on ESP32. I assume it is via HCI commands, but maybe it is lower level.

acassis commented 4 years ago

Thank you @btashton and @v01d for this explanation. Yes, the ESP nimBLE port is based on FreeRTOS used on IDF. I think it will be necessary to create a similar HCI for ESP32 on NuttX to use the nimBLE port that @v01d is doing.

acassis commented 4 years ago

Good question @v01d! Because the ESP32 BLE registers are not public I think they are using the IDF library to create this nimBLE port.

protobits commented 4 years ago

Oof... took some work and fixing some very subtle bugs but I made it work, I can retrieve the GATT characteristics of the GATT server which runs in nimBLE example app, yay!

Tomorrow I will open a draft PR mostly so that you can at least see some code. I will then start upstreaming some minor changes and slowly clean it up.

I have to get back to get the RAW HCI ready first, though. And also try to get NuttX port of nimBLE upstreamed as well.

acassis commented 4 years ago

Great work @v01d kudos!!! @btashton what you think if we wait to get BLE and ESP32 WiFi on NuttX 10.0? I think we need it! ;-)

rdmeneze commented 4 years ago

@v01d , if you need beta testers, I'm also available.

btashton commented 4 years ago

@btashton what you think if we wait to get BLE and ESP32 WiFi on NuttX 10.0? I think we need it! ;-)

I don't think that is wise, the plan is to branch the release on Saturday and this week was supposed to be for stability. This release has already been a month longer than it should have been.

If you feel strongly about this you should reply to the list, otherwise we can make it 10.1 there are a lot of Bluetooth changes that will be in that release.

protobits commented 4 years ago

I'd also prefer to leave this for future release, it will be rushing this otherwise. As I mentioned, this still requires some work.

@rdmeneze thanks!

patacongo commented 4 years ago

There is no advantage to taking that risk.  I think that any significant new features should wait for 10.1 at this point.

On 10/1/2020 8:28 AM, Brennan Ashton wrote:

@btashton <https://github.com/btashton> what you think if we wait
to get BLE and ESP32 WiFi on NuttX 10.0? I think we need it! ;-)

I don't think that is wise, the plan is to branch the release on Saturday and this week was supposed to be for stability. This release has already been a month longer than it should have been.

If you feel strongly about this you should reply to the list, otherwise we can make it 10.1 there are a lot of Bluetooth changes that will be in that release.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/apache/incubator-nuttx/issues/1474#issuecomment-702174266, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABFUG6Q33L3JLNDVBSDXI3DSISGX5ANCNFSM4PKY53KA.

acassis commented 4 years ago

Ok guys! I agree!

protobits commented 4 years ago

Update on this work: I made several improvements to the link-layer and feels much more robust now. However I realized that to be of any use I need to support link-layer encryption. When trying to connect my phone or my PC to the nRF52832, they attempt to start encryption themselves and fail when realize that I do not support it. At least that is my understanding.

I will see if I manage to get this working as well. The upside is that the LL will be very feature complete once I'm finished.

Also, I have not opened the draft PR since my LL branch is on top of the RAW HCI support branch, which is not yet merged in NuttX, so it would include a lot of unrelated changes. I will open it once we get that branch merged.

protobits commented 4 years ago

More updates: I tackled encryption and pairing. Took some work but wasn't that hard as nimBLE takes care of some part of the cryptographic work and the nRF52832 has two peripherals handling also low-level crypto (and they are actually designed to encrypt/decrypt on the fly, very neat).

I've also wrote a HID-over-GATT service for nimBLE and managed to make my Linux (and Smartphone) see the device as a mouse and have the pointer move.

Currently I'm chasing some problem where after some time the connection drops and I cannot see why. I will try to deal with this and hopefully have this completed.

It is possible that data throughput may not be optimal. It will require some deep insight into latency between host to controller level. Maybe it would even be better to consider prioritized interrupts for link-layer interrupts. I actually had to minimize printing via serial since it seems to interfere (but this may be due to serial driver not using DMA).

acassis commented 4 years ago

Hi @v01d that is amazing! It should be nice to have an IMU (ex. MPU9255) with it creating a mouse that you move in the space, similar to Wii Mote or the LG TV Remote Control

protobits commented 3 years ago

Closing since I will not be working on this anymore (see #2140). If someone else wishes to reconsider this in the future it can be reopened.