ge0rg / aprsdroid

APRSdroid - Geo-Location for Radio Amateurs
https://aprsdroid.org/
GNU General Public License v2.0
495 stars 96 forks source link

Bluetooth LE (BLE / Bluetooth Smart) TNC #124

Open stephensworld opened 8 years ago

stephensworld commented 8 years ago

Is it possible to add Bluetooth LE support in aprsdroid? e.g. HM-10 module

Thanks

Stephan

ge0rg commented 8 years ago

Principally, yes. However, Bluetooth LE does not provide a "serial" profile comparable to Bluetooth SPP, so a specific profile would be needed.

I have created a proof-of-concept for APRS-IS-over-BLE here: https://github.com/ge0rg/bluetoothle-tnc

Something similar needs to be created for exchanging KISS packets over BLE, and then implemented both on the IC and on the Android side. Such a protocol could even be deployed on iOS.

I'd gladly volunteer to write out the specification of the protocol and do the Android side implementation, provided somebody creates an according BLE TNC (or at least a BLE-to-serial adapter of some sort).

mobilinkd commented 8 years ago

I am working on new TNC prototypes right now so, what the heck, I might at well do a version that has a dual-mode BT module on it. I'm not convinced that BLE is a good fit for a general purpose TNC, but I do like to see hams experimenting with this stuff. If I can get hardware into peoples hands and something good comes of it, that would be great. The TNC will also have a working USB CDC and BT SPP.

I will try to get one out to you in the next month or so. The firmware will be very alpha quality. And I will have little time to work on the BLE code.

Now I just need to find my HM-12 modules...

ge0rg commented 8 years ago

@mobilinkd I think that BLE is the only reasonable way to attach a TNC to an iPhone, so that would be the main reason. Besides of that, it has sufficient capacity for 1200bd APRS, so it might be an interesting low-power alternative for Android as well, provided the AFSK codec doesn't eat too much in comparison to BT.

Ping me up some weeks before you start your work, and I'll sketch out the KISS-over-BLE protocol.

kf7rcs commented 8 years ago

I too would be interested in this. I have a Joying Android head unit for my car. It will not connect to my mobilinkd via blue tooth. I believe it does not support SPP profile. It sees the mobilinked as a phone. So i am willing to try out any beta code and provide feedback.

ge0rg commented 7 years ago

Are there any news on BLE hardware?

rriggs commented 7 years ago

@ge0rg I received my first packets over BLE today.

rriggs commented 7 years ago

@ge0rg First packet TX happened this evening.

ge0rg commented 7 years ago

Hey @rriggs, that's some awesome news. From your twitter feed I implied that you are running a HM-10 module of some sort, but I wasn't yet able to find its BLE serial protocol spec. If it's as simple as "read/write strings from certain characteristic", it shouldn't be too hard to add to APRSdroid.

Maybe you can contact me off-list with some more details? :)

markqvist commented 5 years ago

Even though this thread has not had activity for some time, I'll venture to add my input as well.

I'd argue that it makes more sense to simply run the communications over a generic UART profile, than creating a more specific KISS-over-BLE profile. KISS has always encapsulated in "raw" UART anyways, and several adapters already support a generic UART profile. Either way, I'd be interested to think what you all think.

ge0rg commented 5 years ago

It looks like this is exactly what's happening in the TNC3. I didn't have the time to implement the BLE protocol yet, but I assume the battery savings will be meager compared to the GPS usage on the phone. OTOH, maybe the TNC battery will last significantly longer in LE mode?

@rriggs, do you happen to have numbers on BLE vs SPP battery life time expectations?

mobilinkd commented 5 years ago

Surprisingly, EDR uses somewhat less power than BLE when connected. I can tune the parameters to make BLE consume slightly less, but then it becomes painfully sluggish in the config app.

I would like to see support for BLE in APRSdroid. Because then I can release a BLE only device. I have a couple BLE-only prototypes on my desk that I am experimenting with.

On Tue, Jan 29, 2019, 09:33 Georg Lukas <notifications@github.com wrote:

It looks like this is exactly what's happening in the TNC3. I didn't have the time to implement the BLE protocol yet, but I assume the battery savings will be meager compared to the GPS usage on the phone. OTOH, maybe the TNC battery will last significantly longer in LE mode?

@rriggs https://github.com/rriggs, do you happen to have numbers on BLE vs SPP battery life time expectations?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-458584985, or mute the thread https://github.com/notifications/unsubscribe-auth/AEfuDBmb0CER69la0qNIGQMrzZIyoxeMks5vIGnfgaJpZM4Hpya8 .

rriggs commented 5 years ago

I should also say that I agree with you. Just use transparent data transfer. Easy to implement, works well, lots of existing modules support it.

I would like to see amateur equipment standardize on the service and characteristic uuids. I just more or less used the ones from your BLE node POC.

On Tue, Jan 29, 2019, 14:02 Mobilinkd LLC <notifications@github.com wrote:

Surprisingly, EDR uses somewhat less power than BLE when connected. I can tune the parameters to make BLE consume slightly less, but then it becomes painfully sluggish in the config app.

I would like to see support for BLE in APRSdroid. Because then I can release a BLE only device. I have a couple BLE-only prototypes on my desk that I am experimenting with.

On Tue, Jan 29, 2019, 09:33 Georg Lukas <notifications@github.com wrote:

It looks like this is exactly what's happening in the TNC3. I didn't have the time to implement the BLE protocol yet, but I assume the battery savings will be meager compared to the GPS usage on the phone. OTOH, maybe the TNC battery will last significantly longer in LE mode?

@rriggs https://github.com/rriggs, do you happen to have numbers on BLE vs SPP battery life time expectations?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-458584985, or mute the thread < https://github.com/notifications/unsubscribe-auth/AEfuDBmb0CER69la0qNIGQMrzZIyoxeMks5vIGnfgaJpZM4Hpya8

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-458686399, or mute the thread https://github.com/notifications/unsubscribe-auth/ABBj3WvLylMhnNKF4fklMhL-2csJ8vdFks5vIKjVgaJpZM4Hpya8 .

markqvist commented 5 years ago

In my upcoming TNC I ended up going for BLE instead of SPP, even though there is not much application support at the moment, but I think it makes more sense from a platform standpoint to support BLE in the long run. It's currently working with a generic UART over BLE profile, where it uses the KISS protocol transparently over UART, same as serial and USB serial. I'd be happy to test any experimental builds of APRS droid on the hardware, or supply you with a prototype if that could help out.

mobilinkd commented 5 years ago

Hi Mark, when you say "generic UART over BLE", is there a new standard service UUID for this? When I last looked, every BLE vendor was doing their own implementation -- service and characteristic UUIDs. Some worked better than others. A few were not 8-bit clean, apparently designed for text-only use. IIRC, the NUS example code from Nordic worked pretty well.

What I would hope to see is that the ham community adopt a standard set of service and characteristic UUIDs for this. That's why I used the ones @ge0rg initially came up with rather than create my own. This standardization will help avoid fragmentation of support in ham radio software for these devices. It is easier and faster to scan for a well-known service than for all nearby devices.

KTS_SERVICE_UUID = '00000001-ba2a-46c9-ae49-01b0961f68bb' KTS_RX_CHAR_UUID = '00000003-ba2a-46c9-ae49-01b0961f68bb' # Notify KTS_TX_CHAR_UUID = '00000002-ba2a-46c9-ae49-01b0961f68bb' # Write (with and without response)

On Wed, Jan 30, 2019 at 4:49 AM Mark Qvist notifications@github.com wrote:

In my upcoming TNC I ended up going for BLE instead of SPP, even though there is not much application support at the moment, but I think it makes more sense from a platform standpoint to support BLE in the long run. It's currently working with a generic UART over BLE profile, where it uses the KISS protocol transparently over UART, same as serial and USB serial. I'd be happy to test any experimental builds of APRS droid on the hardware, or supply you with a prototype if that could help out.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-458899852, or mute the thread https://github.com/notifications/unsubscribe-auth/AEfuDIwiU53kGdCjrCASHgtSFIoqXS1Nks5vIXjVgaJpZM4Hpya8 .

markqvist commented 5 years ago

I was referring to the example UART GATT service from Nordic, with UUID 6E400001-B5A3-F393-­E0A9-­E50E24DCCA9E. In my (admittedly still rather limited) testing, it seems to work as intended.

But I had assumed this service profile was used as a reference and was thus relatively standard, which apparently I was wrong in assuming.

What are your thoughts, do you think we should "stick" with just running everything over a generic UART profile (whatever profile that turns out to be), or would you rather see a specific profiles for different protocols, say a profile for KISS endpoints, a profile for CAT control and so on?

I just read through @ge0rg's specifications, and it looks soild enough, although I think there's a bit of an issue if the specification only talks about AX.25 frames, as opposed to KISS frames. It might just be me being pedantic here, but I think it's worth chiseling out clearly in the spec, if this is to be a standard we all converge on. The data frame command is only one command out of a potential 256 KISS commands (or 16 if using the high nibble as HDLC port number).

To be honest I don't see much added benefit to using the custom profile over a generic profile just advertising UART capability though, apart from the discoverability aspect, which might very well be worth it anyhow. The additional features from ge0rg's spec like audio volume and other diagnostics messages can just be implemented in KISS frames, and seems to me like a more elegant solution, than having to side-channel them into a separate characteristic.

mobilinkd commented 5 years ago

Hi @markqvist ,

Every module manufacturer uses a different service UUID for transparent data transfer.

We have a product on the market using those UUIDs, and @hessu supports those UUIDs in the aprs.fi iOS app. Using those existing UUIDs will allow you and others to build on existing infrastructure.

In the end, as you aptly note, it is just doing KISS over a transparent data connection [1].

The primary reason to use a custom service UUID is that we are not just providing a transparent data service -- we are providing a very specific service that sends and receives KISS packets over a BLE endpoint. An entire KISS TNC firmware can be implemented inside a BLE module, without a single UART on the module enabled.

Things like rig control belong under a separate service UUID. Rig control should not be tied to a TNC service. The idea behind BLE is that its services implement the purest form of a microservice.

I need to publish the current implementation with input from @hessu and @ge0rg. It is pretty straight-forward, but there are a few things to clarify. For example, we support packing multiple KISS packets into a single MTU transfer. That impacts things like setting of KISS parameters (0x01-0x05) and hardware parameters (0x06). And a single KISS packets can span multiple MTU transfers.

I agree that using KISS hardware parameters (0x06) to update things like volume levels is the way to go. That is how we configure our TNCs.

The Mobilinkd iOS config app is an open reference implementation of the BLE client side code in Swift. If you look at the SLIP and KISS code, there is nothing at all surprising in there.

The Mobilinkd TNC3 use a Microchip BM78 module. I have a number of prototypes using various BLE-only modules, including various Nordic nRF51 & nRF52 modules. At last count there were 7 or 8 different modules in various stages of testing on my bench. The nRF52 modules are certainly the way to go for a BLE-only product.

markqvist commented 5 years ago

@mobilinkd, you are right, it's the correct way to go about it. A KISS service should be advertised as that over BLE, not a UART service, which as you mention is even quite a misnomer here, since there is no serial parameters involved.

If a more formal specification is worked out, I will support that in my upcoming products as well. Having an agreed upon standard is the way to go here.

hessu commented 4 years ago

I guess nobody wrote the updated formal spec yet? @mobilinkd ? I was just thinking I might do it if nobody else did. It'd be good to have it out there before someone makes yet another protocol which is different, and then we have to start reimplementing clients.

markqvist commented 4 years ago

I know you probably also have a ton of other things to work on, but I really like the idea of you being the author of it @hessu. It seems more fitting than if it originates from one of us “hardware implementers”. We can all help out, of course. But you being the main author, I think would be very suitable.

And I agree completely, it would be very good to have it formalised sooner than later.

mobilinkd commented 4 years ago

Hessu, if you have time to do it, please do. I will certainly review and can provide additional details where required.

I think it's important that we leave BLE-specific implementation details such as MTU size open. As long as the spec is clear that it is a KISS protocol and that a KISS packet can span multiple BLE transfers, we should be OK. The first BLE module I worked with had a fixed MTU of 27.

I would like to provide guidance to anyone considering extending the spec. I would like to specify that any extensions to the protocol should be done using additional attributes. I use vendor-specific KISS hardware records to configure and control the TNC3, but anything that is meant to be open should be done as a GATT attribute. This will limit what I can do with the TNC3 because of the BLE module used, but I think it is best for the standard.

As an example, an extension to the protocol such as SMACK should be done on a separate pair of attributes. Same for command mode. The reason for this is that it advertises this additional capability to the BLE client. Another example, one I think you requested, is audio level information.

Beyond that, I think we just need to specify the service GUID and KISS R/W attribute GUIDs.

Kind Regards,

Rob Riggs WX9O Mobilinkd LLC

On Fri, May 15, 2020 at 4:03 PM Mark Qvist notifications@github.com wrote:

I know you probably also have a ton of other things to work on, but I really like the idea of you being the author of it @hessu https://github.com/hessu. It seems more fitting than if it originates from one of us “hardware implementers”. We can all help out, of course. But you being the main author, I think would be very suitable.

And I agree completely, it would be very good to have it formalised sooner than later.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-629482463, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD64DBM5MYDA6FSUVXDIULRRWU3BANCNFSM4B5HE26A .

hessu commented 4 years ago

Here goes, initial proposal:

https://github.com/hessu/aprsfi/blob/kiss-ble/AX25-KISS-BLE.md

Please review and provide feedback. I was very tired so there may be something silly in there. Maybe Georg does not mind if we hijack this aprsdroid ticket for the discussion since we've been here so long already. :)

markqvist commented 4 years ago

Great work. That is clear, concise and easy to follow for implementation, and I don’t think it leaves anything up to the imagination. Thanks a lot for the effort.

I’d like to discuss whether it’s beneficial to tie it exclusively to AX.25. I know that in practice, AX.25 is almost all that is ever encapsulated in KISS, but that’s not a limitation of KISS, since it can encapsulate and transport frames for anything to and from a TNC. It would also be “free” specification-wise to state it more as just “KISS over BLE”, since it would not pose any restrictions for the current primary use-case of AX.25 over KISS over BLE. Personally I have several uses of KISS TNCs that do not involve AX.25, but use other link layers.

I think the primary focus should still be on AX.25, and that the frame and buffer size calculations and rationales you present should be based on AX.25, but opening it up a little will make it come across as more generic, which I think is good in this case.

If you’re interested I can submit an edit proposal as a pull request, and you can look it over and incorporate what you find useful, if you find that my points are sensible at all, that is.

mobilinkd commented 4 years ago

It looks like someone else is a fan of RFC 2119. :-)

One change I believe needs to be made is with regards to the vendor extensions section. One cannot just pick a random UUID for an attribute. The BLE spec is written to say that, but Nordic software is written such that the service and attribute UUIDs must share a base, and you get 16 bits for the attribute UUIDs. This is why I changed the UUIDs used by @ge0rg's original implementation.

The way it is done with Nordic (and I think I have seen others do the same) is to create a UUID, then zero out the third and fourth octet. Set it to 1 for the service UUID and then increment from there for the service attributes.

Maybe the right thing to do is ask people to pick a 13-bit number for their vendor code and use the bottom 3-bits for their own 8 unique attributes?

Additionally, you are welcome to refer to the Mobilinkd iOS Config App as well. This is written in Swift and released under the Apache license. https://github.com/mobilinkd/iosTncConfig It implements part of the spec for configuring the TNC using SetHardware codes.

Should this be added as a reference for the KISS protocol? http://www.ax25.net/kiss.aspx

hessu commented 4 years ago

Thanks, good points.

@markqvist I think the application should be made aware of whether the KISS device can accept AX.25 or something else. For example, my application can only do AX.25 at the moment, and it might cause issues if it would send AX.25 to a non-AX.25 KISS device. Should there be a different service UUID for non-AX.25 KISS-over-BLE device, so that they wouldn't even show up in my app?

markqvist commented 4 years ago

In practice, most TNCs will happily (and should) transmit whatever is encapsulated in its KISS data frame, whether that be AX.25 or not. That was part of the point with KISS, to leave only the physical layer processing to the TNC, and actual data link-layer protocols to the host.

A KISS-device is per definition network- and link-layer protocol agnostic, so there is not really a concept of "AX.25 vs non-AX.25" KISS device. You could go as far as saying that all KISS devices are essentially non-AX.25 devices. Even the Frame Check Sequence, which is sometimes taken as a part of AX.25 is technically part of the physical layer HDLC-like framing.

My argument is that any KISS device should in theory adhere to the following operating principle:

  1. Accept any data frame that is correctly encapsulated in a KISS frame and does not exceed the devices MTU.
  2. Reconstruct the actual frame data from the potentially escaped byte sequence in the KISS frame.
  3. Transmit the data on the physical interface encapsulated in HDLC and add the Frame Check Sequence.

Conversely, a receiving KISS device should transparently pass any received frames with a valid Frame Check Sequence to the host device encapsulated as a KISS frame, without making any judgements on the content.

mobilinkd commented 4 years ago

I agree with Mark.  KISS is agnostic as to the contents of the payload.  I have used the TNC3 to receive satellite telemetry data which are not in AX.25 format, but which are embedded in HDLC frames.

The spec should probably make it clear that it is up to the client application to determine whether the data it receives from the TNC is properly formatted.

hessu commented 4 years ago

Please find an updated version here:

https://github.com/hessu/aprs-specs/blob/master/BLE-KISS-API.md

diff: https://github.com/hessu/aprs-specs/commit/e72f376fb461526aef943b2e26b0cdd34ca63659

I tried to implement all of your suggestions, and give a fairly precise method to allocate extension UUIDs. Did I miss something?

hessu commented 4 years ago

Hi, If there's no further feedback on the spec, I'll send some announcement emails and tweets and other sorts of posts on Friday to make other vendors aware.

markqvist commented 4 years ago

I have nothing to add. I can wholeheartedly stand behind this :) I will implement this in OpenModem in the near future. Thanks for taking the time and effort to write it!

mumrah commented 3 years ago

Will generic serial emulator modules like the HM-10 work with APRSDroid? The spec looks nice, but I wonder if it could be possible to also discover a generic device and select the right characteristic. @ge0rg or @rriggs did either of you manage to get an HM-10 working?

mobilinkd commented 3 years ago

If the module does not allow you to fully specify the service and characteristic UUIDs to match those in the spec, it will not work. IIRC, the HM-10 is fairly limited in this regard. That said, I have not touched an HM-10 in probably 4 years. I don't know if the firmware has changed at all.

On Tue, Sep 15, 2020 at 8:12 PM David Arthur notifications@github.com wrote:

Will generic serial emulator modules like the HM-10 work with APRSDroid? The spec looks nice, but I wonder if it could be possible to also discover a generic device and select the right characteristic. @ge0rg https://github.com/ge0rg or @rriggs https://github.com/rriggs did either of you manage to get an HM-10 working?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ge0rg/aprsdroid/issues/124#issuecomment-693111041, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD64DDBQNBGAHBSI7VEAC3SGAGHZANCNFSM4B5HE26A .

hessu commented 3 years ago

For good UX I would not bother with generic devices with characteristics which are not unique to KISS support.

mumrah commented 3 years ago

Are there plans to support BLE and the aforementioned spec in APRSDroid?

penguin359 commented 3 years ago

I have been playing with the HM-10 module as a possible BLE serial port, but I'm finding both it's documentation and implementation as abysmal. The docs, poorly written, do not document the basic command structure as implemented, but seem to hint at it being Hayes modem-like. However, it is anything but. I have worked with other Bluetooth serial modules that do a far better job of implementing the Hayes modem protocol. For one, a regular Hayes modem compatible interfaces should use carriage return to delimit the end of a command and carriage return/line feed to delimit responses. However, the HM-10 seems to just use time-limiting. That means, to query it's version, the command AT+VERS? must be sent all at once with some idle guard time before and after it. If trying manually type the command into a terminal, It will see AT and then return two bytes, O and K, with no other delimiters. Unless the full command is pasted into the terminal, it's impossible to type and get it to respond correctly. This caused me quite a bit of debugging time to figure out why I couldn't get any commands beyond AT which returns OK to work. Even now that I can get other commands to work, there's no clear end to the response beyond just waiting for it to stop sending characters (or carefully parsing the response until it's known all the bytes are in).

As for creating a Kiss-specific profile for BLE, I haven't had a chance to review that spec in detail, but it looks like it's basically just a serial conduit. I quote, "Longer frames are split into multiple BLE transfer units. Each unit may contain multiple KISS frames." That sounds like each read/write of the characteristic for KISS frames has no relation to actual packet boundaries. Does the spec really contain anything in it that makes it more than a serial port over BLE that just happens to be flagged as transferring bytes for KISS frames?

I'm just wondering if it would be better to call it a serial protocol with a characteristic descriptor that indicates it's for KISS since it still appears to require the KISS frame escaping and doesn't preserve boundaries on it's own.

mumrah commented 3 years ago

@ge0rg I was wondering if you had considered this any further. I have prototyped a BLE TNC for use with the NinoTNC and so far it has been working well using the aforementioned spec and the aprs.fi iOS app.