Adding support for Meshtastic (Bircom)

[ketan]

Hello there -

As you may or may not be aware, sat comm devices (like Inreach and Spot) are illegal to operate in India without an operating license from the government. I'm part of volunteer team that is building a tracking network for pilots in Bir/Biling in India based on a mesh technology called meshtastic mesh network. We already have a tracker in place at https://tracker.bircom.in.

Since flyxc is a great app that's used and trusted by many pilots - I'm wondering if you'd be open to integrating with our tracker. I have some questions in terms of how you integrate with other trackers right now, so I can contribute back to this repository.

For context - our mesh network captures the following details:

• unique device identifier
• short name (initials) to show on the map
• long name - for better identification
• gps co-ordinates, along with timestamps
• baro altitude, along with timestamps
• (optional) some device details - hardware and firmware versions

From a design point of view, we can push these packets to you as and when we see the packets (we have a mqtt backend), or we can offer any APIs as you see fit for you to periodically pull this information.

Let me know, thanks!

[vicb]

Sounds super cool. I'm busy today let's discuss tomorrow!

On Sun, Sep 1, 2024, 11:51 Ketan Padegaonkar @.***> wrote:

Hello there -

As you may or may not be aware, sat comm devices (like Inreach and Spot) are illegal to operate in India without an operating license from the government. I'm part of volunteer team that is building a tracking network for pilots in Bir/Biling in India based on a mesh technology called meshtastic https://meshtastic.org mesh network. We already have a tracker in place at https://tracker.bircom.in.

Since flyxc is a great app that's used and trusted by many pilots - I'm wondering if you'd be open to integrating with our tracker. I have some questions in terms of how you integrate with other trackers right now, so I can contribute back to this repository.

For context - our mesh network captures the following details:

• unique device identifier
• short name (initials) to show on the map
• long name - for better identification
• gps co-ordinates, along with timestamps
• baro altitude, along with timestamps
• (optional) some device details - hardware and firmware versions

From a design point of view, we can push these packets to you as and when we see the packets (we have a mqtt backend), or we can offer any APIs as you see fit for you to periodically pull this information.

Let me know, thanks!

— Reply to this email directly, view it on GitHub https://github.com/vicb/flyXC/issues/301, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAB4X4XGZVJ2DAQ3JTNJQITZULPSLAVCNFSM6AAAAABNOV6KY2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGQ4TSMZVGIYTGOI . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[vicb]

Ketan,

First of all thanks for contacting flyXC.

I'm all for integrating your platform in flyXC.

A few notes on how flyXC works:

1) Pilots have to create an account on flyXC via the live tracking > settings menu. There they can connect to any of the platforms that are supported by flyXC. If they connect to multiple platforms, their track will be the union of those platforms. It's usual that pilots connect a satellite tracker for the reliability and an app (ie XCtrack) to get a more detailed trace when they have phone coverage.

2) flyXC supports pulling data from those platforms (most of them) and also supports push from the platforms (OGN, Zoleo)

For 1, there needs to be some kind of id/SSO to connect your platform to flyXC. The pilot name is defined on the flyXC account so that every platform uses the same name.

For 2, pushing to flyXC sounds better to me so that flyXC does not keep pulling your server while nobody is flying. flyXC needs to receive lat/lon/alt and some kind of id. You can optional send the speed, heading and messages (landed ok, need help, ...)

I'm based on France, Paris TZ. We can discuss more tomorrow. Please send any question you have.

[ketan]

For the push based system - do you have some sample code that I can look at? I suppose that for the identifier bit, you'd probably expect that a tile is added to https://flyxc.app/devices that allows collecting some identifier/token corresponding to the user on my tracking system?

[cyberorg]

Here is our current way of getting these devices to flyxc via skylines. https://bircom.in

We would like to avoid skylines step, we could use the same way to push to flyxc as we do to skylines if that works.

{"payload":"leolive=1&user=1B8D47F&lat=32.036799099999996&lon=76.70709049999999&sid=33412863&password=123&sog=0&alt=1356&tm=1725239499","_msgid":"21c1c55a9ea8c718","method":"POST","url":"https://skylines.aero/track.php","headers":{"Content-Type":"application/x-www-form-urlencoded","Content-Length":122}}

Please note that the devices don’t have internet connection, the data is sent by base stations that are connected to internet. We cannot collect passwords so the push has to be key based or dummy password like skylines. Livetrack24 doesn’t work as it needsa proper password.

This is what our raw data looks like: https://tracker.bircom.in/api/nodes

[vicb]

Yes, there would be a new card on the devices page.

I can create a POST endpoint on flyxc.app (i.e. flyxc.app/api/bircom/push) for your platform to push the data. It would be similar to what is available for Zoleo

https://github.com/vicb/flyXC/blob/6be1593a194638e9a453f3eb617ae136f6f36cc8/apps/fxc-server/src/app/routes/zoleo.ts#L131-L147

I can update the code for those 2 steps - it should not take me much longer than 1h.

What is left to be defined:

• what token/id should be entered in flyXC (or created by flyXC and entered on your platform)
• what mechanism flyXC uses to authenticate your push requests. It can be basic Auth (same as zooleo), a secret passed in an HTTP header or in the JSON payload, ... I guess using https is possible on your end?

[cyberorg]

Each of the device has unique ID like "!674c3f41" which we can ask user to register their device with flyxc.

This is the raw packet we get from the device.

{"payload":[{"lat":32.0367098,"lon":76.7070858,"alt":1359,"tm":1725257499,"satsInView":5,"groundSpeed":1,"groundTrack":0,"pDOP":394},{"from":"!674c3f41","fromShortName":"Pgid","fromLongName":"Paraguide In/5652D866","fromHardware":"T_ECHO","to":"!ffffffff","toShortName":"mesh","toLongName":"Mesh","toHardware":"MESH","channelId":"LongFast","gatewayId":"!da52924c"}],"_msgid":"c126d40bd21bbad7"}

[vicb]

Each of the device has unique ID like "!674c3f41" which we can ask user to register their device with flyxc.

How can we validate those id:

• Is there a known structure, i.e. "!" followed by 8 hex digits that we can use?
• Can I query an endpoint to make sure the id is registered?

About the payload:

• How many messages would a single push contain?
• tm seems to be the timestamp in sec
• Is groundSpeed km/h
• Is groundTrack degree
• What is pDOP
• Can you send text messages (Looks like https://tracker.bircom.in/api/nodes has some)

[cyberorg]

Is there a known structure, i.e. "!" followed by 8 hex digits that we can use?

Yes, this is the known structure "!" followed by 8 hex digits, like the skylines user ID, users can add this to flyxc settings.

Can I query an endpoint to make sure the id is registered?

Converting that hex get you the decimal, which can be used to query like this: https://tracker.bircom.in/api/node/1733050177/positions

How many messages would a single push contain?

One message like the packet example shared per push.

tm seems to be the timestamp in sec

Yes, unix timestamp

Is groundSpeed km/h

yes

Is groundTrack degree

not sure about that

What is pDOP

Position Dilution of Precision (PDOP), which can be ignored

Can you send text messages (Looks like https://tracker.bircom.in/api/nodes has some)

yes, this is what the text packet look like:

{"payload":{"packet":{"from":1733050177,"to":4294967295,"channel":0,"decoded":{"portnum":1,"payload":"Test for flyxc","wantResponse":false,"dest":0,"source":0,"requestId":0,"replyId":0,"emoji":0},"id":3631729303,"rxTime":1725260218,"rxSnr":6.75,"hopLimit":3,"wantAck":false,"priority":0,"rxRssi":-52,"delayed":0,"toHex":"!ffffffff","fromHex":"!674c3f41","fromShortName":"Pgid","fromLongName":"Paraguide In/5652D866","fromHardware":"T_ECHO","toShortName":"mesh","toLongName":"Mesh","toHardware":"MESH"},"channelId":"LongFast","gatewayId":"!da52924c"},"_msgid":"c7f75a8c20e963cd"}

[vicb]

yes, this is what the text packet look like:

Is there any way to add the lat/lon/alt in the packet on your side? If not I can get the position at around rxTime.

What name should I use for your platform? "meshtastic", "bircom", ... and what stable link can I use to give users more info about the platform.

I think I can the implementation ready by tomorrow.

[vicb]

What would help is a payload sample for each of the message you can push (position, text message, any other?)

Thanks!

[cyberorg]

yes, this is what the text packet look like:

Is there any way to add the lat/lon/alt in the packet on your side? If not I can get the position at around rxTime.

Nope, you can use last position received.

What name should I use for your platform? "meshtastic", "bircom", ... and what stable link can I use to give users more info about the platform.

Meshtastic, as that is the project firmware we are using. More information on https://bircom.in. We will update the page for getting the devices registered on flyxc.

I think I can the implementation ready by tomorrow.

You're the best :)

[cyberorg]

What would help is a payload sample for each of the message you can push (position, text message, any other?)

Position packet:

{"payload":[{"lat":32.036736499999996,"lon":76.7071666,"alt":1343,"tm":1725261844,"satsInView":8,"groundSpeed":0,"groundTrack":0,"pDOP":155},{"from":"!674c3f41","fromShortName":"Pgid","fromLongName":"Paraguide In/5652D866","fromHardware":"T_ECHO","to":"!ffffffff","toShortName":"mesh","toLongName":"Mesh","toHardware":"MESH","channelId":"LongFast","gatewayId":"!da52924c"}],"_msgid":"4244095cfa19969f"} Text packet:

{"payload":{"packet":{"from":1733050177,"to":4294967295,"channel":0,"decoded":{"portnum":1,"payload":"One more test for flyxc","wantResponse":false,"dest":0,"source":0,"requestId":0,"replyId":0,"emoji":0},"id":3631729304,"rxTime":1725261821,"rxSnr":6.25,"hopLimit":3,"wantAck":false,"priority":0,"rxRssi":-54,"delayed":0,"toHex":"!ffffffff","fromHex":"!674c3f41","fromShortName":"Pgid","fromLongName":"Paraguide In/5652D866","fromHardware":"T_ECHO","toShortName":"mesh","toLongName":"Mesh","toHardware":"MESH"},"channelId":"LongFast","gatewayId":"!da52924c"},"_msgid":"0b104298b56d0074"}

Thanks!

🙏

[vicb]

You can email me at victor at the app domain to agree on a way to authenticate your push requests

[cyberorg]

We are revisiting device ID. Thinking of using hash of id + shortname + longname. This way the device can be used by different pilots and the tracks will show under their account.

[vicb]

We are revisiting device ID. Thinking of using hash of id + shortname + longname. This way the device can be used by different pilots and the tracks will show under their account.

Wouldn't that be a problem if a pilot updates its name? (they would have to update the ID in flyxc, right?)

[cyberorg]

Yes, new pilot will get unique key if he uses the device used by someone else already registered to flyxc and if pilot changes his name then they'll update the key on flyxc.

[vicb]

Let me know what the format looks like so that I can validate with a regexp when a device is registered

[cyberorg]

Is groundTrack degree

Yes, 0-359

[ketan]

Hi @vicb - I'll take over this implementation from @cyberorg from pushing the data point of view. Here's what I can send you as a reference. Can change the contract as you wish, if there's any optional data or additional that you can use, let me know:

Again, all of this is just indicative as reference, and open to change based on your needs.

curl -X POST https://flyxc.app/api/bircom/push \
      -H 'Accept: application/json' \
      -H 'Content-Type: application/json' \
      -H 'Authorization: bearer [TOKEN]' \
      -d '{
            "type": "message",
            "user_id": "xxxxx",
            "time": "2024-09-02T10:01:12.000Z",
            "message": "I have landed safely, waiting for pickup"
      }'

curl -X POST https://flyxc.app/api/bircom/push \
      -H 'Accept: application/json' \
      -H 'Content-Type: application/json' \
      -H 'Authorization: bearer [TOKEN]' \
      -d '{
            "type": "position",
            "user_id": "xxxxx",
            "time": "2024-09-02T10:01:12.000Z",
            "latitude": 32.1927,
            "longitude": 76.4506,
            "altitude": 1778.12,
            "baro_pressure": 1012.24,
            "ground_speed": 30,
            "heading": 90,
            "satsInView": 12,
      }'

The payload structure. This is the data we have:

Attribute	Type	Description
type	string	Type of payload (message, position)
user_id	string	Unique identifier for the device (not user)
time	string	ISO 8601 time 2024-09-02T10:01:12.000Z (can be seconds/millseconds from epoch)
latitude	number	Latitude ± 180 range
longitude	number	Longitude ±90 range
altitude	number	GPS altitude, in meters
baro_pressure	number	Barometric pressure, if available, in hPa.
message	string	For type message, the text message from the user
ground_speed	number	The GPS speed over ground (kmph), if available
heading	number	The GPS heading, if available (0-359 degrees), if available
satsInView	number	The number of GPS sats in view, if available

[ketan]

Let me know what the format looks like so that I can validate with a regexp when a device is registered

I believe this may not be needed, we'll ensure that we're sending valid device IDs right away. That said, we'll be making improvements (probably in the next few weeks/months) to allow signups so we get more stable user-ids.

[ketan]

If you'd like to have a quick connect, let me know - we can setup a quick zoom per your convenience.

[vicb]

Nice format!

A few comments:

• I'd prefer to have timeMSec in milliseconds rather than a date as a string
• satsInView should probably be sats_in_view for consistent naming - but flyXC will not use it anyways
• It would be nice to known the format of user_id so that I can do some validation when a device is registered - to be sure that there is no typo/copy paste error

[ketan]

Nice format!

Thanks!

A few comments:

• I'd prefer to have timeMSec in milliseconds rather than a date as a string
• satsInView should probably be sats_in_view for consistent naming - but flyXC will not use it anyways
• It would be nice to known the format of user_id so that I can do some validation when a device is registered - to be sure that there is no typo/copy paste error

Updating based on your inputs

Attribute	Type	Description
type	string	Type of payload (message, position)
user_id	string	Unique identifier for the device (not user) (uuid format, for e.g. aa752cea-8222-5bc8-acd9-555b090c0ccb)
time	number	milliseconds since epoch
latitude	number	Latitude ± 180 range
longitude	number	Longitude ±90 range
altitude	number	GPS altitude, in meters
baro_pressure	number	Barometric pressure, if available, in hPa.
message	string	For type message, the text message from the user
ground_speed	number	The GPS speed over ground (kmph), if available
heading	number	The GPS heading, if available (0-359 degrees), if available

Removed fields

Attribute	Type	Description
satsInView	number	The number of GPS sats in view, if available

[ketan]

I've added my implementation here

https://github.com/ketan/paragliding-meshmap/pull/48/files#diff-fc763e0bf3cc5d6c20112d7af42872cd92c466627e470f6cf6a3862a74e53aedR48-R62

The payload is here

https://github.com/ketan/paragliding-meshmap/pull/48/files#diff-5260fa6e1c85f706e8a7460dd6aad80973ee086346568538546ca447f5f1ec0aR221-R230

[vicb]

flyXC changes have been merged, let me know how the testing goes.

Thanks 🙏

[cyberorg]

Awesome! You must be tired of hearing “you’re the best”.

On Tue, 3 Sep 2024 at 12:58 AM, Victor Berchet @.***> wrote:

flyXC changes have been merged, let me know how the testing goes.

Thanks 🙏

— Reply to this email directly, view it on GitHub https://github.com/vicb/flyXC/issues/301#issuecomment-2325230505, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ5Q6UNVDECA6CHBEQPJB3ZUS35FAVCNFSM6AAAAABNOV6KY2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGMRVGIZTANJQGU . You are receiving this because you were mentioned.Message ID: @.***>

[ketan]

Hi @vicb - thanks for implementing this quickly.

I'm unable to push to the endpoint, getting a 405 method not allowed.

[start:be]   meshmap:decode Sending job to https://flyxc.app/api/bircom/push +3s
[start:be]   meshmap:decode { headers: { Accept: 'application/json', 'Content-Type': 'application/json', Authorization: 'Bearer xxx' } } +0ms
[start:be]   meshmap:decode { body: '{"time":1725331945607,"type":"message","message":"Test","user_id":"xxx"}' } +0ms
[start:be]   meshmap:decode Response Response { status: 405, statusText: 'Method Not Allowed', headers: Headers { date: 'Tue, 03 Sep 2024 02:52:28 GMT', 'content-length': '0', connection: 'keep-alive', 'access-control-allow-origin': '*', 'referrer-policy': 'strict-origin-when-cross-origin', 'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=N1Zh%2FWz3GcklGS9cHcFIdWGtj2yFXXX5n678a%2BQJmktUYxKX82QzbAuNQMtirQ%2FOkyajeAgpO6qOFVzXAFdShnh1c%2B%2FYJV0bgKY%2FOfJCwoTLXG0wFT5CN3d%2F7h7htt6KXQByibWeGgc%3D"}],"group":"cf-nel","max_age":604800}', nel: '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}', vary: 'Accept-Encoding', 'cf-cache-status': 'DYNAMIC', server: 'cloudflare', 'cf-ray': '8bd271a7ed15d381-FRA', 'alt-svc': 'h3=":443"; ma=86400' }, body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true }, bodyUsed: false, ok: false, redirected: false, type: 'basic', url: 'https://flyxc.app/api/bircom/push' } +0ms

I also tried with plain old curl, to no avail.

[ketan]

Quick update - I changed the post url to api.flyxc.app/api/bircom/push, and I'm now getting a 500 internal server error.

When you get the time, please let me know if this is because of some input errors?

Thanks

[vicb]

That's probably a validation error Could you email me the payload - I have to update a SSH key in my VM to be able to retrieve the logs

[vicb]

Reopening this issue:

• to fix positions
• to add tes message

[ketan]

Here's my curl request (I've removed the token, my user ID is specified here)

$ curl https://api.flyxc.app/api/bircom/push -H 'Accept: application/json' -H 'Content-Type application/json' -H 'Authorization: Bearer XXXXXX'  -d '{"altitude":1393,"time":1725340010504,"type":"position","user_id":"6d032188-142a-5538-8c6a-ff14a67ac166","latitude":32.0411907,"longitude":76.7091656,"ground_speed":20}' -D -
HTTP/2 500
content-type: text/plain; charset=utf-8
vary: Origin
access-control-allow-credentials: true
access-control-expose-headers: xsrf-token
etag: W/"15-/6VXivhc2MKdLfIkLcUE47K6aH0"
x-cloud-trace-context: c66fd0bc9ed6ac38f69a1bd6c6a137cb
date: Tue, 03 Sep 2024 07:01:19 GMT
server: Google Frontend
content-length: 21

Internal Server Error

[vicb]

You have a typo, it should be

curl https://api.flyxc.app/api/bircom/push \
-H 'Content-Type: application/json'  \  <- missing ":"
-H 'Authorization: Bearer XXXXXX'  \
-d '...'

or

curl https://api.flyxc.app/api/bircom/push \
-H 'Authorization: Bearer XXXXXX'  \
--json '...'

[vicb]

I added support for Bircom messages to flyXC.

Please test the integration and report on that issue.

Your platform is very nice. Hopefully more PG communities around the world will get inspired by your use of Meshtastic. Thanks for reaching out in the first place and being super responsive on this issue.

Have a great day

[ketan]

I'm able to see myself on the map! Thanks a bunch, although unsure why @cyberorg is not able to see himself.

One interesting side-effect of how this is implemented is that I'm seeing 500 errors for when submitting data points for devices that are not registered on flyxc. Could you perhaps respond with a 404 or something if the device_id is not available at your end? That way I won't attempt to retry sending packets if the device is not registered?

[vicb]

There is no validation of the account on the endpoint

https://github.com/vicb/flyXC/blob/9155021e88404acfabfe02d7eb7ee6af09fc337d/apps/fxc-server/src/app/routes/meshbir.ts#L14-L30

The reason why you receive a 500 error is that one of the field is malformed (user_id?)

Anyway I'll change the return status to:

• 403: auth error (as of today)
• 429: queue full, if you ever receive that, I can bump the queue size (new)
• 400: validation error (currently 500)

[ketan]

The reason why you receive a 500 error is that one of the field is malformed (user_id?)

You are right. Then I don't believe a change is necessary at your end. I'm getting a 200 OK if I send a device id that looks valid, but does not exist in your DB, and that I think is reasonable. You accepted the packet, it looked good, and you accepted it, decided to do nothing with it :)

About 429 - how long is the queue length, and processing rate?

[ketan]

Could you please also regenerate our token please? @cyberorg accidentally pasted the token here. My email address is `ketanpadegaonkar [at] gmail.com]. Thanks

Apologies for the inconvenience.

[vicb]

See https://github.com/vicb/flyXC/pull/307 for the status changes.

MESHBIR_MAX_MSG is 4000 (messages/min) - happy to bump this when/if needed.

np, I'll email you an updated token

[vicb]

The VM that processes the messages ("fetcher") has no static/public IP currently.

That's why the server (Google App Engine) collects the messages and enqueue them in Redis for the fetcher to consume on every tick (1min). The fetcher will discard any message that is not associated with a registered ID.

[vicb]

And for @cyberorg, the only reasons why you might not show up on the map after a 200 are:

• the user_id is not wrong (not associated with a flyXC account)
• the fix is too old (currently older than 20min)

[vicb]

Looks like everything is working now 🎉

[cyberorg]

It worked, then it stopped again! What would you want me to try?

position is not being updated and my device messages haven’t showed up yet.

[vicb]

@cyberorg Are you using the updated token? @ketan emailed me that everything is working for him now.

BTW did you ever receive the stickers?

[cyberorg]

Yes, updated token, kept my device on, it is sending positions every 2 min.

Nope something messed up with post if you sent by that.

[vicb]

Can you capture a request and email it to me?

[vicb]

I have some incoming requests with "user_id":"eda3... which doesn't match any registered pilots, could that explain the pb?

[cyberorg]

Mine is 9d5bb902-...

Sent you mail about txt message.

For position, curl works, so @ketan will have to figure out at his end why push from there is not working.