the-modem-distro / pinephone_modem_sdk

Pinephone Modem SDK: Tools to build your own bootloader, kernel and rootfs
GNU General Public License v3.0
595 stars 64 forks source link

A-GPS, SUPL and IMSI #144

Closed ghost closed 1 year ago

ghost commented 1 year ago

what is known concerning the A-GPS mechanism sending the IMSI to the SUPL server ?

how is this related to the firmware ?

what mechanisms can be deployed inside the modem in order to avoid the IMSI leaking and just download the Almanac in order to avoid a fully manual downloading of the Almanac ?

it is known that the carrier can redirect the SUPL requests so the IMSI leaking is not dependent on the modem only, the carrier can do the leaking too

why is nobody interested in a so important privacy risk which in fact cancels all the privacy gain of the open source if you just turn on the GPS in the modem by letting Google know your IMSI and your location ?

Biktorgj commented 1 year ago

As far as I know that doesn't happen in the open firmware. I don't know if it would while using stock, depending on the configuration used, but probably doesn't happen either in its default configuration.

The modem in stock state, has a location client in charge of retrieving the AGPS data from Qualcomm via IzAT service. That doesn't work when used in bridged mode (default), since the modem itself doesn't have network access, only its host.

In the open firmware, there's absolutely nothing messing around with GPS or able to add AGPS data. There's only a proxy between the USB port and the internal SMD GPS port.

Overall, the only thing injecting AGPS data is eg25-manager, which retrieves it (as far as I know) without reading the IMSI from the modem and pushing it via the AT interface to the modem.

why is nobody interested in a so important privacy risk which in fact cancels all the privacy gain of the open source if you just turn on the GPS in the modem by letting Google know your IMSI and your location ?

You're first saying you don't know how it is done, and then blaming others for not caring about something that is in fact solved as far as I know, but you didn't take your time to learn it. I don't have a problem explaining things to anyone. Even more, I sometimes try to make some documentation on things that aren't widely know so people can understand better how part of the stack works. Admitting you don't know how something works, only to assume later on that we're doing things in a privacy risky way will only get people angry.

You can configure whatever server you want for retrieving the GNSS assistance data in eg25-manager's configuration file: https://gitlab.com/mobian1/eg25-manager/-/blob/master/data/pine64%2Cpinephone-1.2.toml#L97 And it's absolutely trivial to get you a 1$/mo VPS and put a script that retrieves the file every 6 hours and then reconfigure your pinephone so it retrieves the almanac from your VPS instead of Qualcomm's izat server, allowing you to mask your real IP address from Qualcomm if that's what worries you

ghost commented 1 year ago

@Biktorgj I have seen the modem's URCs updating the xtradata without mobile data or Wi-Fi connection (I don't know whether the modem can access the Wi-Fi), as I have understood the modem can use special GSM/4G control channels to do this and so the carrier gets in the way

beyond the mechanism, my question was why this is not a point of attention in any wiki or forum, that is if you just enable A-GPS and the SUPL server you may send your IMSI and your location to third parties

this mechanism is quite complicated especially in what the positioning through the cellular network is concerned

I will try to understand how it works and whether the mechanism is fully disclosed

ghost commented 1 year ago

@Biktorgj your analysis was exclusively on the Almanac, this should be a purely passive action, though the AGPS can retrieve the approximate location by sending the Cell ID somewhere and getting back the coordinates

the question is how this mechanism works, what data is sent, where and by what part of the firmware?

is it enough to set the AGPS integer value (AT+QGPSCFG="agpsposmode") to 0 so as to disable the approximate cellular positioning ?

ghost commented 1 year ago

@Biktorgj one of my mistakes on the cellular positioning is that it takes place through Gnome Maps, that is the Location service provides the Cell ID to the application

I will compact my original question in the following way

if someone enables A-GPS and provides a SUPL server, what kind of data exchange happens, between what parties and how can the carrier get in the way for example by pushing your IMSI and your location to the SUPL or to other servers

if there is a risk there it should be clearly stated on the Pine64 website and it would mean that the GNSS engine should be separated from the modem or be used without a SIM card

Biktorgj commented 1 year ago

@mouffa I really don't understand that defensive-blaming-shaming position all the time when you still have no idea on what is being retrieved from the modem or what is sent to the SUPL service.

A-GPS location retrieval is done by ModemManager (https://www.freedesktop.org/software/ModemManager/doc/latest/ModemManager/gdbus-org.freedesktop.ModemManager1.Modem.Location.html)

The update function seems to be here: https://github.com/freedesktop/ModemManager/blob/eae2e28577c53e8deaa25d46d6032d5132be6b58/src/mm-iface-modem-location.c#L344

...and it uses MMLocation3gpp object as a set of helper functions to retrieve the relevant data to be able to query a server to find a location (https://www.freedesktop.org/software/ModemManager/libmm-glib/1.4.0/MMLocation3gpp.html). This object contains a set of functions to get the relevant data:

guint mm_location_3gpp_get_mobile_country_code ()
guint mm_location_3gpp_get_mobile_network_code ()
gulong mm_location_3gpp_get_location_area_code ()
gulong mm_location_3gpp_get_cell_id ()

guint mm_location_3gpp_get_mobile_country_code () guint mm_location_3gpp_get_mobile_network_code () gulong mm_location_3gpp_get_location_area_code () gulong mm_location_3gpp_get_cell_id ()

So no API exposed to retrieve the IMSI. The one doing all the work on retrieving AGPS and merging location sources (at least in Mobian) is geoclue (https://gitlab.freedesktop.org/geoclue/geoclue/-/wikis/home) which in turn uses the Mozilla location service API, which doesn't seem to provide any item in its request struct to include the IMSI either (https://ichnaea.readthedocs.io/en/latest/api/geosubmit2.html)

ghost commented 1 year ago

@Biktorgj I am working with the location only on Phosh, the ModemManager does nothing for me no matter if I give mmcli commands in the terminal

the only way to get a location is to turn on the GNSS with AT commands, then I cat NMEA port and when I get a fix I start Gnome Maps, before that the maps provide the cellular or WiFi position

the automatic XTRADATA injection started after I set up the Google SUPL server with AT commands and as I told you I am sure I have seen the URC stating that the XTARADATA expired and were updated with the mobile data turned off and no WiFi

the SUPL is supposed to use the UserPlane Location Protocol, I believe there is some kind of negotiation between the modem and the SUPL server through the GSM/LTE and I can not understand what part of the firmware would do that

since the modem accepts a configuration for the SUPL it should be able to do "something" with that and I want to know this "something" independently of anything running on the SoC like the ModemManager

could it be in the remaining ADSP blob ?

Biktorgj commented 1 year ago

So what you mean is you're telling the ADSP to connect to your carrier's APN, manually setting up a custom SUPL server in it to download the almanac, and manually enabling the gnss engine?

ghost commented 1 year ago

@Biktorgj I think I am getting somewhere, let's set aside the Almanac for the time being, which is probably independent from the SUPL and it is probably downloaded by the ModemManager after I enable the GNSS with the AT command

looking at the AT+QGPSCFG="agpsposmode" command you can see that you can select "Mobile Station Based" and "Mobile Station Assisted" modes

I guess that the function which reads that command's input if it reads "Assisted" it will try to connect to the SUPL server

do you have in mind any function calling the modem's ULP capability or any other networking capabilities in order to connect to the SUPL server and do something ?

if yes, what kind of authentication takes place and what kind of data are exchanged ?

I found something on an Android forum and they were trying to catch those packets with Wireshark but there was some problem to analyse them because of the ULP protocol

if you don't know anything about this then it means that it happens in a very covert way else all those AT commands are there only for decoration

analysing this will also answer whether the SUPL can provide an Almanac and whether it can go through the GSM without any other data connection

ghost commented 1 year ago

@Biktorgj the AT+QGPSCFG="agnssprotocol" command enables by default 3 protocols LPP, RRLP, RRC

where are the libraries for these protocols ?

as I have understood those protocols work through "system broadcasts" and similar channels with strange names which are not related with the ordinary mobile data IP connection

the OMA specifications are complicated and there is not much layman information on the web

ghost commented 1 year ago

@Biktorgj according to the Wikipedia article

https://en.wikipedia.org/wiki/Assisted_GNSS

the MS Based function means downloading the Almanac and the time while the MS Assisted is when you send data to a server and makes the calculations for you

Google SUPL is only MS Based, this traffic is charged as "mobile data", I don't know whether the modem started an unsolicited IP connection though I had the data turned off or it went through another channel

so the observation that the Almanac was injected after the configuration of the SUPL is probably correct

I still don't know if the IMSI is sent to Google, this depends on the mechanism that deploys those protocols when accessing the SUPL server

the RRLP protocol is supposed to be the new standard in place of the older LPP

this 2011 article from GPSWORLD explains many things and can be found with the pictures on the Wayback machine

https://www.gpsworld.com/wirelessexpert-advice-positioning-protocol-next-gen-cell-phones-11125/

ghost commented 1 year ago

@Biktorgj the AT+QGPSCFG="agnssprotocol" command has also the option "User Plane" and "Control Plane"

the "Control Plane" generates packets on a hardware level below TCP/IP, the default values are all protocols on both planes

I don't know how much hardware-based can be the "Control Plane", that is, is it possible to be hidden completely in the ADSP firmware ?

ghost commented 1 year ago

@Biktorgj I had a look at and in the code for the first time, first of all the 39MBytes for a binary is enormous, just imagine that all the Linux firmware is around 220MBytes, I thought it was some KBytes just for implementing the channel sharing, the device must be to a very large extent a SDR modem, it seems that implements whole stacks of protocols, how much in binary size is an Osmocom implementation by the way ?

I tried to follow the commands, I guess the atfwd.c routes them but I can not make much sense because I have not studied the whole project, is this file the "AT SERVER" ?

there must be a point where the commands are lost in the jungle of the binary firmware and there the thing happens

in the following paper it is clearly stated in the introduction that

https://www.researchgate.net/publication/4372824_Secure_User-Plane_Location_SUPL_Architecture_For_Assisted_GPS_A-GPS

Control-plane A-GPS: Here, the assistance messages are sent using explicit, pre-defined cellular network (GSM/WCDMA) signaling structures. The initial A- GPS systems all used the control-plane for the transmission of assistance data. However, as the number of A-GPS capable users increase in the cellular system, there is significant potential of the control-plane A-GPS exhausing the system capacity.

User-plane A-GPS: Here, the assistance messages are sent using an IP data connection and thus use a GPRS/EDGE/HSPA connection without requiring any wireless standard specific messaging. The user- plane A-GPS has recently been proposed as an alternative to the control-plane A-GPS to avoid system capacity exhaustion due to assistance related signaling.

the practical solution is to delete the SUPL server entry and load the data with manual injection and leave the "MS Assisted" bits to 0 to avoid leaking of information to the cellular network or make a home SUPL server though Google has bought the patent according to Rob Braxman and you may run into legal problems

I think this is enough for the time being except if you know something more about the code, up to what point the AT commands can be followed, where their values are stored and what mechanism reads those values and how it implements the commands etc

Biktorgj commented 1 year ago

@Biktorgj I had a look at and in the code for the first time, first of all the 39MBytes for a binary is enormous, just imagine that all the Linux firmware is around 220MBytes, I thought it was some KBytes just for implementing the channel sharing, the device must be to a very large extent a SDR modem, it seems that implements whole stacks of protocols, how much in binary size is an Osmocom implementation by the way ?

Of course it is, the ADSP has its own bootloader, is it's own real time operating system in itself (rex), and then has its own set of applications. It not only includes the RF parts, it is an entire mobile equipment unit (think srsUE), has different applications to handle multiple kinds of stuff (from the AT & QMI server to an http client, email SMTP client or a ftp client) and you can even install (trusted, signed) applications (think widevine in an Android phone).

I tried to follow the commands, I guess the atfwd.c routes them but I can not make much sense because I have not studied the whole project, is this file the "AT SERVER" ?

atfwd.c is the AT command forwarder. It registers commands onto the ADSP AT server, so it has priority over them to answer to things. The ADSP has a base set of commands, which mostly consist of the 3GPP base commands + some qualcomm and quectel specific ones. The ATFWD service overrides some of them and adds a bunch more to do specific things from the userspace.

there must be a point where the commands are lost in the jungle of the binary firmware and there the thing happens ... the practical solution is to delete the SUPL server entry and load the data with manual injection and leave the "MS Assisted" bits to 0 to avoid leaking of information to the cellular network or make a home SUPL server though Google has bought the patent according to Rob Braxman and you may run into legal problems

What I still don't understand is why you're trying to use it the closed source alternative when you can use the opensource one. See, the default method in every distro, as far as I know, is to let the main SoC do all the unified location tracking and not the modem, it adds GPS, cell info as provided from the modem, and available wifi networks, and queries that against Mozilla's location services. The A-GPS data is preloaded by EG25-manager in a privacy friendly way so the modem can get a GPS fix faster, and cell data doesn't contain any user information in the request. What you're doing is telling the modem to do all that by itself via the closed source OS in the DSP, which, incidentally, can't add nearby SSIDs since it doesn't have an internal WLAN adapter, so accuracy will probably be worse in some scenarios.

Is there some accuracy or speed issue you're trying to fix? The only way to actually know what is being sent via the ADSP interface would be to set up your own SUPL server for which you can track down every sent request (maybe before then forwarding it to somewhere else). Given the constraints of the ADSP I have my doubts if that would even be sent via a secure, encrypted channel, so probably all those requests will be sent in the open (nobody would want an expired cert)

ghost commented 1 year ago

@Biktorgj I am using the stock firmware for various legal reasons, I will try to study the project in detail, the modem gives the choice to insert an SSL certificate for the SUPL server, else it is a regular connection, I will may try to capture an inspect those packages

ghost commented 1 year ago

the MDM drives the Qorvo 77031 and the last stage amplifier Qorvo 77033, maybe the 77033 is solely driven by the 77031

since there doesn't seem anything to exist in the middle this means that everything else is software defined so theoretically someone can start porting the Osmocom if it is easy to drive the Qorvo chips for which the datasheets are available on the web

AndreySV commented 1 year ago

@mouffa have you been able to enable supl functionality? I've set SUPL url to my public server

AT+QGPSSUPLURL="XX.XX.XX.XX:7276"
AT+QGPSCFG="lbsapn",16,1,"myprovider.internet.apn"'
AT+QGPSCFG="plane",0
AT+QGPS=1,2

But I don't see any connection attempt on my server. Do you have any idea, what is wrong?

ghost commented 1 year ago

@AndreySV I tested it after many months by inserting a SIM and opening 3 terminals

  1. sudo cat /dev/ttyUSB2
  2. journalctl -f
  3. AT commands

I had the wifi and modile data off so after turning on the engine I got in (1) a URC of the last injected Almanac back in October, after some minutes I turned on the wifi and I got a URC of the new Almanac and in (2) that the data were successfully injected by eg25-manager.

You could try the same let's say for an hour by keeping the wifi and mobile data off to see whether the SUPL can go through the SS7, eg25-manager will be trying to download and inject the data.

I don't know whether you can tell the manager not to do so without changing the code and recompiling it, that way you could test whether the engine will try to use the SUPL instead.

If the engine can use the SUPL only through a data connection then you will need the mobile data on and this the reason you have to stop the manager trying to inject the data.

AndreySV commented 1 year ago

@mouffa , thanks for the answer. I've done all of that.

eg25-manager injects XTRA data, that has similar functionality. I explicitly disabled XTRA on the modem and in eg25-manager as well, so nothing is going to be injected.

eg25-manager doesn't support SUPL. IMHO firmware should connect supl server by itself. Data connection is on. AT+CGPADDR returns ip addresses for active data context. I've tried all of that with ModemManager and eg25-manager completely disabled. But no lock so far. I don't see any connection attempt on my server.

ghost commented 1 year ago

@AndreySV the SUPL server provides the XTRA data as an Almanac file, since you are in Mobile Station based mode you need the XTRA enabled, in MS assisted mode you would be assisted by the SS7 but it depends on the infrastructure, probably no carrier supports such mechanisms anymore

ghost commented 1 year ago

@AndreySV eg25-manager gets the initial configuration from here,

https://gitlab.com/mobian1/eg25-manager/-/tree/master/data

those files are located somewhere either in /etc/eg25-manager or /usr/share/eg25-manager, you can try to edit the appropriate file to get the desirable initial conditions

eg25-manager is supposed to be able to be controlled by mmcli but for me it hadn't been working despite mmcli's messages on the terminal, it even offers the configuration of a SUPL server which is probably achieved by AT commands through eg25-manager though the manager itself is not programmed to configure the SUPL as far as I can see in the source code

AndreySV commented 1 year ago

@mouffa Let's make it clear., XTRA and SUPL are different things to achieve the same goal - quick TTFF. XTRA is Qualcomm specific method to provide AGNSS data. There are analogs for other vendors like EPO for MediaTek. In both cases simple file in proprietary is downloaded over HTTP(S) and is uploaded to GNSS engine. eg25-manager handles only Qualcomm's XTRA.

SUPL is generic vendor independent protocol on top of TCP/IP to get AGNSS data. It is possible to use non-Carrier's SUPL server usually. For example, most Android phones use supl.google.com. According to the Quectel's documentation SUPL is implemented by firmware and commands to configure it are in the documentation. I'm trying to configure it manually without ModemManager (mmcli) and eg25-manager to see whether it's worth to implement at all.

ghost commented 1 year ago

@AndreySV I just noticed that the AT+QGPS=1,2 command doesn't look right, the first number is the mode and the second is an optional parametre that sets the maximum time to fix the position, this means that you start the engine as Stand-alone and you let it only 2 seconds to fix the position, this is why it does not try to access the SUPL, the right commands are

AT+QGPS=1 AT+QGPS=2 AT+QGPS=3 AT+QGPS=4

let's assume the 2 systems are completely independent, that is the SUPL can not be used to inject XTRA data

the first step is to try to start the engine in "MS Assisted"(3) mode in case the SUPL is considered as an assisted process and in "Speed Optimal"(4) mode so that the engine is forced to try all the available methods

a further step is to inject a certificate though it is not stated somewhere that this is mandatory, maybe without a certificate it won't attempt to connect to the server

eg25-manager can be configured through the .toml files as to what almanac file to download and how to inject it, so it is would be able to download an EPO file and inject it with the appropriate commands

ghost commented 1 year ago

@AndreySV try the MS Based too AT+QGPS=2 and keep watching the modem output for any URCs concerning the SUPL, it should inform you that it connected to the server

ghost commented 1 year ago

@AndreySV I tried modes 2,3,4 indoors with XTRA off, mobile data on and the Google SUPL but I neither got any unusual URCs nor any kind of fixing (Error 516)

AndreySV commented 1 year ago

@mouffa I've tried different values for AT+QGPS= as well already. But result is the same, no connection to my server, no GPS speedup.