n76 / Local-GSM-Backend

GSM LocationProvider backend for µg UnifiedNlp
Apache License 2.0
91 stars 23 forks source link

Doesn't work on dual SIM phone #71

Closed tastytea closed 7 years ago

tastytea commented 8 years ago

Hi, GSM backend never worked for me. I'm using 1.4.1 from F-Droid. I downloaded the data from OpenCellID, Mozilla, OpenCellID and Mozilla, and lacells – no difference. adb logcat | grep NlpLocationBackendFuser outputs nothing while running for ~10 minutes.

Other backends work:

n76 commented 8 years ago

How did you download the data? Using the built in facility within the backend on the phone or using scripts on a PC?

If using scripts on a PC, the database location changed to better fit the Android app storage scheme and is now located in the /sdcard/Android/data/org.fitchfamily.android.gsmlocation/files/ directory

tastytea commented 8 years ago

I used the built in functionality.

n76 commented 8 years ago

Can you forward me the log that was written when the database was generated? It should be located at /sdcard/Android/data/org.fitchfamily.android.gsmlocation/files/lacells_gen.log

tastytea commented 8 years ago

Here it is: lacells_gen.log.txt

n76 commented 8 years ago

Looks like the database was written properly. I've put a copy of the backend a https://dl.dropboxusercontent.com/u/19663978/gsmLocation.apk which has some additional logging enabled. It might (if I've put enough logging in suitable places) help figure out what is going wrong.

Unfortunately, you will have to uninstall the F-Droid version to install mine and that will cause your current database to be deleted.

tastytea commented 8 years ago

Thanks, I'll try that tomorrow, it's 2:30 here :-)

n76 commented 8 years ago

When you do, pull down the one time stamped after this comment, I've put up a second one with a lot more debug logging enabled.

p.s. You can use something like: adb logcat | grep -i gsmloc_

Isgar commented 8 years ago

As I seem to have the same problem (#70), I want to try the debug version. Is there a way to preserve/reinsert the database without downloading and parsing the whole thing again?

I have a shell with root access.

n76 commented 8 years ago

You can grab the current database by

adb pull /sdcard/Android/data/org.fitchfamily.android.gsmlocation/files/lacells.db

And you should be able to restore it by pushing that file by

adb push lacells.db /sdcard/Android/data/org.fitchfamily.android.gsmlocation/files/lacells.db.new

I am using "should" rather than "can" as I have an issue on my phone where I can't pull files from my phone that are as large as my lacells.db so I can't verify this at the moment.

Isgar commented 8 years ago

I moved the database around on my sdcard, worked well

I/gsmloc_service( 9892): Binder OPEN called
I/gsmloc_service( 9892): Starting location backend
I/gsmloc_service( 9892): Starting reporter thread
I/gsmloc_database( 9892): Attempting to open database.
I/gsmloc_database( 9892): query result: xxx, xx, xxxxx, xxxxxxxxx, xx.xxxxxx, xx.xxxxxx, xxx.x, xxx
I/gsmloc_database( 9892): Final result: xx.xxxxxxxxxxxxxxxxx, xx.xxxxxxxxx, xxx.x
I/gsmloc_database( 9892): Cell info found: mcc=xxx, mnc=xx, lac=xxxxx, cid=xxxxxxxxx
I/gsmloc_database( 9892): DB Cursor empty for: mcc=xxxxxxxxxx, mnc=xxxxxxxxxx, lac=xxxxxxxxxx, cid=xxxxxxxxxxx
I/gsmloc_service( 9892): onServiceStateChanged: Location[gsm xx.xxxxxx,xx.xxxxxx acc=xxx et=?!? {Bundle[{AVERAGED_OF=1}]}]
I/gsmloc_service( 9892): onCellLocationChanged: Location[gsm xx.xxxxxx,xx.xxxxxx acc=xxx et=?!? {Bundle[{AVERAGED_OF=1}]}]
I/gsmloc_service( 9892): onCellInfoChanged: Location[gsm xx.xxxxxx,xx.xxxxxx acc=xxx et=?!? {Bundle[{AVERAGED_OF=1}]}]

I blanked out all the numbers as I don't want to publish precise information of my position, but if you need them uncensored I can send them to you.

In any way, the location information were quite accurate.

n76 commented 8 years ago

That DB Cursor empty for log indicates that at least one cell/mobile tower you phone is seeing is not it the database. You could contribute to OCID or Mozilla to improve your results.

What version of Android are you running and what is the hardware? Your logging looks a bit different than mine, so your phone's TelephonyManager may be returning valid data for some calls that mine returns empty or null results.

Isgar commented 8 years ago

I am using a Nexus 4 (mako) running CM12.1

Will look into contributing ;)

But the last 3 entries mean that it went well so far?

tastytea commented 8 years ago

Sorry for the delay, had to go to the hospital. Here's the log: I/gsmloc_service(12881): Binder OPEN called I/gsmloc_service(12881): Starting location backend I/gsmloc_service(12881): Starting reporter thread I/gsmloc_TelephonyHelper(12881): getAllCellInfo() returned null or empty set I/gsmloc_TelephonyHelper(12881): getAllCellInfoWrapper() returned nothing, trying legacyGetCellTowers( ). I/gsmloc_database(12881): Attempting to open database. I/gsmloc_database(12881): DB Cursor empty for: mcc=262, mnc=2, lac=40012, cid=131895734 I/gsmloc_TelephonyHelper(12881): Unknown cell tower detected: mcc=262, mnc=262, cid=131895734, lac=40012 I/gsmloc_TelephonyHelper(12881): getNeighboringCellInfo() returned null or empty set. I/gsmloc_TelephonyHelper(12881): No known cell towers found. I/gsmloc_TelephonyHelper(12881): getTowerLocations(): No tower information. I/gsmloc_service(12881): onServiceStateChanged: null position I/gsmloc_TelephonyHelper(12881): getAllCellInfo() returned null or empty set I/gsmloc_TelephonyHelper(12881): getAllCellInfoWrapper() returned nothing, trying legacyGetCellTowers(). I/gsmloc_TelephonyHelper(12881): Unknown cell tower detected: mcc=262, mnc=262, cid=131895734, lac=40012 I/gsmloc_TelephonyHelper(12881): getNeighboringCellInfo() returned null or empty set. I/gsmloc_TelephonyHelper(12881): No known cell towers found. I/gsmloc_TelephonyHelper(12881): getTowerLocations(): No tower information. I/gsmloc_service(12881): onCellLocationChanged: null position I/gsmloc_TelephonyHelper(12881): getAllCellInfo() returned null or empty set I/gsmloc_TelephonyHelper(12881): getAllCellInfoWrapper() returned nothing, trying legacyGetCellTowers(). I/gsmloc_TelephonyHelper(12881): Unknown cell tower detected: mcc=262, mnc=262, cid=131895734, lac=40012 I/gsmloc_TelephonyHelper(12881): getNeighboringCellInfo() returned null or empty set. I/gsmloc_TelephonyHelper(12881): No known cell towers found. I/gsmloc_TelephonyHelper(12881): getTowerLocations(): No tower information. I/gsmloc_service(12881): onCellInfoChanged: null position

I'm pretty sure the Cell Tower is in the Database, since I uploaded it and it worked with LocalGSMLocationProvider, which uses opencellid too.

tastytea commented 8 years ago

I've also noticed that AIMSICD reports different cell info: LAC: 514 CID: 9163

I'm using a Huawai Holly running CM 12.1 with 2 SIM cards

tastytea commented 8 years ago

I drove around town today and checked numerous places, got no location anywhere.

n76 commented 8 years ago

I downloaded the latest cell data from Mozilla (which includes data from OCID) and did a grep for the cell ID that showed as not in your database:

I/gsmloc_TelephonyHelper(12881): Unknown cell tower detected: mcc=262, mnc=262, cid=131895734, lac=40012

$ grep 131895734 MLS-full-cell-export-2016-05-24T000000.csv
UMTS,262,3,40012,131895734,282,9.7623619,52.4134604,1346,273,1,1429864902,1462470649,
UMTS,262,3,40312,131895734,,9.7618626,52.4084585,643,10,1,1438068836,1443197717,

Columns are: radio,mcc,net,area,cell,unit,lon,lat,range,samples,changeable,created,updated,averageSignal

which is really: radio,mcc,mnc,lac,cid,...

It shows a known cell in your area with matching lac but the mnc differs. Your phone is detecting a '2' (Vodaphone) while the database has '3' (E-Plus) so my backend does not think this is a match.

The information reported by AIMSICD is insufficient for me to tell if there is a match or not, but it looks like there should be one in your database for Vodaphone. Why AIMSICD detects it but the backend does not cannot be answered from the logcat you posted.

$ grep '514,9163' MLS-full-cell-export-2016-05-24T000000.csv
GSM,262,2,514,9163,,9.7349619,52.4110454,2624,17,1,1420671808,1457994451,
GSM,724,4,15065,45912,,-59.3356319,-15.2543514,9163,7,1,1423226003,1425320188,

It looks like your phone, like mine, only returns information about the tower it is connected to. In my area my phone will hand off between three or so towers even if it is not moving. It might be interesting to see if yours will do that too and what is reported in the Unknown cell tower detected: lines in logcat if/when that happens.

At present, it looks to me like the reason the backend is not working for you is because the towers in your area, as reported by Mozilla and/or OCID, do not match what your phone is seeing. That could be corrected by running a network stumbling app from one of those two projects.

tastytea commented 8 years ago

Your phone is detecting a '2' (Vodaphone) while the database has '3' (E-Plus) so my backend does not think this is a match.

I have 2 Simcards, Vodafone and E-Plus/O2. So maybe my phone screws things up?

It looks like your phone, like mine, only returns information about the tower it is connected to. In my area my phone will hand off between three or so towers even if it is not moving. It might be interesting to see if yours will do that too and what is reported in the Unknown cell tower detected: lines in logcat if/when that happens.

I'll try to catch that in logcat.

n76 commented 8 years ago

I may have a logic error that only manifests itself on a dual SIM phone. If you run the Mozilla backend I think it has some logging already enabled that shows what cell/mobile towers and WLAN/WiFi APs it sees. Does that show a different carrier for the mobile than mine?

tastytea commented 8 years ago

Mozilla back-end doesn't work and shows the same data: D/IchnaeaBackendService( 8217): request: {"radioType":"gsm","cellTowers":[{"radioType":"gsm","mobileCountryCode":262,"mobileNetworkCode":2,"locationAreaCode":40012,"cellId":131895734,"signalStrength":2147483647,"asu":0}],"wifiAccessPoints":[],"fallbacks":{"lacf":true,"ipf":false}}

org.gfd.gsmlocation doesn't report cell information but works.

So I guess my Phone/ROM is the problem and org.gfd.gsmlocation takes a more "fuzzy" approach?

n76 commented 8 years ago

I will take a look at org.gfd.gsmlocation and see how it gets the list of cell towers

n76 commented 8 years ago

Looks like the org.gfd.gsmlocation backend only matches LAC and CID with looking up the cell tower information so a mismatch in NMC is ignored by it. I am not sure that is a good way to do things. Could different operators end up using the same LAC and CID? Seems like that is possible.

I am going to leave this report open on the hope that I can come across some information on how to handle dual SIM phones better but I don't think I can do much to help you at the moment. I am going to change the title though to make it clearer to someone looking at the list of issues what I think the underlying problem is.

uvok commented 8 years ago

I'm using a dual-sim phone (Asus Zenfone 2), the location lookup works for me (although I sometimes have to disable and re-enable location services via Android settings). So it shouldn't be a problem with Dual-SIM phones in general.

tastytea commented 8 years ago

Thanks for the info. Looks like my ROM is broken then. For future reference: If you're using CyanogenMod 12.1-20160107-UNOFFICIAL-HOL-U19, your ROM is probably the problem.

tastytea commented 8 years ago

Hi, I found a "solution" today. I had configured SIM card 1 for internet and SIM card 2 for calls/SMS. When I switched calls/SMS to SIM 1, I got correct information.

Lee-Carre commented 2 years ago

I am going to change the title though to make it clearer to someone looking at the list of issues what I think the underlying problem is.

👍. I read this ticket because of the title, and my future intentions.

I found a [workaround] today. I had configured SIM card 1 for internet and SIM card 2 for calls/SMS. When I switched calls/SMS to SIM 1, I got correct information.

Thanks very much for posting that.

I have a phone with 2 SIM slots. Since I do (among other types) network surveying (and seem to be the only (regular) contributor in my area(!) (MCC=234, for the curious)), on my to-do list is getting SIMs from other operators, to collect data about their networks, too.

I'll keep this workaround in mind, for when that time comes.