Keep local database of scanned APs so users can export their data

[cpeterso]

With a local database, we can add enable users to export the data they've scanned. We also need a local database so the stumbler can show the user a map of their scanning history.

We might consider filtering our reports for already-reported APs unless:

• the latest scan has a stronger signal strength (i.e. has a more accurate location)
• the AP has moved since the most recent scan
• the most recent scan is older than X days (to remind the server that the AP is still exists)

[hannosch]

Looking at signal strength is probably hard. The studies I've seen suggest that you get a Gaussian distribution of strength measurements, even for the exact same device and constant environmental factors. In order to figure out what the mean signal strength is, we need multiple measurements. And then it's useful for the server side to get all of them, to average over multiple devices, multiple people holding their devices differently and different external factors like an empty or crowded Starbucks :)

One thing we could do is limit the number of AP's we report back. So for example only report back the top ten of AP's sorted by signal strength.

Another thing is to limit the number of measurements for the "same place". Currently the stumbler seems to report data almost every second. At walking speed, while waiting on green lights or while waiting on public transport we report a lot of duplicates. So it might be useful if there's a way to only report data when the location changed at least by some margin like 1-10 meters. A good approximation might be any change in the 4th decimal place of either the lat or lon (http://en.wikipedia.org/wiki/Decimal_degrees). Currently Google claims to be able to do 6m precision in indoor buildings, so if we aim for 10m, we shouldn't lose any practically relevant data for the foreseeable future.

[cpeterso]

Android's Location API lets us specify a minimum distance between updates, but our current minimum is only 0.5 meters! :)

https://github.com/dougt/MozStumbler/blob/5ed430179604618fdaf231463f58e591cbf54487/src/org/mozilla/mozstumbler/Scanner.java#L28

[dougt]

{
   "items":[
      {
         "wifi":[
            {
               "signal":-73,
               "key":"CC0B53201A6E5AB04E40F339B695E8CDBDAEC7C3",
               "frequency":2432
            },
            {
               "signal":-60,
               "key":"E5963091E2C58F0AA4340287DECC769F727CEF49",
               "frequency":2412
            },
            {
               "signal":-67,
               "key":"62264F32D3BE01EEBA01398C36990C603F32A6B5",
               "frequency":2457
            },
            {
               "signal":-57,
               "key":"BE33E0E4D0FED376ED1D4659549F1C87CE250A13",
               "frequency":2432
            },
            {
               "signal":-69,
               "key":"F16401059E047D31869E59A806FA36DD17829CAC",
               "frequency":2437
            },
            {
               "signal":-80,
               "key":"1E2C4154EB2247241E42CED680A62539B8F0A64A",
               "frequency":2417
            },
            {
               "signal":-78,
               "key":"2BD9DF16343C9E6736E7AB15BCB14D1CFB167A50",
               "frequency":2432
            },
            {
               "signal":-85,
               "key":"2A721CB6F8BA775C1D443C33B25F837477A5039D",
               "frequency":5765
            },
            {
               "signal":-71,
               "key":BB65DF6C91D37407C066ADAA106A1D4855C67D29",
               "frequency":5200
            },
            {
               "signal":-72,
               "key":"A0F42719D75FAAA9EF08637CCFBFC3F87E238CFE",
               "frequency":2447
            }
         ],
         "lon":-117.85019075497985,
         "time":2013-07-27T14:38Z",
         "token":"5c836fb2-70bc-4b6a-b42f-9f0ae68d2916",
         "altitude":-124,
         "cell":[
            {
               "asu":20,
               "radio":"umts",
               "mnc":"410",
               "psc":-1,
               "mcc":"310",
               "cid":55674569,
               "lac":55537
            }
         ],
         radio":"gsm",
         "accuracy":10,
         "lat":33.67917337920517
      }
   ]
}

{
   "items":[
      {
         "wifi":[
            {
               "signal":-73,
               "key":"CC0B53201A6E5AB04E40F339B695E8CDBDAEC7C3",
               "frequency":2432
            },
            {
               "signal":-60,
               "key":"E5963091E2C58F0AA4340287DECC769F727CEF49",
               "frequency":2412
            },
            {
               "signal":-67,
               "key":"62264F32D3BE01EEBA01398C36990C603F32A6B5",
               "frequency":2457
            },
            {
               "signal":-57,
               "key":"BE33E0E4D0FED376ED1D4659549F1C87CE250A13",
               "frequency":2432
            },
            {
               "signal":-69,
               "key":"F16401059E047D31869E59A806FA36DD17829CAC",
               "frequency":2437
            },
            {
               "signal":-80,
               "key":"1E2C4154EB2247241E42CED680A62539B8F0A64A",
               "frequency":2417
            },
            {
               "signal":-78,
               "key":"2BD9DF16343C9E6736E7AB15BCB14D1CFB167A50",
               "frequency":2432
            },
            {
               "signal":-85,
               "key":"2A721CB6F8BA775C1D443C33B25F837477A5039D",
               "frequency":5765
            },
            {
               "signal":-71,
               "key":BB65DF6C91D37407C066ADAA106A1D4855C67D29",
               "frequency":5200
            },
            {
               "signal":-72,
               "key":"A0F42719D75FAAA9EF08637CCFBFC3F87E238CFE",
               "frequency":2447
            }
         ],
         "lon":-117.85018304362893,
         "time":2013-07-27T14:38Z",
         "token":"5c836fb2-70bc-4b6a-b42f-9f0ae68d2916",
         "altitude":-105,
         "cell":[
            {
               "asu":20,
               "radio":"umts",
               "mnc":"410",
               "psc":-1,
               "mcc":"310",
               "cid":55674569,
               "lac":55537
            }
         ],
         radio":"gsm",
         "accuracy":25,
         "lat":33.67908130399883
      }
   ]
}

These are two reports fired within a few seconds of each other. As you'll note, the only difference is gps. It looks like with whatever we're doing, we see two gps reports. The second one being better. I'll look at what the first report had that the second report didn't (Maybe we need to restrict something to the # of sats)

[Djfe]

quote from the mailing list: "would be nice if could use the stumbler app as GeoTracker, too. If I go hiking or visiting a foreign city, I always start Runtastic to track my route. So I can show it my friends later or watch where I was going. Stumbling and tracking at same time would be great (+ some way to share the route with my friends, or a simple API to get the data so I can implement sharing my self)."

->gpx support or maybe even more formats

and for guessing the position based on wifi aps: you can possibly guess the direction and distance to the source and build a function that represents your position on the radius with a similar signal strength around it if there is more than one wifi ap you can overlap them and get the intersection where you are standing which gets more accurate the more aps are available

cell towers are probably only helpful for a general position in the world but if there are no wifi aps (or only one) around it you can still guess a radius around the tower where you are standing

EDIT: I don't know what of this is implemented into the server already but I know that a few weeks ago the default radius of wifis aps was still 500meters and the one of cell towers several kilometers so I guess this needs to get changed still, or where there major changes to the positioning calculation in the past weeks?

[Djfe]

We should change the priority of this issue to p1 as 337 (edit: #336) needs this to be fixed first and is a p1 issue itself.

[lhirlimann]

I'd love to share my gps traces with openstreetmap with a one click button.

[naoliv]

Could it export GPX and maybe OpenCellID data too? (to avoid running 3 apps and having a possible concurrence like #708)

[crankycoder]

Closing this as we have local storage in the form of JSON bundles. I've opened #1255 to formally add exportable data.