search

mozilla / MozStumbler

Android Stumbler for Mozilla
http://location.services.mozilla.com
Mozilla Public License 2.0
618 stars 212 forks source link

No collection: observation count increases, but no data in queue, no data uploaded #869

Closed gpoo closed 10 years ago

gpoo commented 10 years ago

I've been stumbling for some a days with 0.30, and it was worrying me a little that the "In queue" section was always in 0. My first thought it was uploading as it was collecting the information. When I saw the log, I saw my multiples positions in green text. I noticed it was kind of different than 0.21, but I thought it was one of the visual changes, too.

However, under "Sent" the number of observations, cell and wifi-access were all kept in the same numbers. I stumbled several new places, I saw the green points on the map while walking for a couple of hours, but it did not upload anything. Then when I checked the "new" places in Mozilla's location map, they were not there, and I cross-checked the points scanned in the leader page, and they were the same as before.

I just downgraded to 0.21, and did a short walk, and the observations started to change (increase) again, as well as I started to see numbers in the queue.

The new icon looks nice and the map seems more responsive, though.

My phone is a Nexus One, with Cyanogenmod-7.2.0-passion (Android 2.3.7).

rodmoreno commented 10 years ago

I can confirm this on my Moto G KitKat Android 4.4.4

I scanned for a few hours and I have come to scan up to +1000 APs Wi-Fi, but when I get to my home and visualize the window to upload the data I see around 300 APs Wi-Fi ... :S Both the observations and cells are correct, it has only happened to me with APs Wi-Fi

garvankeeley commented 10 years ago

are you seeing 'Send data' messages in the log? screenshot 2014-09-22 11 38 40

gpoo commented 10 years ago

Nope, and 0 observations to upload. Although, it seems to capture data, the map shows plenty of green dots, too.

garvankeeley commented 10 years ago

I wonder if it is a filesystem error, the app expects external storage (or 'virtual' external storage). I should make sure these types of errors are going to the GUI log, instead of just ADB log.

gpoo commented 10 years ago

What would it happen if tools like Link2SD were in use? That might make the app to think it is using the internal memory, I guess. FWIW, a while ago (for a bug in a version I don't recall at this moment) MozStumbler filled the memory. Then I got Link2SD to prevent such kind of events.

Djfe commented 10 years ago

This is the same issue I have been experiencing since the update. 0.30 seems to be broken the scanning works. Observations are "recorded" and shown on the map and in the statistics in the main activity. But nothing appears in the queue and no upload is happening at all. ->upload date doesn't change and upload button stays greyed out. (screenshots were made on 19th, last upload was on 13th)

In the log I only see GPS data (speed, lat/lon etc.) and no wifi or cell data...

I hope that this issue is only a bug in the display of the data.

Device: Xperia Arc S Kitkat

might have something to do with, might not: Kitkat doesn't allow apps access to the external sd card in certain cases

Two screenshots showing it: 1 2

garvankeeley commented 10 years ago

@djfe can you check the log window for 'Send data' messages. If you are not seeing those, that is bad.

gpoo commented 10 years ago

FWIW, this kind of record is all what I get:

screenshot-1411444743111-d

Which is, no data.

I was going to paste the other screens, but @djfe already put something similar that what I see, as well.

Djfe commented 10 years ago

@gpoo log looks exactly the same for me. @garvankeeley no send data messages at all, sadly :(

garvankeeley commented 10 years ago

The good thing, is this isn't an upload issue. The bad thing is I don't know exactly what is happening. Each blue GPS line means that the app has received a GPS that is valid for stumbling. It should be followed by a line with the cells and wifis scanned. No collection is taking place here, but why? The observation point display on the map is a bit disconnected from the reporting system, I think it will drop an observation point when no scan took place (which is wrong obviously), so you may still be seeing observation dots appear on the map.

swarnava commented 10 years ago

reproducible on Samsung Galaxy SII and Android 4.1.2

ioana-chiorean commented 10 years ago

I've lost a week of traveling abroad. Any idea how to retrieve data/?

Djfe commented 10 years ago

There is no way of retrieving it, I guess.

But you can switch to an older version until this bug is fixed: https://github.com/mozilla/MozStumbler/releases

you just have to ignore the update dialog on start.

crankycoder commented 10 years ago

We've added substantially better logging for discrete events that will show up in the debug log in the dev branch.

Specifically, we've added logging around every stumble upload is that we can see the bytes sent, and the http response that ichnaea returns.

I'm afraid we can't properly diagnose the issue, of there is an issue at all given the state of logging in the last 0.30 release. Some of these logs look like no collections have taken place.

We've closed off all the blocking bugs for a new release, so we fully expect to so a new release on Monday morning.

I'm going to close this bug for now, but please reopen a new bug if you continue to see this problem. Sorry for the unusual logging in. 0.30.

gpoo commented 10 years ago

I wonder if you are planning to release 1.0, or beta/preview.

Djfe commented 10 years ago

it won't be 1.0 since not all issues/bugs are fixed, yet

https://github.com/mozilla/MozStumbler/milestones/1.0

crankycoder commented 10 years ago

I'm pretty sure that most people reporting this bug are actually seeing a variant of : https://github.com/mozilla/MozStumbler/issues/877

The mapview tile loader doesn't seem to expire tiles ever, so this almost certainly causing confusion with users.

JensErat commented 10 years ago

No, it definitely isn't the same bug. I've been able to observe the exactly same behavior as descriped by the bug reporter with three devices:

All with most current Stock Firmware, and Stumbler installed manually and from F-Droid (tried both on each device).

The device counts seen WiFi networks (probably same issue for mobile networks), eg. I came across 6k WiFi hotspots (displayed on main screen), but when it comes to uploading the data only 1,4k hotspots have been left (the number of observed locations stayed the same, though). Unlike the report with screenshots attached, I was collecting data, but it is incomplete.

I am able to reproduce the missing wireless networks. If I'm passing the same route twice, with each software version once (0.30 and the one before), the main screen shows similar network counts, but the upload screen is missing about two thirds (again: same number of observed locations!).

By the way (although I don't believe this is relevant for the issue): language is set to German an all devices, the screenshots above are also German.

gpoo commented 10 years ago

@crankycoder How can tiling cache be related to (not) scanning data? It might be possible, but I think they should not.

I upgraded again to 0.30. Before running it, I deleted all data and cache, thinking that at least should scan 1 (to retrieve one tile, if so). After a 15 minutes (4-6 blocks), I got everything empty (0) in "Upload observations".

The main screen says it scanned 106 locations, 93 wi-fi access points and 7 cells. Still, no reports, no data.

FWIW, I did not open the map while stumbling. Then I disconnected the phone from the network, and I went to see the map (for cached tilings), and nothing, so, there was no cache either.

gpoo commented 10 years ago

FWIW, the log looks similar to what I pasted in https://github.com/mozilla/MozStumbler/issues/869#issuecomment-56555711

Djfe commented 10 years ago

this bug doesn't appear in older versions unlike #877

the bug is that you cannot upload anything as the title says

garvankeeley commented 10 years ago

As best I can tell there are 2 bugs here: 1) In 0.30 the observation counts are off, and 2) No wifi and cell scanning is happening when expected to do so

Hopefully today's update will see the first issue resolved. The second issue we will have to investigate after the update.

gpoo commented 10 years ago

What can we do to help to debug the issue?

garvankeeley commented 10 years ago

When we get this next build out today, please retest. We were a bit stuck completing other items for this coming release and couldn't focus enough on this issue until now, sorry about that.

gpoo commented 10 years ago

Just tested 0.40. The UI looks nice (it missing a withe space in Metrics, but overall it looking way better).

@garvankeeley I think by counters you mean the left part of the screenshots (it says the number of satellites, cell towers, observations and wifi access points). If so, it seems to be working.

However, there are no observations "Ready to be send". I did not put a screenshot with the log, because it is similar to the one in https://github.com/mozilla/MozStumbler/issues/869#issuecomment-56555711

So, it seems part (1) of the issue is fixed, and it missing part (2). Right?

screenshot-1412720611746

FWIW, the numbers in "Sent" it was just the observations while stumbling with 0.21 (right before to upgrade to 0.40).

garvankeeley commented 10 years ago

This is so bizarre. We added a ton more logging in hopes of catching this, obviously not enough. I am trying on a 2.3.6 device now trying to reproduce this, but haven't seen this yet.

crankycoder commented 10 years ago

Are you still using link2sd? On Oct 7, 2014 6:34 PM, "Germán Poo-Caamaño" notifications@github.com wrote:

Just tested 0.40. The UI looks nice (it missing a withe space in Metrics, but overall it looking way better).

@garvankeeley https://github.com/garvankeeley I think by counters you mean the left part of the screenshots (it says the number of satellites, cell towers, observations and wifi access points). If so, it seems to be working.

However, there are no observations "Ready to be send". I did not put a screenshot with the log, because it is similar to the one in #869 (comment) https://github.com/mozilla/MozStumbler/issues/869#issuecomment-56555711

So, it seems part (1) of the issue is fixed, and it missing part (2). Right?

[image: screenshot-1412720611746] https://cloud.githubusercontent.com/assets/877181/4551476/e931d618-4e71-11e4-8e3d-537237f083a4.png

— Reply to this email directly or view it on GitHub https://github.com/mozilla/MozStumbler/issues/869#issuecomment-58274194.

gpoo commented 10 years ago

I tried both settings, with the same outcome.

That is, yesterday I also suspected of Link2SD, so I switched back to no link2SD, uninstalled Link2SD and S2E, cleared cache, uninstalled mozstumbler, installed back (I had to reinstall Google Play Service)... several reboots... The same thing.

FWIW, I only had linked the cache to the SD.

JensErat commented 10 years ago

I just got home from running (which is a nice occasion for testing the builds, as I can always do exactly the same route and velocity is rather slow, thus results very comparable). Build 0.40 seems fine until now on my S5. Will test the other phones and builds next time, to see whether the numbers diverge between builds. The new user interface is really decent. :)

Djfe commented 10 years ago

I'm using link2sd, too. But I don't think that that is the issue.

It still doesn't show any entries in the queue

gpoo commented 10 years ago

Just to be sure there were not left overs of my previous Link2sd/s2e setting:

The same outcome. Therefore, I would discard link2sd/s2e as the cause.

garvankeeley commented 10 years ago

Are you able to get a logcat? If so, adb logat | grep -i stumbler might provide the clue we need

gpoo commented 10 years ago

Does this help?

$ platform-tools/adb logcat | grep -i stumbler
I/ActivityManager(  184): Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x10200000 cmp=org.mozilla.mozstumbler/.client.navdrawer.MainDrawerActivity bnds=[5,415][115,533] } from pid 291
I/ActivityManager(  184): Start proc org.mozilla.mozstumbler for activity org.mozilla.mozstumbler/.client.navdrawer.MainDrawerActivity: pid=802 uid=10059 gids={3003, 1015}
D/dalvikvm(  802): VFY: dead code 0x0087-0090 in Lorg/mozilla/mozstumbler/client/cellscanner/DefaultCellScanner;.addCellToList (Ljava/util/List;Landroid/telephony/CellInfo;Landroid/telephony/TelephonyManager;)Z
E/dalvikvm(  802): Could not find class 'android.telephony.CellInfoGsm', referenced from method org.mozilla.mozstumbler.service.stumblerthread.scanners.cellscanner.CellScannerNoWCDMA.addCellToList
W/dalvikvm(  802): VFY: unable to resolve instanceof 409 (Landroid/telephony/CellInfoGsm;) in Lorg/mozilla/mozstumbler/service/stumblerthread/scanners/cellscanner/CellScannerNoWCDMA;
D/dalvikvm(  802): VFY: dead code 0x0003-014b in Lorg/mozilla/mozstumbler/service/stumblerthread/scanners/cellscanner/CellScannerNoWCDMA;.addCellToList (Ljava/util/List;Landroid/telephony/CellInfo;Landroid/telephony/TelephonyManager;)Z
E/dalvikvm(  802): Could not find class 'android.widget.Switch', referenced from method org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity.onCreateOptionsMenu
W/dalvikvm(  802): VFY: unable to resolve new-instance 543 (Landroid/widget/Switch;) in Lorg/mozilla/mozstumbler/client/navdrawer/MainDrawerActivity;
D/dalvikvm(  802): VFY: dead code 0x001e-0035 in Lorg/mozilla/mozstumbler/client/navdrawer/MainDrawerActivity;.onCreateOptionsMenu (Landroid/view/Menu;)Z
I/dalvikvm(  802): Could not find method android.view.MenuItem.getActionView, referenced from method org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity.setStartStopMenuItemState
D/dalvikvm(  802): VFY: dead code 0x001c-002a in Lorg/mozilla/mozstumbler/client/navdrawer/MainDrawerActivity;.setStartStopMenuItemState ()V
I/dalvikvm(  802): Could not find method android.view.View.setAlpha, referenced from method org.mozilla.mozstumbler.client.mapview.MapActivity.dimToolbar
D/dalvikvm(  802): VFY: dead code 0x0025-0025 in Lorg/mozilla/mozstumbler/client/mapview/MapActivity;.dimToolbar ()V
D/Stumbler:MapActivity(  802): onCreate
D/Stumbler:MapActivity(  802): onStart
D/Stumbler:MainApp(  802): Service connected
D/Stumbler:MapActivity(  802): ZOOM 16
I/ActivityManager(  184): Displayed org.mozilla.mozstumbler/.client.navdrawer.MainDrawerActivity: +1s565ms (total +2m10s290ms)
V/Stumbler:ScreenOffWorkaround(  802): Total cell location updates when the screen is off: 0
D/Stumbler:Updater(  802): Installed version: 0.40.0.0
D/Stumbler:Updater(  802): Latest version: 0.40.0.0
D/Stumbler:MapActivity(  802): onPause
D/Stumbler:MapActivity(  802): onStop
D/Stumbler:MapActivity(  802): onStart
V/Stumbler:ScreenOffWorkaround(  802): Total cell location updates when the screen is off: 0
V/Stumbler:ScreenOffWorkaround(  802): Total cell location updates when the screen is off: 0
D/Stumbler:MapActivity(  802): onPause
D/Stumbler:MapActivity(  802): onStop
D/Stumbler:MapActivity(  802): onDestroy
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@407439d8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@407439d8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@40861fa0 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@40861fa0 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@4075b4b8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@4075b4b8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@406f8ae8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@406f8ae8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@408f01d8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@408f01d8 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@408f4b40 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.osmdroid.tileprovider.modules.MapTileFileStorageProviderBase$MyBroadcastReceiver@408f4b40 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.AbstractMapOverlay.<init>(AbstractMapOverlay.java:39)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.tiles.CoverageOverlay.<init>(CoverageOverlay.java:20)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.mapNetworkConnectionChanged(MapActivity.java:335)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.access$900(MapActivity.java:62)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity$5.onReceive(MapActivity.java:341)
E/ActivityThread(  802): Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.mozilla.mozstumbler.client.mapview.MapActivity$5@406ebf00 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802): android.app.IntentReceiverLeaked: Activity org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity has leaked IntentReceiver org.mozilla.mozstumbler.client.mapview.MapActivity$5@406ebf00 that was originally registered here. Are you missing a call to unregisterReceiver()?
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.mapview.MapActivity.onCreateView(MapActivity.java:202)
E/ActivityThread(  802):    at org.mozilla.mozstumbler.client.navdrawer.MainDrawerActivity.onStart(MainDrawerActivity.java:165)
garvankeeley commented 10 years ago

I added a bug for the leaking receiver, but that is minor. Other than that, everything looks correct in the log. I was hoping for some data storage warning, like the app isn't allowed to access the storage or something.

Djfe commented 10 years ago

where is the app writing the data by the way? about link2sd: I have another device, a Samsung tablet that isn't rooted and doesn't use link2sd and shows the same issues (it's android 4.2)

why can't we return to a real database? I can't see any advantages in using a file based db a normal db is very fast, data isn't deleted if the application crashes/is killed (I'm wondering whether this happens as well if your device shuts down because of an empty battery) and the switch to a file based db caused this issue a normal db has the advantage that you don't need to care about file system permissions

is there a way to detect this issue by exporting the kml file? or is the db completely empty -> so that you only get an empty kml file? could you write a special debug version for us using a new branch that writes the content of important variables into the log (like breakpoints) so that you could better see where it goes wrong

I don't think this issue is depended on the android version, I'm wondering what is the difference between our devices and your devices.

garvankeeley commented 10 years ago

where is the app writing the data by the way? http://developer.android.com/reference/android/content/Context.html#getFilesDir()

why can't we return to a real database? DB is not allowed in Fennec, and the core of this app is re-used in Fennec. There shouldn't be a need for the DB in general as we aren't doing any SQL queries.

is there a way to detect this issue by exporting the kml file? We have no idea what is happening, other than you and gpoo we don't know of anyone else seeing this. Hopefully people reading this will let us know if they see the exact same behaviour with the latest build. We have test devices for Nexus 4 and 5, Galaxy Nexus, Samsung Duos, not sure what other ones are being used, but none of them exhibit the problem.

I am stumped right now as to what the problem is. We'll try some more logging and get another build out and see if we can figure this out.

Djfe commented 10 years ago

Could you switch from saving every 50 observation to saving every 30 seconds(if there are new observations)? I think that would make more sense (walking -> less observations, driving -> more observations) -> less data would get lost (while walking) and it still wouldn't save too often

Could you make us a test fennec build with a custom api so that you could test through the server whether the fennec implementation doesn't work as well

gpoo commented 10 years ago

IIRC, the swtich from db to json file was before 0.21. If that is correct, changing the db won't change anything.

So far, the only deterministic way I foresee to catch the issue is to bisect, that would help us to know which commit introduced the regression for some devices. We know 0.21 works and 0.30 does not. Considering that there were 219 commits, it should take ~7 rebuilds.

Unfortunately, I have a deadline close so I have no time to help with anything other than testing. If somebody else has the time and energy, I would be happy to test it.

garvankeeley commented 10 years ago

Can you try these builds, based on major changes The major changes were (in order): aaa25f6b - https://www.dropbox.com/s/n127pm7jyf5fjr9/MozStumbler-release.apk?dl=0 01f8603 - https://www.dropbox.com/s/h5r3i5i3h3akhp2/MozStumbler-release.apk?dl=0

gpoo commented 10 years ago

The first one works (aaa25f6), but not the second one.

msylw commented 10 years ago

Got beaten by few minutes, but got same result, first one works, second one doesn't. On Oct 10, 2014 4:35 AM, "Germán Poo-Caamaño" notifications@github.com wrote:

The first one works (aaa25f6 https://github.com/mozilla/MozStumbler/commit/aaa25f6b8a20f120a38809eec3467ea0f7d69a2d), but not the second one.

— Reply to this email directly or view it on GitHub https://github.com/mozilla/MozStumbler/issues/869#issuecomment-58564974.

garvankeeley commented 10 years ago

Thanks! Half way between those is: 2609288 - https://www.dropbox.com/s/ps10mvds3c2ymyd/MozStumbler-release.apk?dl=0

If that fails then: df10750 - https://www.dropbox.com/s/hsyyx3hlb0apkp3/MozStumbler-release.apk?dl=0

Else if it succeeds, try: ebafaf7 - https://www.dropbox.com/s/gny5xq7xr8wvt4t/MozStumbler-release.apk?dl=0

msylw commented 10 years ago

Both work for me. On Oct 10, 2014 5:30 AM, "Garvan Keeley" notifications@github.com wrote:

Thanks! Half way between those is: 2609288 https://github.com/mozilla/MozStumbler/commit/2609288ea474205bd0ea2c90dba04bbba3edd8bc

If that fails then: df10750 https://github.com/mozilla/MozStumbler/commit/df10750cf40767e85c384340bec8e2cff2aa2b7a

Else if it succeeds, try: ebafaf7 https://github.com/mozilla/MozStumbler/commit/ebafaf755b57c347fc3d7fccb406c7cbd997b748

— Reply to this email directly or view it on GitHub https://github.com/mozilla/MozStumbler/issues/869#issuecomment-58572513.

msylw commented 10 years ago

By both I ment of course 2609288 and ebafaf7. On Oct 10, 2014 5:53 AM, "Michał Sylwester" msylwester@gmail.com wrote:

Both work for me. On Oct 10, 2014 5:30 AM, "Garvan Keeley" notifications@github.com wrote:

Thanks! Half way between those is: 2609288 https://github.com/mozilla/MozStumbler/commit/2609288ea474205bd0ea2c90dba04bbba3edd8bc

If that fails then: df10750 https://github.com/mozilla/MozStumbler/commit/df10750cf40767e85c384340bec8e2cff2aa2b7a

Else if it succeeds, try: ebafaf7 https://github.com/mozilla/MozStumbler/commit/ebafaf755b57c347fc3d7fccb406c7cbd997b748

— Reply to this email directly or view it on GitHub https://github.com/mozilla/MozStumbler/issues/869#issuecomment-58572513 .

gpoo commented 10 years ago

Same here.

garvankeeley commented 10 years ago

Excellent, and the only commit left is 7445a82, I don't need to do a build for, because that commit was overwritten by later changes. So commit 01f8603 - block-fake-GPS is the one.

Djfe commented 10 years ago

should I test those builds as well? or is the cause/issue clear already?

Djfe commented 10 years ago

we might need a different db for mozstumbler anyways to achieve #3 then it would actually make more sense to use a real db, to easily filter out already sent observations when uploading but keep the sent ones to show them on the map they could be filtered through their lat/lon as well to only load the visible ones (where lat/lon < or > x/y)

crankycoder commented 10 years ago

It looks like I introduced the problem when we added filtering of gps data if there was no corresponding NMEA data.

We'll revert that change tonight. On Oct 9, 2014 6:10 PM, "Felix Baumann" notifications@github.com wrote:

should I test those builds as well? or is the cause/issue clear already?

— Reply to this email directly or view it on GitHub https://github.com/mozilla/MozStumbler/issues/869#issuecomment-58586006.

Djfe commented 10 years ago

ok, lol :D that was unexpected so it seems like there are devices like my Sony Xperia Arc S (LT18i) that don't expose NMEA data or is that ROM dependent?

EDIT: anyway, thx for fixing that bug, was probably quite some hassle