Tile Indexer failed to index feature tile table

spyhunter99 commented 7 years ago

Please fill out as much known and relevant information as possible.

Version Information:

GeoPackage Android Map Version: 1.4.1, build script is below
GeoPackage Android Map Source: central
Device or Emulator: Genymotion
Android Studio Version:
Android Version Number:
Google Play services Version: n/a
Android SDK Version: API23
Platform & OS: 6.0.0 Genymotion vm
Other Relevant Libraries:

Expected Results:

Not entirely sure, still experimenting with the feature tiles, however i'm not sure why the indexer is failing

Observed Results:

indexer spit out the following error message. no tiles were produced either, but this may be not related

Output:

08-19 20:05:49.290 4365 4365 W CursorWindow: Window is full: requested allocation 9063240 bytes, free space 1919484 bytes, window size 2097152 bytes 08-19 20:05:49.349 4365 4365 W CursorWindow: Window is full: requested allocation 9063240 bytes, free space 2096540 bytes, window size 2097152 bytes 08-19 20:05:49.350 4365 4365 E CursorWindow: Failed to read row 0, column 0 from a CursorWindow which has 0 rows, 16 columns. 08-19 20:05:49.350 4365 4365 E CursorWindow: Failed to read row 0, column 0 from a CursorWindow which has 0 rows, 16 columns. 08-19 20:05:49.350 4365 4365 E CursorWindow: Failed to read row 0, column 1 from a CursorWindow which has 0 rows, 16 columns. 08-19 20:05:49.350 4365 4365 E CursorWindow: Failed to read row 0, column 1 from a CursorWindow which has 0 rows, 16 columns. 08-19 20:05:49.353 4365 4365 W System.err: mil.nga.geopackage.GeoPackageException: Failed to Index Table. GeoPackage: USA_adm, Table: USA_adm1 08-19 20:05:49.353 4365 4365 W System.err: at mil.nga.geopackage.extension.index.FeatureTableIndex.indexTable(FeatureTableIndex.java:122) 08-19 20:05:49.353 4365 4365 W System.err: at mil.nga.geopackage.extension.index.FeatureTableCoreIndex.index(FeatureTableCoreIndex.java:162) 08-19 20:05:49.353 4365 4365 W System.err: at mil.nga.geopackage.features.index.FeatureIndexManager.index(FeatureIndexManager.java:203) 08-19 20:05:49.353 4365 4365 W System.err: at mil.nga.geopackage.features.index.FeatureIndexManager.index(FeatureIndexManager.java:166)

Steps to Reproduce:

Step One
Step Two
...

Relevant Code:

This part is the entry point, finds the geo package files, then creates the provider and tile sources

https://github.com/osmdroid/osmdroid/blob/feature/%23665/OpenStreetMapViewer/src/main/java/org/osmdroid/samplefragments/data/GeopackageFeatureTiles.java

The tile provider, doesn't do much besides calls drawTile

https://github.com/osmdroid/osmdroid/blob/feature/%23665/osmdroid-geopackage/src/main/java/org/osmdroid/gpkg/GeoPackageFeatureTileProvider.java

This part handles the opening/closing and setting up the index.

https://github.com/osmdroid/osmdroid/blob/feature/%23665/osmdroid-geopackage/src/main/java/org/osmdroid/gpkg/GeopackageFeatureTilesOverlay.java

Test Files:

Linked from http://www.gadm.org/download, the USA state borders, queried against the second feature tile table "USA_adm1" containing state boundaries.

http://biogeo.ucdavis.edu/data/gadm2.8/gpkg/USA_adm_gpkg.zip

Additional Information:

build script


    compile ("mil.nga.geopackage:geopackage-android:1.4.1"){
        exclude group: 'com.google.android.gms', module:'play-services'
        exclude group: 'com.google.maps.android', module:'android-maps-utils'
        exclude group: 'com.android.support',module: 'support-v13'
    }

    compile 'com.j256.ormlite:ormlite-android:5.0'
    compile 'mil.nga.geopackage:geopackage-core:1.3.1'
    compile 'ar.com.hjg:pngj:2.1.0'
    compile 'mil.nga:tiff:1.0.3'

    compile 'ar.com.hjg:pngj:2.1.0'
    compile 'mil.nga:tiff:1.0.0'

spyhunter99 commented 7 years ago

actually, i narrowed it down this

     FeatureIndexManager indexer = new FeatureIndexManager(ctx, geoPackage, featureDao);
        indexer.setIndexLocation(FeatureIndexType.GEOPACKAGE);
        int indexedCount = indexer.index();   //throws here

after disabling the indexer, the same error message is displayed while attempting to draw the tiles.

Failed to read row 0, column 0 from a CursorWindow which has 0 rows, 16 columns.

bosborn commented 7 years ago

So this appears to be caused by large blobs in the geometry columns. For the GeoPackage you specified, the USA_adm1 table has two Alaska features and a Florida feature with geometry blobs larger than 2mb. Apparently the CursorWindow limit in Android is 2mb. Off the top of my head and a few quick searches, I'm not aware of a workaround.

I put some error checking in on geopackage-android which allows me to index and draw tiles for all states except Florida and Alaska. https://github.com/ngageoint/geopackage-android/commit/d74cbd305ca6169bdd335a3fbe1148553f1a47ea

spyhunter99 commented 7 years ago

Thanks for looking into this. Android strikes again. Can you recommend any other sample feature sets that I can do some tests with?

On Aug 21, 2017 4:24 PM, "Brian Osborn" notifications@github.com wrote:

So this appears to be caused by large blobs in the geometry columns. For the GeoPackage you specified, the USA_adm1 table has two Alaska features and a Florida feature with geometry blobs larger than 2mb. Apparently the CursorWindow limit in Android is 2mb. Off the top of my head and a few quick searches, I'm not aware of a workaround.

I put some error checking in on geopackage-android which allows me to index and draw tiles for all states except Florida and Alaska. ngageoint/geopackage-android@d74cbd3 https://github.com/ngageoint/geopackage-android/commit/d74cbd305ca6169bdd335a3fbe1148553f1a47ea

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ngageoint/geopackage-android-map/issues/5#issuecomment-323843686, or mute the thread https://github.com/notifications/unsubscribe-auth/AB4kGZSWqYqRGbVNxyvMXBk_uIdV7M0yks5saeeYgaJpZM4O8elQ .

bosborn commented 7 years ago

Most of these are linked from the GeoPackage sample data section, but example data below github file limits are here. states10.gpkg would give you something similar to your example. sample1_2.gpkg is United States counties. This new Washington DC Vector one has 28,584 features.

We also have a rivers one here.

bosborn commented 7 years ago

We might be able to use this library to workaround Android API SQL limitations. https://bitbucket.org/almworks/sqlite4java (Apache License 2.0)

bosborn commented 7 years ago

Created issue ngageoint/geopackage-android#41 and closing here.

ngageoint / geopackage-android-map