Closed eclectice closed 2 years ago
Are you doing to this on a background thread? Are you using a PowerManager WakeLock? Wondering if the file is partially copied over causing different errors. But not sure why it fully works the second time unless it is a temporary partial thing.
The first error makes it a little farther and fails to find the required tables. The second error fails a few steps earlier trying to enable foreign keys (pragma) due to a corrupted database.
Does this happen to all GeoPackages or only some? If not all, can you provide an example file?
Turned your code into a test. Ran with version 6.1.2 on a Nexus 5X API 31 emulator (closest to a Samsung Note 5 I could think of). Works fine and could not duplicate the errors.
public void safGeoPackageMapTileModuleProvider(DocumentFile[] pFile,
final Context context /*, IFilesystemCache cache*/) {
String GEO_PACKAGE_EXTENSION = ".gpkg";
//int pThreadPoolSize, final int pPendingQueueSize
//super(Configuration.getInstance().getTileFileSystemThreads(), Configuration.getInstance().getTileFileSystemMaxQueueSize());
//Log.i(IMapView.LOGTAG, "Geopackage support is BETA. Please report any issues");
//tileWriter = cache;
// Get a manager
GeoPackageManager manager = GeoPackageFactory.getManager(context);
// Available databases
//String[] list = manager.databases().stream().toArray(String[]::new);
//Log.i(IMapView.LOGTAG, "Geopackage database list: " + Arrays.toString(list));
List<String> names = new ArrayList<>();
// Import database but strip the .gpkg extension from the file name
for (int i = 0; i < pFile.length; i++) {
String name = pFile[i].getName();
name = name.replace(GEO_PACKAGE_EXTENSION, "");
//Log.i(IMapView.LOGTAG, "Geopackage database new entry: " + name);
//if (StringUtils.indexOfAny(name, list) == -1) {
//Log.i(IMapView.LOGTAG, "Geopackage database new entry: " + pFile[i].getName());
manager.importGeoPackage((pFile[i]));
names.add(name);
//}
}
// Available databases
List<String> databases = manager.databases();
//Log.i(IMapView.LOGTAG, String.format(Locale.getDefault(), "Geopackage database size: %d", databases.size()));
// Open database
for(String name : names){
//for (int i = 0; i < databases.size(); i++) {
//GeoPackage open = manager.open(databases.get(i));
GeoPackage open = manager.open(name);
//boolean newTile = tileSources.add(open);
//Log.i(IMapView.LOGTAG, String.format(Locale.getDefault(), "Geopackage database open: %s (%s)", databases.get(i), newTile ? "true":"false"));
assertNull(open.foreignKeyCheck());
assertTrue(open.enableForeignKeys());
GeoPackageValidate.validateMinimumTables(open);
assertNull(open.quickCheck());
assertNull(open.integrityCheck());
open.close();
}
}
@bosborn Thanks for your input.
Yes, my app already has the android.permission.WAKE_LOCK
permission in the app manifest because it uses the permission, and I discovered that the ImportGeoTask class within the ImportTask class also uses the PowerManager's WakeLock.
As an Intent result of the activity in the main thread, the main activity calls the ImportTask object to import the selected Geopackage file.
However, the SafGeoPackageMapTileModuleProvider object is called within a AsyncTask.OnPreExecute()
call to set the tile sources from the entire or individual raster Geopackage file returned by either SafGeoPackageMapTileModuleProvider.getTileSources()
or SafGeoPackageMapTileModuleProvider.getTileSource(database)
before rendering them with the MapView.setTileSource(tilesource)
call in the AsyncTask.doInBackground()
.
In the end, I have resolved the errors listed above by combining your solution for SafGeoPackageMapTileModuleProvider and my solution for a few issues listed below:
I found out that looping allGeos.getDatabases()
via iterator causes a ConcurrentModificationException from this code snippet that I call it inside the AsyncTask.OnPreExecute()
call:
geoPackageViewModel.getGeos().observe(getViewLifecycleOwner(), newGeos -> {
allGeos = newGeos;
for (GeoPackageDatabase : allGeos.getDatabases()) {}
});
As a result, I revert the code snippet to a canonical for-loop with indexing instead.
If any of the imported Geopackage files stored in data/user/0/{app package}/databases
is corrupted, the GeoPackageRepository.regenerateTableList()
call from the GeoPackageViewModel.init()
call can cause the app to crash with the following stacktrace:
In the GeoPackageRepository.regenerateTableList()
method, I made the following change to properly catch the exception when GeoPackageManager.validateHeader()
is called:
public List<List<GeoPackageTable>> regenerateTableList() {
geoPackages.clear();
geos.getValue().getDatabases().clear();
List<List<GeoPackageTable>> databaseTables = new ArrayList<List<GeoPackageTable>>();
StringBuilder errorMessage = new StringBuilder();
Iterator<String> databasesIterator = manager.databases().iterator();
// Read the feature and tile tables from the GeoPackage
while (databasesIterator.hasNext()) {
boolean invalidGP = false;
boolean validHeader = false;
String database = databasesIterator.next();
List<Exception> headerExceptions = new ArrayList<>();
try {
validHeader = manager.validateHeader(database);
} catch (Exception e) {
headerExceptions.add(e);
}
if (!headerExceptions.isEmpty())
{
if (errorMessage.length() > 0) {
errorMessage.append("\n\n\n");
}
errorMessage.append(database).append(" Errors:");
for (Exception exception : headerExceptions) {
errorMessage.append("\n\n");
errorMessage.append(exception.getMessage());
}
}
// Delete any databases with invalid headers
if (!validHeader) {
if (manager.delete(database)) {
databasesIterator.remove();
}
}
else {
//...truncate whatever in here since it is very long
}
}
// Make sure to still post the value of Geos if it's empty
if(geos.getValue().isEmpty()){
geos.setValue(geos.getValue());
}
return databaseTables;
}
Please fill out as much known and relevant information as possible.
Version Information:
api 'mil.nga.geopackage:geopackage-android:6.1.2'
Expected Results:
However, following import, it appears "corrupt." However, after restarting the app, the "failed" map renders correctly.
Import feature
![import from file 1](https://user-images.githubusercontent.com/1452656/162661139-c56c128b-2f77-480a-b9be-3fac8f2318b7.png)Importing with copy locally option selected
![import from file 2](https://user-images.githubusercontent.com/1452656/162661326-1914a433-6379-4d9e-a7ff-916567c43a82.png)Observed Results:
Failed to import popup message
![import from file 3](https://user-images.githubusercontent.com/1452656/162661426-d6906bee-f733-4ad2-abff-ae76db49aae4.png)But it loads fine after app restart
![import from file 4](https://user-images.githubusercontent.com/1452656/162663437-109cab04-f574-48a2-b261-b1b79231021c.png)Output:
Error 1
``` 2022-04-11 10:46:28.214 16539-16539/? I/OsmDroid: Geopackage database list: [5312] 2022-04-11 10:46:28.217 16539-16539/? I/OsmDroid: Geopackage database size: 1 2022-04-11 10:46:28.704 16539-16539/? E/com.my.app.android.component.OsmMapFragment: GPKG file setupMap() load error: mil.nga.geopackage.validate.GeoPackageValidate.validateMinimumTables(GeoPackageValidate.java:155) mil.nga.geopackage.GeoPackageException: Invalid GeoPackage. Does not contain required tables: gpkg_spatial_ref_sys & gpkg_contents, GeoPackage Name: 5312 at mil.nga.geopackage.validate.GeoPackageValidate.validateMinimumTables(GeoPackageValidate.java:155) at mil.nga.geopackage.GeoPackageCreator.createGeoPackage(GeoPackageCreator.java:228) at mil.nga.geopackage.GeoPackageManagerImpl.open(GeoPackageManagerImpl.java:1101) at mil.nga.geopackage.GeoPackageManagerImpl.open(GeoPackageManagerImpl.java:1070) at com.my.app.android.gpkg.tiles.raster.SafGeoPackageMapTileModuleProvider.Error 2
``` 2022-04-11 11:16:15.032 1920-1920/? I/OsmDroid: Geopackage database list: [5312] 2022-04-11 11:16:15.037 1920-1920/? I/OsmDroid: Geopackage database size: 1 2022-04-11 11:16:15.118 1920-1920/? E/com.my.app.android.component.OsmMapFragment: GPKG file setupMap() load error: android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method) android.database.sqlite.SQLiteDatabaseCorruptException: database disk image is malformed (code 11) ################################################################# Error Code : 11 (SQLITE_CORRUPT) Caused By : The database disk image is malformed. (database disk image is malformed (code 11)) ################################################################# at android.database.sqlite.SQLiteConnection.nativeExecuteForCursorWindow(Native Method) at android.database.sqlite.SQLiteConnection.executeForCursorWindow(SQLiteConnection.java:930) at android.database.sqlite.SQLiteSession.executeForCursorWindow(SQLiteSession.java:836) at android.database.sqlite.SQLiteQuery.fillWindow(SQLiteQuery.java:62) at android.database.sqlite.SQLiteCursor.fillWindow(SQLiteCursor.java:143) at android.database.sqlite.SQLiteCursor.getCount(SQLiteCursor.java:132) at android.database.AbstractCursor.moveToPosition(AbstractCursor.java:219) at android.database.AbstractCursor.moveToNext(AbstractCursor.java:268) at android.database.CursorWrapper.moveToNext(CursorWrapper.java:202) at mil.nga.geopackage.db.ResultUtils.buildResults(ResultUtils.java:395) at mil.nga.geopackage.db.GeoPackageConnection.queryResults(GeoPackageConnection.java:192) at mil.nga.geopackage.db.GeoPackageCoreConnection.queryResults(GeoPackageCoreConnection.java:998) at mil.nga.geopackage.db.CoreSQLUtils.foreignKeyCheck(CoreSQLUtils.java:363) at mil.nga.geopackage.db.GeoPackageCoreConnection.foreignKeyCheck(GeoPackageCoreConnection.java:194) at mil.nga.geopackage.db.GeoPackageCoreConnection.enableForeignKeys(GeoPackageCoreConnection.java:147) at mil.nga.geopackage.GeoPackageCreator.createGeoPackage(GeoPackageCreator.java:222) at mil.nga.geopackage.GeoPackageManagerImpl.open(GeoPackageManagerImpl.java:1101) at mil.nga.geopackage.GeoPackageManagerImpl.open(GeoPackageManagerImpl.java:1070) at com.my.app.android.gpkg.tiles.raster.SafGeoPackageMapTileModuleProvider.Steps to Reproduce:
It is not relevant since it is my custom codes
Relevant Code:
The code snippet where the exception occurs at
(SafGeoPackageMapTileModuleProvider.java:90)
:GeoPackage open = manager.open(databases.get(i));
I'm not sure if removing the extension ".gpkg" (GEO_PACKAGE_EXTENSION) from the file name before adding the file into the
GeoPackageManager.importGeoPackage()
call is a problem or not.Test Files:
Additional Information: