Check of AIP database failed

[baloan]

Installed the software a few weeks ago when it worked without issues. Starting yesterday I got the following error message. Needless to say my network is fine, I can reach github... . Running AIP Browser DE-1.1.4-windows-x86_64. I've deleted the .AIPBrowserDE folder and tried a re-install but it still doesn't work. PS: Just found the log file, and yes, access to https://aip.dfs.de/BasicVFR/pages/C00001.html fails.

Feature Request: application should continue with local cached data. Add a "last update date" to the main window.

2024-07-10T09:51:40.364+0200 [main] INFO ModelBridgeImpl - java.vm.name: OpenJDK 64-Bit Server VM
2024-07-10T09:51:40.365+0200 [main] INFO ModelBridgeImpl - java.runtime.version: 21.0.1+12-LTS
2024-07-10T09:51:40.365+0200 [main] INFO ModelBridgeImpl - os.name: Windows 10
2024-07-10T09:51:40.365+0200 [main] INFO ModelBridgeImpl - os.arch: amd64
2024-07-10T09:51:40.365+0200 [main] INFO ModelBridgeImpl - java.vendor: Eclipse Adoptium
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Max Heap Size = maxMemory() = 8422162432
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Current Heap Size = totalMemory() = 528482304
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Available in Current Heap = freeMemory() = 512254408
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Currently Used Heap = 16227896
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Unassigned Heap = 7893680128
2024-07-10T09:51:40.369+0200 [main] INFO ModelBridgeImpl - Currently Totally Available Heap Space = 8405934536
2024-07-10T09:51:40.445+0200 [main] INFO ModelBridgeImpl - Config(release=Release(vendor=mpMediaSoft GmbH, appname=AIP Browser DE, version=1.1.4))
2024-07-10T09:51:44.205+0200 [DefaultDispatcher-worker-6] ERROR de.mpmediasoft.aipbrowserde.model.aip.internal.HTMLScraperKt - Could not download resource 'https://aip.dfs.de/BasicVFR/pages/C00001.html'.
2024-07-10T09:51:44.210+0200 [DefaultDispatcher-worker-6] WARN de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImplKt - AIP update: !!! Checking of AIP database failed !!!<br>The effective date of the current AIP database could not be checked.<br>Maybe you don't have a network connection.
2024-07-10T09:54:25.701+0200 [DefaultDispatcher-worker-6] ERROR de.mpmediasoft.aipbrowserde.model.aip.internal.HTMLScraperKt - Could not download resource 'https://aip.dfs.de/BasicVFR/pages/C00001.html'.
2024-07-10T09:54:25.702+0200 [DefaultDispatcher-worker-6] ERROR de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImplKt - Folder selection: Could not select folder 'Aeronautical Information Publication VFR'.
2024-07-10T09:54:25.702+0200 [DefaultDispatcher-worker-6] ERROR de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImplKt - Exception:
java.lang.Exception: Could not get document for 'https://aip.dfs.de/BasicVFR/pages/C00001.html'.
    at de.mpmediasoft.aipbrowserde.model.aip.internal.KsoupHTMLScraper.getDoc(KsoupHTMLScraper.kt:56)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.KsoupHTMLScraper.<init>(KsoupHTMLScraper.kt:34)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.KsoupHTMLScraperFactory.instance(KsoupHTMLScraper.kt:19)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.HTMLScraperFactory$DefaultImpls.instance$default(HTMLScraper.kt:20)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPImporter.resolveFolderLinksAndThumbnails(AIPImporter.kt:222)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl$resolveFolder$1.invoke(AIPModelImpl.kt:813)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl$resolveFolder$1.invoke(AIPModelImpl.kt:813)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager$resolveFolder$1.invoke(AIPDBManager.kt:370)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager$resolveFolder$1.invoke(AIPDBManager.kt:368)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager$runInsideTransaction$1.invoke(AIPDBManager.kt:416)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager$runInsideTransaction$1.invoke(AIPDBManager.kt:416)
    at app.cash.sqldelight.TransacterImpl.transactionWithWrapper(Transacter.kt:370)
    at app.cash.sqldelight.TransacterImpl.transaction(Transacter.kt:349)
    at app.cash.sqldelight.Transacter$DefaultImpls.transaction$default(Transacter.kt:94)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager.runInsideTransaction(AIPDBManager.kt:416)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPDBManager.resolveFolder(AIPDBManager.kt:368)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl.resolveFolder(AIPModelImpl.kt:813)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl.doSelectFolder(AIPModelImpl.kt:786)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl.selectFolder(AIPModelImpl.kt:774)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl.access$selectFolder(AIPModelImpl.kt:32)
    at de.mpmediasoft.aipbrowserde.model.aip.internal.AIPModelImpl$select$1.invokeSuspend(AIPModelImpl.kt:748)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:108)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:589)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.executeTask(CoroutineScheduler.kt:806)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.runWorker(CoroutineScheduler.kt:710)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:697)

[mipastgt]

The reason for this is that DFS has started to prevent massive downloads of its data. It usually helps if you download as little data as possible in future and, secondly, if you briefly disconnect your DSL connection and then reconnect. This will usually give you a new IP address and you should be fine again. Deleting the .AIPBrowserDE folder is not reccomended because that also deletes all cached data.

[baloan]

Getting new IP addresses worked. I have tried creating trip kits using a json file which took several attempts. That may have triggered the traffic limits.

[mipastgt]

Therefore, it is important to keep the cache. Once a page has been downloaded it will only be downloaded again if it was changed and normally there are not that many files changed per month.

[baloan]

I've only deleted the database in response to the error message "Checking AIP database failed". I was not sure whether this was a hint to the local cached database inconsistency. So I attempted to reset the (local) database. Maybe the error message should be changed to: "Remote DFS AIP lookup fails with \<HTML response>. Continuing with local cached data. This may happen when the DFS AIP service applied extensive traffic protection. You may try to acquire a new IP address by reconnecting your home router (if you have one)." or the currently displayed message if the HTML request fails. Actually I've renamed the folder, so I now have restored the original cached local database ;-)

[mipastgt]

The reason for these blockages might be the same as described here. https://www.heise.de/news/Fehler-403-Anti-Bot-Massnahme-von-Youtube-hebelt-Drittanbieter-Apps-aus-9798059.html

[df1paw]

How about incorporating some kind of P2P technology? If someone else has already downloaded it from dfs-ais.de, you don't necessarily have to query the main server again. In an ideal world, at some point there would no longer be a load on their web server...

[mipastgt]

A preview of the upcoming version 1.2.0 of the AIP Browser DE, which should fix this issue, can be obtained via the web page. https://mpmediasoft.de/products/AIPBrowserDE/help/AIPBrowserDE-Preview.html I'd appreciate it if some people would try out this version and give me some feedback on it. If no severe issues are found, this version will become the new release version.

[zoernert]

• +1 For the standard Tripkit selection
• +1 Downloads seem to be more relaxed from DFS servers

However, I've tested with a company VPN - so my results might not be the same for all. Windows/Linux tested.

If you like any more specif tests - you might open an Issue for Bughunting and Feedback :)

[mipastgt]

Thanks a lot. I just uploaded a new version (build 14) to the server because there was an issue with the compatibility of the generated trip-kits with Enroute Flight Navigation.

[zoernert]

compatibility of the generated trip-kits with Enroute Flight Navigation.

As I am using Enroute Flight Navigation - I will check and report. However, my latest download is still running as I started with a brand new installation, so it will take some more hours...

[mipastgt]

With a brand new installation a full trip-kit of the whole of Germany should not take more than ca. 25 Minutes. That's what I meassured and others confirmed as well.

[zoernert]

Just in case someone beside of myself is moving from Linux to Windows and back... it would be nice to display the export folder location with the "done" message.

The export and import to "Enroute Flight Navigation" worked. However I have to do some more tests if geo locations etc,,, are right (what I assume).

It would be nice to use a Nextcloud Client to use a single folder on the PC and in Enroute Flight Navigation - would save some copy and past work.

[mipastgt]

Just in case someone beside of myself is moving from Linux to Windows and back... it would be nice to display the export folder location with the "done" message.

Just implemented.

The export and import to "Enroute Flight Navigation" worked. However I have to do some more tests if geo locations etc,,, are right (what I assume).

They should be :-)

It would be nice to use a Nextcloud Client to use a single folder on the PC and in Enroute Flight Navigation - would save some copy and past work.

This should already work if you create the trip-kit via a configuration file. In that case you can specify an alternate output folder. Search for "exportDir" in the documentation. This "exportDir" could be your local NextCloud folder but I don't know if/how that works on the other side on your mobile device and Enroute. But some other user has reported to have done something similar via DropBox.

[mipastgt]

This should all be fixed now in the release version 1.2.0. See: https://mpmediasoft.de/products/AIPBrowserDE/help/AIPBrowserDE.html