Make it possible to set per-layer connection timeout

[M4rtinK]

Add the connection_timeout key for layer in the map config file, which can be used to set the connection timeout for tile downloads.

This is mostly needed for offline rendering localhost tileservers, such as OSM Scout Server, which might in some cases take some time to render a tile but will still eventually return it.

Like this we can set a longer connection timeout for layers provided by localhost renderers while keeping a short connection timeout for online layers to allow for quick recovery from connectivity issues.

[rinigus]

Excellent, I will test it tonight.

[rinigus]

How do I make RPM from this? make rsync-sailfish leads to

mv: cannot stat 'modrana_source/modules/gui_modules/gui_qt5/qml': No such file or directory
mv: cannot stat 'modrana_source/qml/universal_components/silica/UC': No such file or directory

[rinigus]

Hi,

I think I stumbled on a case showing why timeouts are bad and should be considered harmful when used with OSM Scout Server. :)

I can trigger "crazy" mode with loads of jobs submitted to the server on OPX. For that I used currently released modRana and the latest server version. In principle, it should be possible to reproduce on any phone with any timeout. Namely, when we browse a map and switch off the screen while its still has 10 jobs in a queue. The phone enters sleep mode where it wakes up once in a while (300 sec sleep are not uncommon). During short time while phone is awake between sleep cycles (insufficient to render a tile by the server), modRana figures out that a timeout has passed and submits a new job. And the pattern repeats. I have seen 67 jobs in a queue when I checked the phone a bit later. Since the device could be left for rather long time without interaction, this scenario should be possible with any CPU and any timeout.

From this example, I would suggest to skip timeouts for localhost server. We are not expected to have any network issues anyway, so let's better allow modRana/OSM Scout Server misfunction if they are used incorrectly (server closed) than to penalize with the correct usage (as in my example). For example, we could set timeout for this layer to -1 and not check for it if its negative.

[M4rtinK]

How do I make RPM from this? make rsync-sailfish leads to

mv: cannot stat 'modrana_source/modules/gui_modules/gui_qt5/qml': No such file or directory mv: cannot stat 'modrana_source/qml/universal_components/silica/UC': No such file or directo

To make RPM targeting desktop Linux distros, like Fedora:

make archive
cp modrana-*.tar.gz ~/rpmbuild/SOURCES/
rpmbuild -ba packaging/modrana.spec

Sailfish OS specific packages are created from the same spec file in a slightly different way using RPM macros so that I don't have to maintain multiple spec files. You can see them at the top of the modrana.spec file:

%{!?with_sailfish: %define with_sailfish 0}
%{!?with_harbour: %define with_harbour 0}

The RPM macros basically enable all the mangling that differentiate Sailfish OS RPM packages from properly packaged application for say Fedora. ;-) And because of Harbour there are actually two levels of them - just with_sailfish creates packages that go to OpenRepos and if also with_harbour is enabled, a more strict mode is triggered, that creates packages with limited functionality that are Harbour compatible.

As creating the actual packages for Sailfish OS - I actually don't do that manually, but use the Sailfish OS target on the Mer OBS to do it for me. These are the two prejects I use, first for Open Repos packages, the second one for Harbour packages:

https://build.merproject.org/project/show/home:MartinK:sailfish:modrana https://build.merproject.org/project/show/home:MartinK:sailfish:modrana-harbour

The macros that make the projects create Sailfish OS & Sailfish OS Harbour specififc packages are set in the "Project Config" tab. This is how it for example looks like for the Harbour compatible packages:

Release: <CI_CNT>.<B_CNT>

Macros:
%with_sailfish 1
%with_harbour 1
%vendor %%foo

Now thinking about it, the packages are noarch & AFAIK have been working just fine on the x86 based tablet, so the armv8el target might not really be needed. Not sure if just running rpmbuild on your distro of choice would work though - IIRC some packages are named differently on Sailfish OS so the spec variant might not be buildable outside of the SDK, even if you somehow pass the macros to rpmbuild (that should be possible - somehow :) ).

So that should be it for "Introduction to the fun and joy of cross-distribution packaging" - and don't feart to ask if there is still something unclear about how the whole house of cards works. ;-)

[M4rtinK]

Hi,

I think I stumbled on a case showing why timeouts are bad and should be considered harmful when used with OSM Scout Server. :)

I can trigger "crazy" mode with loads of jobs submitted to the server on OPX. For that I used currently released modRana and the latest server version. In principle, it should be possible to reproduce on any phone with any timeout. Namely, when we browse a map and switch off the screen while its still has 10 jobs in a queue. The phone enters sleep mode where it wakes up once in a while (300 sec sleep are not uncommon). During short time while phone is awake between sleep cycles (insufficient to render a tile by the server), modRana figures out that a timeout has passed and submits a new job. And the pattern repeats. I have seen 67 jobs in a queue when I checked the phone a bit later. Since the device could be left for rather long time without interaction, this scenario should be possible with any CPU and any timeout.

Interesting, pretty crazy - and I agree, pretty bad.

From this example, I would suggest to skip timeouts for localhost server. We are not expected to have any network issues anyway, so let's better allow modRana/OSM Scout Server misfunction if they are used incorrectly (server closed) than to penalize with the correct usage (as in my example). For example, we could set timeout for this layer to -1 and not check for it if its negative.

Yeah, that really does sound like the way to go - I've changed the PR so that -1 can be used in the configuration file to disable the timeout & set -1 for the OSM Scout Server provided layers. This sets the timeout parameter of the connection pool to None, which should "disable timeout" according to the docs. Hopefully it really does that vs just setting a really long timeout. :) So testing definitely welcome & especially in your IMHO not so crazy scenario.

BTW, there is actually supposed to be a mechanism on the GUI side of modRana tile handling code that should be keeping track of failed download attempts of each visible tile & stop further attempt after three failures. Looks like that's apparently not working - which is probably not so surprising given that I've rewritten that part a while ago get more optimized & flicker free map display. Another thing to check I guess. :)

[rinigus]

I have been testing it during the day and the latest version clearly fixes the issue, thank you! From brief testing of one online service I haven't seen any problems either.

I will write the warning for other developers to avoid timeouts when using the server on mobile.

[M4rtinK]

Nice! My brief testing also indicates things work correctly, so I'll merge this now. :)

BTW, I plan to do a new modRana bugfix release in a few days so the users should be able to benefit from this fix as well in a relatively short order. :)

[M4rtinK]

modRana 0.54.7 with the fix has been released :)

[rinigus]

Great, thank you!