search

claeis / ili2db

interlis import/export to relational databases
30 stars 30 forks source link

not responding model repositories #525

Closed michmuel closed 11 months ago

michmuel commented 1 year ago

We encountered following issue several times over the last months:

ili2db:

lookup model <DM01AVBL24LV95D_20120301> in repository <http://models.interlis.ch/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <http://models.interlis.ch/>
lookup model <DM01AVBL24LV95D_20120301> in repository <http://models.geo.admin.ch/>
lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.kgk-cgc.ch/>
lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.interlis.ch/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <http://models.geo.admin.ch/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.kgk-cgc.ch/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.interlis.ch/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.geo.llv.li/>
lookup model <DM01AVBL24LV95D_20120301> in repository <https://models.geo.llv.li/>
lookup model <DM01AVBL24LV95D_20120301> in repository <https://405.sia.ch/models/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <https://405.sia.ch/models/>lookup model <DM01AVBL24LV95D_20120301> in repository <https://repositorio.proadmintierra.info/>
Info: lookup model <DM01AVBL24LV95D_20120301> in repository <https://repositorio.proadmintierra.info/>

Gitlab-Pipeline:

ERROR: Job failed: execution took longer than 6h0m0s seconds

It seems that the mechanism of finding a model in a repository may fail if one of the directories is not responding.

Is an option already implemented in the ili-tools which would allow to skip not responding directories? If not, could the process be improved?

edigonzales commented 1 year ago

@michmuel If you use most of the time models from the same repos, you can set modeldir explicitly, e.g. --modeldir "https://geo.so.ch/models/;https://models.geo.admin.ch". Probably the second one is not even necessary since it searches also upstream (?).

Another option is to set timeout values:

System.setProperty("sun.net.client.defaultConnectTimeout", connectTimeout);
System.setProperty("sun.net.client.defaultReadTimeout", readTimeout);

You can set the also as cli param: https://www.ibm.com/docs/en/ztpf/2019?topic=options-dsunnetclientdefaultconnecttimeout-option

Be aware that this will affect every connection made in the JVM made with URLConnection (and other classes I guess).

In most of our applications we use either one or the other option.

michmuel commented 1 year ago

Thanks for your hints @edigonzales. However, I think that a little bit more robust default behavior could improve the tool.

edigonzales commented 1 year ago

@michmuel see https://github.com/claeis/ili2c/commit/54a9bd84a34c2bbdbf7e638b16ff0ad1e27e51e1

Will try to include it in the gradle plugin as soon as there is a released version.

michmuel commented 1 year ago

Fast response! Many thanks.

edigonzales commented 11 months ago

@michmuel Kannst du mal die aktuelle Version probieren https://plugins.gradle.org/plugin/ch.so.agi.interlis-repository-creator Version 1.4.29 und grösser