search

INSPIRE-MIF / helpdesk-validator

Community discussion forum for INSPIRE validation issues
42 stars 23 forks source link

INSPIRE Registry unstable: Does it affect validation results in INSPIRE Validator? #454

Closed ghost closed 1 year ago

ghost commented 3 years ago

Hi everyone,

it seems that INSPIRE Registry is quite unstable and that connection to the registry breaks very often.

As INSPIRE validator is using Registry for many use cases (validation of INSPIRE Themes, Spatial Scope, Priority Data Sets) can we expect stable validation results for such use cases?

Where can I post an issue regarding INSPIRE Registry?

Kind regards, Dejan

carlospzurita commented 3 years ago

Dear @DeordD

You may try to open a new thread on the INSPIRE forum to highlight it to the registry maintainers.

ghost commented 3 years ago

@carlospzurita Could you also give some feedback on the question:

As INSPIRE validator is using Registry for many use cases (validation of INSPIRE Themes, Spatial Scope, Priority Data Sets) can we expect stable validation results for such use cases?

I will post it on the INSPIRE Forum after someone confirms that INSPIRE validator also has to cope with this problem...

ghost commented 3 years ago

@carlospzurita After some search I found similar topic like this one: https://github.com/inspire-eu-validation/community/issues/437#issuecomment-721686658

It seems that workaround for this problem is to cache the resources. It seems that one of the reasons for introducing schema-caching solution in INSPIRE validator was also unstable INSPIRE Registry connection.

Can you confirm this?

carlospzurita commented 3 years ago

Dear @DeordD

We are indeed caching the resources from the INSPIRE registry. However, as we have set up a midnight reboot on the production instance, the cache is emptied every night.

What instances of this errors did you find, and at what time are you encountering this error? The only time there should be caching problems is after the restart, but once the resource has been requested, it should available for the validator.

iuriemaxim commented 3 years ago

Dear @DeordD, @carlospzurita

I also documented the huge latency of the INSPIRE registry here https://github.com/inspire-eu-validation/community/issues/399 The INSPIRE registry responds in seconds (avg. 3.7 s) while other servers responds in miliseconds (https://github.com/inspire-eu-validation/community/files/5372023/20201003_Log_Geo1.xlsx).

The thread also shows that there is no caching of the INSPIRE registry, or if it is a chachimg mechansim in place, then it does not function.

ghost commented 3 years ago

Dear @carlospzurita , @iuriemaxim ,

like suggested I have communicated the issue with connection break and latency (reported problem from @iuriemaxim) in INSPIRE Community Forum: https://inspire.ec.europa.eu/forum/discussion/view/269974/inspire-registry-unstable-and-has-significant-latency

Kind regards

iuriemaxim commented 3 years ago

Dear @DeordD You wrote on a forum. From what I know, as is not the registry helpdesk, most probably no action will be taken. But who knows. Issues for the INSPIRE registry should be reported here: https://ies-svn.jrc.ec.europa.eu/projects/inspire-registry/issues

Finger cross that somone will treat the issue. However I think that the fix will not be an easy one, as probably the technology behind will need to be changed to respond in miliseconds instead of seconds as it is doing now. Not sure if the registry code is under maintanance.

Connection issues are most probably related to latency. If the registry receives 100 requests it does not respond to about 5% of them and if another user has some other 100 requests to the registry than the percentage of requests without response from the registry increase. We logged all our requests to the INSPIRE registry and all responses from the registry. Thats how we found that the registry is responding in seconds and also is not responding to a significant number of requests.

I also think that it may be very easy for some hackers to block the INSPIRE registry completly. As this is an important piece of INSPIRE infrastructure ...

ghost commented 3 years ago

Dear @iuriemaxim , @carlospzurita ,

I got an answer to my questions:

More information here: https://inspire.ec.europa.eu/forum/discussion/view/269974/inspire-registry-unstable-and-has-significant-latency

ghost commented 3 years ago

@carlospzurita Any news on this?

fabiovinci commented 1 year ago

Dear all,

the issue seems no longer present, so I will close the issue.

In addition, the latest release provides the file inspire-registry-resources.zip, which includes the resources that the INSPIRE Reference Validator requests to the INSPIRE Registry to execute the validations defined in the Executable Test Suites (ETSs). Using this content, you may configure your own deployment to access these resources without the need to make a call to the INSPIRE Registry to obtain them, increasing the performance of your own instance while decreasing the dependency on external resources in your installation.