DE: request for a new harvesting

[alitka]

Dear Geoportal Helpdesk,

As mentioned in Roberts Mail from 2020/03/02 we would like to initiate a new push of our metadata records to the EU Geoportal. For this reason, we kindly ask you to start a new harvesting of our catalogue instance and publish them for us in the Geoportal harvesting "sandbox", please.

Thanks in advance and best regards,

Anja (on behalf of Coordination Office SDI Germany)

[alitka]

Dear @dartasensi, We noticed, that you started a new harvest process on Wednesday (2021-07-14) whose estimated end time was scheduled for Friday night. After an internal check, we have determined that the harvest process don't run any more. So we check the status on the harvest console and had to realize, that our process is still ongoing instead of being properly finished. Could you tell us, please, what happened? Thanks and kind regards.

[dartasensi]

Dear @alitka

apologies for the delay, I was quite busy with other activities. I will have a look asap and keep you posted.

Davide

[alitka]

Dear @dartasensi We would appreciate it if you could start a new harvesting process, as the results of the last harvesting process are already two weeks old and have never been made available to us in the Geoportal harvesting "sandbox". Thanks in advance and best regards.

[dartasensi]

Dear @alitka

I would really like to. Unfortunately, I can not.

As you may have noticed, I launched a harvest on Friday late aft, expecting to see the results this morning. Instead, (again), I found at least one thread locked up on some resources, so the entire harvest process is waiting for this task to finish. Since this process worked until now, I have the suspicion that there's something new inside your catalogue/services that prevent the Geoportal to move on, over the harvest process. Obviously I cannot say what or on which side it relies: on ours (eg. an uncaught/unexpected Geoportal exception), or yours (due to some misconfiguration). I'm still investigating.

Thanks for your patience, Davide

[alitka]

Dear @dartasensi After checking our systems, we found that our (nginx) web server was causing some problems on Saturday, which is why you have to restart the harvest anyway. Furthermore, we would prefer it if you could limit the harvesting to weekdays (until mid-September) since our system load is very high on the weekends due to internal harvesting processes. Thanks in advance and best regards.

[dartasensi]

Dear @alitka

(hoping to find you well) would be it now a good moment to give Geoportal another try? :) I'll do my best to monitor this issue, then schedule it, monitor it and make it work flawlessly.

Many thanks for your patience.

BR, Davide

[dartasensi]

I will try to run the harvest locally on my machine, to assess that at least the issue is not coming from a bug on Geoportal source code. ;)

[alitka]

Dear @dartasensi, Just a question: did you already start a harvest process locally on your machine, as we couldn't see any actions on our systems over the last week. I would be glad to get a quick reply. Best regards.

[dartasensi]

Dear @alitka

apologies for the late reply. Indeed you're right: I ran some test on your services (not the Discovery Service, since I had already your catalogue stored on my machine) few weeks ago but now I'm on vacation :) so I will surely test it as soon as I will come back to the office.

Cheers, Davide

[alitka]

Dear @dartasensi, It would be great, if you could start a harvest process locally on your machine next week. We have been waiting for already more than two months to get a “refresh” of our records. Thanks in advance and best regards.

[dartasensi]

Dear @alitka

Apologies for the delay due to the summer break. I've just restored my local installation of Geoportal and now it is up and running. In order to assess if the issue is still present, I scheduled and started a harvest on my machine, yesterday, around 16.09. At the moment, it is still running smooth.

KR, Davide

[jrc-inspire]

Duplicate of #54. Hence, this issue will be addressed as part of issue #54