DE: request for a new harvesting

[alitka]

Dear Geoportal Helpdesk,

As mentioned in Roberts Mail from 2020/03/02 we would like to initiate a new push of our metadata records to the EU Geoportal. For this reason, we kindly ask you to start a new harvesting of our catalogue instance and publish them for us in the Geoportal harvesting "sandbox", please.

We would appreciate it if you could start the new harvesting process before February 18, 2022 or after February 21, 2022. During this time slot, we have internally planned extensive maintenance work.

Thanks in advance and best regards, Anja (on behalf of Coordination Office SDI Germany)

[jescriu]

Dear @alitka, We are currently processing your request. Due to the size of your catalogue, there is no space enough in our servers to maintain the files associated to the new and the previous harvest. Our IT Geoportal team is working in automatizing the cleaning of files from the previous harvest when a new one is scheduled. We will let you know when we start the new harvest.

[jescriu]

Dear @alitka, Finally our IT Geoportal Team made an evaluation of the disk size available on the server, which seems to be sufficient if other countries with large catalogues run a harvest of their endpoints concurrently. Since the harvest of your catalogue is currently taking approx. 5 days, following your initial request and maintenance constraints we will be scheduling the new harvest on 22nd Feb.

[alitka]

Thanks, @jescriu, for the information + update.

[jescriu]

Dear @alitka, The new harvest of your endpoint started on 22nd, as reported in my last post. Unfortunately it got stuck this morning, and this time the size on the server seems not to be the issue. We will be restarting the harvest today, so please keep your infrastructure as stable as possible during the approximately 5 days of processing that it might take - Note that it might be also running during the weekend, which is when you regularly update your production instance. With the aim to avoid possible collisions with the running harvest, could you avoid the update scheduled for this weekend? If after this attempt the problem persists, we will analyse again the situation.

[alitka]

Dear @jescriu , Thanks again for the detailed information you have given us. If you tell us the precise time of the harvesting start, we can monitor the process on our side. We would appreciate, if you could start the harvesting process after 7 am tomorrow (2022-02-24), due to internal maintenance work on our side between 5 am and 7 am. Thanks in advance and best regards.

[jescriu]

Dear @alitka, Following you indications we will re-start the harvest tomorrow after 7:00 CET, and will let you know when started. Please, let us know if you could avoid the update of your catalogue this weekend (to prevent potential issues with this harvest).

[alitka]

Dear @jescriu, Just a question: did you already start a harvest process locally on your machine, as we couldn't see any actions on our systems over the day. I would be glad to get a quick reply. Also, we'll skip the internal updates of our catalogue this weekend (to prevent potential issues with this harvest). Best regards.

[jescriu]

Dear @alitka, Excuse me, I had pending to inform you. We restarted the harvest around 15:15 CET today, since we need to perform other checks before. Hope it is running OK this time.

[alitka]

Dear @jescriu, Just a note: the harvest was running since 3 pm yesterday. But currently, we couldn't see any actions on our systems. Is the harvesting still running? Best regards.

[jescriu]

Dear @alitka, Thank you for your feedback. I will check it with our IT Geoportal Team. and will let you know. Best, Jordi

[jescriu]

Dear @alitka, As per our IT Geoportal Team, the harvest process is running and alive. Nor sure why you might not be detecting it. Let's see the status of the process on Monday. Best, Jordi

[alitka]

Dear @jescriu, As already reported on Friday, we can't detect any activities on our system. However, is the process still running on your system? Best regards.

[jescriu]

Dear @alitka, The harvest was finished. You can access the results via harvest console. Let's contact by email to comment on them. Jordi