OBPS AWS EDS does not display the stats BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS .

paulineobps commented 1 year ago

I have reported this problem several times in GitHub and emails. The urgency is overwhelming particularly for number 2 below and cannot be left to continue occurring

Coinciding together:

The OBPS website https://www.oceanbestpractices.org/ no longer displays the number of the records in the repository
The OBPS AWS EDS https://search.oceanbestpractices.org/ does not display the stats  (Sorry, we can't display our stats at this time.) BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS .

This situation occurs randomly but lasts for hours and this time (yesterday to now) after some 20 hours has still not returned to normal searching.and stats display.

I am copying Paul (E84) in case it could have something to do with the updated bulk indexing schedule or he has other thoughts

May I please ask you to treat this as Urgent and a priority to fix.

Thank you

paulineobps commented 1 year ago

see https://github.com/iodepo/dspace-obps/issues/39 for the number of times this is happening

paulpilone commented 1 year ago

@paulineobps for

The OBPS website https://www.oceanbestpractices.org/ no longer displays the number of the records in the repository

is that data coming from the search.oceanbestpractices.org API? I can't see the issue you linked to - I get a 404 not found - maybe I don't have permission to see it.

I can look into this issue but my initial guess (without any debugging) is that the small instance of OpenSearch we're using is getting clogged up during nightly ingest. Not surprising - we've seen this issue occasionally when you run a bulk index. We've tried to throttle ingest to help prevent this but it's possible that even with that throttling we just need a bigger instance of OpenSearch. 2 options I'll look into (assuming this is in fact the issue):

Bumping up our OpenSearch instance. I'll provide options and $ amounts.
Adding an instance to the cluster. I'll provide options and $ amounts.
AWS very recently released a Serverless OpenSearch option - meaning you would only pay for when OpenSearch is in use. It's a very new feature and I'm not yet familiar with how it scales (in our case at night when we bulk ingest documents) but could be a good option. I'll research that because it's a brand new offering from AWS.

paulineobps commented 1 year ago

The number displayed on the obps website I thought came from DSpace, but at the same time as that disappears, the stats disappear from the AWS EDS and no search can be performed on AWS EDS. [cid:88754752-31db-4104-9f9f-587e3320a5b1]

Pauline Pauline SIMPSON Ocean Best Practices System Project Manager https://www.oceanbestpractices.org/

UNESCO /IOC Project Office for IODE, InnovOcean Campus – Jacobsenstraat 1 8400 Oostende, Belgium Email: @.**@.>; Alternate email: @.**@.>; Skype: Pauline Simpson

ORCID: https://orcid.org/0000-0003-2551-5740 https://orcid.org/0000-0002-3237-4547

http://www.iode.org

From: Paul Pilone @.> Sent: 16 October 2023 20:07 To: iodepo/OceanBestPractices @.> Cc: Simpson, Pauline @.>; Mention @.> Subject: Re: [iodepo/OceanBestPractices] OBPS AWS EDS does not display the stats BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS . (Issue #284)

CAUTION: This email is external from UNESCO. Please be vigilant on its sender and content. ATTENTION : Cet e-mail est externe à l'UNESCO. Soyez vigilant sur son expéditeur et contenu.

@paulineobpshttps://github.com/paulineobps for

The OBPS website https://www.oceanbestpractices.org/ no longer displays the number of the records in the repository

is that data coming from the search.oceanbestpractices.org API? I can't see the issue you linked to - I get a 404 not found - maybe I don't have permission to see it.

I can look into this issue but my initial guess (without any debugging) is that the small instance of OpenSearch we're using is getting clogged up during nightly ingest. Not surprising - we've seen this issue occasionally when you run a bulk index. We've tried to throttle ingest to help prevent this but it's possible that even with that throttling we just need a bigger instance of OpenSearch. 2 options I'll look into (assuming this is in fact the issue):

Bumping up our OpenSearch instance. I'll provide options and $ amounts.
Adding an instance to the cluster. I'll provide options and $ amounts.
AWS very recently released a Serverless OpenSearch option - meaning you would only pay for when OpenSearch is in use. It's a very new feature and I'm not yet familiar with how it scales (in our case at night when we bulk ingest documents) but could be a good option. I'll research that because it's a brand new offering from AWS.

— Reply to this email directly, view it on GitHubhttps://github.com/iodepo/OceanBestPractices/issues/284#issuecomment-1765495803, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZQUQZ7KGD2MD65ZCZOC33X7XK5TAVCNFSM6AAAAAA3IJQNI6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGQ4TKOBQGM. You are receiving this because you were mentioned.Message ID: @.***>

arnounesco commented 1 year ago

the stats are not coming from DSpace but from https://mremzmye87.execute-api.us-east-1.amazonaws.com/prod/statistics/

paulineobps commented 1 year ago

Thanks Paul - noted the actions you are taking

Pauline Pauline SIMPSON Ocean Best Practices System Project Manager https://www.oceanbestpractices.org/

UNESCO /IOC Project Office for IODE, InnovOcean Campus – Jacobsenstraat 1 8400 Oostende, Belgium Email: @.**@.>; Alternate email: @.**@.>; Skype: Pauline Simpson

ORCID: https://orcid.org/0000-0003-2551-5740 https://orcid.org/0000-0002-3237-4547

http://www.iode.org

From: Paul Pilone @.> Sent: 16 October 2023 20:07 To: iodepo/OceanBestPractices @.> Cc: Simpson, Pauline @.>; Mention @.> Subject: Re: [iodepo/OceanBestPractices] OBPS AWS EDS does not display the stats BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS . (Issue #284)

CAUTION: This email is external from UNESCO. Please be vigilant on its sender and content. ATTENTION : Cet e-mail est externe à l'UNESCO. Soyez vigilant sur son expéditeur et contenu.

@paulineobpshttps://github.com/paulineobps for

The OBPS website https://www.oceanbestpractices.org/ no longer displays the number of the records in the repository

is that data coming from the search.oceanbestpractices.org API? I can't see the issue you linked to - I get a 404 not found - maybe I don't have permission to see it.

I can look into this issue but my initial guess (without any debugging) is that the small instance of OpenSearch we're using is getting clogged up during nightly ingest. Not surprising - we've seen this issue occasionally when you run a bulk index. We've tried to throttle ingest to help prevent this but it's possible that even with that throttling we just need a bigger instance of OpenSearch. 2 options I'll look into (assuming this is in fact the issue):

Bumping up our OpenSearch instance. I'll provide options and $ amounts.
Adding an instance to the cluster. I'll provide options and $ amounts.
AWS very recently released a Serverless OpenSearch option - meaning you would only pay for when OpenSearch is in use. It's a very new feature and I'm not yet familiar with how it scales (in our case at night when we bulk ingest documents) but could be a good option. I'll research that because it's a brand new offering from AWS.

— Reply to this email directly, view it on GitHubhttps://github.com/iodepo/OceanBestPractices/issues/284#issuecomment-1765495803, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZQUQZ7KGD2MD65ZCZOC33X7XK5TAVCNFSM6AAAAAA3IJQNI6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRVGQ4TKOBQGM. You are receiving this because you were mentioned.Message ID: @.***>

paulpilone commented 11 months ago

@paulineobps I believe this issue should be resolved as a result of increasing the size of OpenSearch. If so we can close this GitHub Issue.

paulineobps commented 11 months ago

THanks Paul - yes increasing OpenSearch did the trick

best wishes and happy holidays

Pauline Pauline SIMPSON IOC Ocean Best Practices System Project Manager https://www.oceanbestpractices.org/

UNESCO /IOC Project Office for IODE, InnovOcean Campus – Jacobsenstraat 1 8400 Oostende, Belgium Email: @.**@.>; Alternate email: @.**@.>; Skype: Pauline Simpson

ORCID: https://orcid.org/https://orcid.org/0000-0002-3237-45470000-0003-2551-5740https://orcid.org/0000-0002-3237-4547

From: Paul Pilone @.> Sent: 22 December 2023 12:47 To: iodepo/OceanBestPractices @.> Cc: Simpson, Pauline @.>; Mention @.> Subject: Re: [iodepo/OceanBestPractices] OBPS AWS EDS does not display the stats BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS . (Issue #284)

CAUTION: This email is external from UNESCO. Please be vigilant on its sender and content. ATTENTION : Cet e-mail est externe à l'UNESCO. Soyez vigilant sur son expéditeur et contenu.

@paulineobpshttps://github.com/paulineobps I believe this issue should be resolved as a result of increasing the size of OpenSearch. If so we can close this GitHub Issue.

— Reply to this email directly, view it on GitHubhttps://github.com/iodepo/OceanBestPractices/issues/284#issuecomment-1867935816, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFZQUQYGSMLMTD74QQK4YVLYKXBUFAVCNFSM6AAAAAA3IJQNI6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRXHEZTKOBRGY. You are receiving this because you were mentioned.Message ID: @.***>

iodepo / OceanBestPractices

OBPS AWS EDS does not display the stats BUT MORE IMPORTANTLY DOES NOT RETURN ANY SEARCH RESULTS . #284