CredentialEngine / CredentialRegistry

Repository for development of the Credential Registry
Apache License 2.0
12 stars 10 forks source link

All organization /resources URLs are returning null #713

Open mparsons-ce opened 4 months ago

mparsons-ce commented 4 months ago

The following URL returns null: https://credentialengineregistry.org/resources/ce-0abf92da-5536-4670-bb47-636a3f330d7c

Yet, these return data: https://credentialengineregistry.org/graph/ce-0abf92da-5536-4670-bb47-636a3f330d7c https://credentialengineregistry.org/envelopes/ce-0abf92da-5536-4670-bb47-636a3f330d7c

Quick testing suggests that this is only an issue for organizations

excelsior commented 4 months ago

@mparsons-ce Some of the organization resources were not extracted, although the envelopes themselves were published successfully. However, instead of 404s, nulls were returned.

I re-processed the envelopes, now the /resources endpoint returns correct data.

Why some resources were not extracted is an open question.

jeannekitchens commented 2 months ago

@excelsior @mparsons-ce can this be closed?

jeannekitchens commented 2 months ago

@excelsior @edgarf what is the follow up on this?

excelsior commented 2 months ago

@jeannekitchens The data issue has been fixed. I also did some improvements towards the publishing flow to reduce the chances of it happening again.

I'm working on adding some data consistency checks, as well as fixing the API to respond with a correct status instead of a null response. The latter is a bug, too, so let's keep this issue open until it resolved.

mparsons-ce commented 4 days ago

@excelsior I ran into this problem again today. The graph and envelope are resolvable https://credentialengineregistry.org/fdoe/graph/ce-71a0ee53-b0b3-4e6f-a772-717785073263 https://credentialengineregistry.org/fdoe/envelopes/ce-71a0ee53-b0b3-4e6f-a772-717785073263

but the resource is not https://credentialengineregistry.org/fdoe/resources/ce-71a0ee53-b0b3-4e6f-a772-717785073263

Others: https://credentialengineregistry.org/fdoe/resources/ce-ec1fb9f7-e2f3-449a-83d9-57ec56929912 https://credentialengineregistry.org/fdoe/resources/ce-2a64f12d-9571-44e7-80ad-9542d366d986 https://credentialengineregistry.org/fdoe/resources/ce-87932301-9747-40e3-b442-6da83827a3ab

excelsior commented 3 days ago

Hey @mparsons-ce! That's weird as they all are resolvable for me. That could have been explained by a possible delay between the events of publishing and background processing due to a busy queue but in this case both events occurred almost simultaneously.

Can you confirm that it looks good for you know?

mparsons-ce commented 3 days ago

@excelsior Yes, the /resources/ links now resolve. The scenario was:

So why would a /graph/ link resolve, but not the /resources/ link?

I suppose regardless of the latter answer, if a busy queue could always be a problem, then perhaps I should do existence checks using /graph/ links?