search

topcoder-platform / tc-project-service

16 stars 55 forks source link

Index projects even if cannot get member details #528

Open maxceem opened 4 years ago

maxceem commented 4 years ago

if we are not able to fetch member details for the active members, it does not index that project:

We should still index the project even if we cannot get member details for project members.

vikasrohit commented 4 years ago

@maxceem I think this one still holds true but we need to update the links in the issue to point to correct line number because code seems to be refactored. And if would be great if we can get it done in May release.

maxceem commented 4 years ago

The code has been updated and moved to another file.

If we didn't get user details from the Member Service, we should log error, but the project should be still added to the ES https://github.com/topcoder-platform/tc-project-service/blob/develop/src/utils/es.js#L168-L172.

vikasrohit commented 4 years ago

This is about admin endpoint, right @maxceem? For admin endpoint, it might make sense to error out if we are not able to find member details, because we use this endpoint when we want to fix some issue with the data e.g. one of the project member's first name, last name or email is not correct or infact was not indexed and it is causing the project search to not return that project when search for that user. So, if this endpoint just indexes it again without member details, we might think it should be fixed after calling this endpoint. However, on the other side, I don't see any good solution for handling this situation. I think may be a separate admin endpoint to fix members of the project, might help us in this direction because that endpoint can fail safely when it does not get required data from member api.

maxceem commented 4 years ago

Yes, it's for admin endpoint. The main idea is to avoid situations when some project is not indexed at all because of user details when we index 1000 projects in batches.

To solve the issue you've raised in the comment above, I suggest to improve the output of admin endpoint response. It should return the log in the response see details https://github.com/topcoder-platform/tc-project-service/issues/562.

So in case we are just reindexing a single project to fix member, we would see if there were any errors or no in the response.

vikasrohit commented 4 years ago

Streaming is something I am not 100% sure as of now. But I think we can go with the fixing the original problem mentioned in the issue.