Affiliate API: Supplement/import missing data & failed search results

mekarpeles commented 3 years ago

Continuation of #1322

[ ] When Affiliate Server pulls data, if that data is not in OL (e.g. a missing cover), let's make sure to back-fill and improve over time
[ ] Let's see if we can extend our usage of the affiliate APIs so if our search result returns no hits, we may still show / perhaps import results.

Describe the problem that you'd like solved

Proposal & Constraints

Additional context

Stakeholders

BrittanyBunk commented 3 years ago

I do like the idea of bringing to attention what's missing in the OL so that it can be filled out more. That is probably the # 1 issue on the OL and so I understand the need to fix it.

So what you're saying is for some (I forgot the term) automated moderator will look for missing info on the OL and when an item's pulled and try to fill in what's missing. Cool (if I'm understanding right) - I mean that is nice if that's able to be created.

Like if it pulls from the internet, that'd be neat. What I'm thinking is maybe have a mode to where a bot pulls info from the internet and anyone who clicks the edit button would be given options to add in what the bot pulled by clicking on them.

Examples:

author: Dr. Seuss and it's surrounded by a red dashed square. Then people can click it to add it in if it's correct
if the bot doesn't know the correct one, then there's multiple choices and it's up to a person to add the correct one or their own.

Cool - I like the streamlining idea to lessen the workload for people. Now how do we make that happen lol is the question.

mekarpeles commented 2 years ago

As mentioned in https://github.com/internetarchive/openlibrary/pull/6953

Affiliate Service (weekend outage fix) + Check imports succeeding (esp. amazon)

[ ] Set a max buffer size to our isbn queue of 1,000 Let’s use this PR to verify.
[ ] Set a Sentry alert if queue size is 1,000 Enabler is this PR + Graphana alert (which can already raise a Slack).
[x] Create an endpoint to clear the queue? Done in this PR
[ ] Change cache for 24 hours. Let’s use this PR to verify.

internetarchive / openlibrary