Update fetcher to handle non-book API attributes

[SteelWagstaff]

Pressbooks staff would like to mark specific books on our hosted EDU networks as 'featured' books so that they could be displayed in a place of prominence on other web resources we control (like the pressbooks.com marketing site and the Pressbooks Directory, to give two examples).

Idea: we should add an 'is_featured' metadata field in the Algolia index and change it directly there. Our front-end or the Pressbooks marketing website could query the Algolia API to get & display requested info about only those books flagged as 'featured'

When selected, the marketing site should be able to query the list of all featured books and display a certain number of featured books (cover, title, link). Probably would need to query the Algolia database? See https://www.algolia.com/doc/rest-api/search/ The Pressbooks directory should also be able to display a certain number of featured books (some subset of metadata collected in Algolia database).

• [x] create Algolia 'is_featured' field with default null or no value which can be manually edited by Algolia admin user
• [x] create Algolia 'is_recommended' field with default null or no value which can be manually edited by Algolia admin user
• [x] change fetcher to preserve non-book API attributes when replacing updated records (use method 1 in Ho Man's sketch.io flow: https://drive.google.com/file/d/1LssAJqInY3pp1OOxox8VRA6wlJvV_Vvu/view)
• [x] Document how a user can manually edit the JSON for a book to change is_featured flag

[hughmcguire]

My preference would be to make this part of the backend of Directory, rather than a Superadmin on PB Networks. Reasons: -- we control what is on the Directory, it is our index of what is available in the PB universe, and should evolve as the best/easiest place to find great books available in the PB ecosystem -- It gives us one place to start light curation of content, rather than many disparate places (that is, it's simpler to do it on directory) -- The "featuring" is really an expression of our curatorial input on the Directory, whereas the PB systems are "owned" by the institutions

I don't have a comment on complexity of doing this in the Directory vs doing this in PB Network backends for Superadmins.

[hughmcguire]

Note that this could result in default sorting, so that eg when you select: biology, the first 10 items could be sorted by featured ...

this then makes it easy to build out the directory so that any given subject can be populated with lightly-curated quality content.

meaning that when we are setting up a new network, we could more easily offer curated selection on quality content to pre-populate the network.

[SteelWagstaff]

@hughmcguire we're thinking to add two metadata fields for each Algolia record for now. One would be 'is_featured' and could be used to indicate that the book should be used in 'featured books' displays on the directory or our marketing site (probably no more than a dozen books at any one time, and could be changed as needed). The other would be 'is_recommended' and could be applied to thousands of books (over time), could eventually become a faceted field, and could also be used to filter/sort search results book cards.

[hughmcguire]

@SteelWagstaff excellent.

[richard015ar]

About update records massively (create a new field or change anything): There is not a way to do it through the interface, and there is not a single API method to do that, except we specify the ObjectID (identifier): https://discourse.algolia.com/t/update-a-field-in-all-objects/7183

In order to achieve that I will create an script that reads the records and update those using custom batch method. Probably it could increment the estimation time.

[richard015ar]

I created an script for massive update. I added the new fields for test_Pressbooks_directory (and replicas) indexes. After confirmation that it looks as expected and fetcher solution is applied, we are ready to update to production, dev and staging indexes.