adhocteam / nytimes-library

A collaborative documentation site, powered by Google Docs.
https://nyt-library-demo.herokuapp.com/
Apache License 2.0
1 stars 1 forks source link

Fix sporadic 500 error on search #95

Closed bkjohnson closed 1 year ago

bkjohnson commented 1 year ago

Closes https://github.com/adhocteam/nytimes-library/issues/72

Whenever a search term was used that would return enough results so that pagination was required, the user would see a 500 error and the server logs would show that we were getting a 400 error from the Google API.

Proposed changes:

TLDR: This makes it so that we use the same query when we make successive calls using nextPageToken.

1. When we first call fullSearch(), we pass it several arguments, including excludedFolders and mimeTypes:

https://github.com/adhocteam/nytimes-library/blob/b4cb37120482ca26e1a785a2429bcfb6c362bfcb/custom/search.js#L25

2. We then use those values to create the options object which we pass to the Drive API (lines 38, 39, and 45):

https://github.com/adhocteam/nytimes-library/blob/b4cb37120482ca26e1a785a2429bcfb6c362bfcb/custom/search.js#L38-L45

3. The options are created using those excludedFolders and mimeTypes values:

https://github.com/adhocteam/nytimes-library/blob/b4cb37120482ca26e1a785a2429bcfb6c362bfcb/custom/search.js#L107-L123


If the search returned more results than can fit into our page size, we get a nextPageToken and make a recursive call, but this time we don't include the excludedFolders and mimeTypes that we did on line 25.

https://github.com/adhocteam/nytimes-library/blob/b4cb37120482ca26e1a785a2429bcfb6c362bfcb/custom/search.js#L51

The API docs say this about pageToken:

The token for continuing a previous list request on the next page. This should be set to the value of 'nextPageToken' from the previous response.

Since the nextPageToken that gets generated is for a specific query, we were getting an error when we tried to use the same token with a different query.

Acceptance criteria validation

timwright12 commented 1 year ago

Yes! 🎉

bkjohnson commented 1 year ago

I kind of feel bad about not adding some kind of test here, but I'll try to familiarize myself with the setup and maybe get that done once the holidays are over :sweat_smile:.