search

thundercomb / poetrydb

The Internet's first Poetry API
http://poetrydb.org
GNU General Public License v2.0
523 stars 35 forks source link

Heroku Application Error #36

Closed hurali97 closed 4 years ago

hurali97 commented 4 years ago

Hi maintainers! This API is really great. There's this issue when getting the random/4 or any number after random. However, random/ works fine but random/4 or any number throws the attached error. It was working fine a couple of weeks ago.

Screenshot from 2020-09-23 22-49-15

thundercomb commented 4 years ago

Hi @hurali97

Thanks for raising the issue. It looks like the backend cluster, for some reason, no longer returns results for the sampling query in the case when there are no other input fields. When there are other input fields it works fine, eg:

https://poetrydb.org/author,random/Shakespeare;3

The cluster was migrated to a different provider last week, which may have someting to do with it. However this issue didn't come up during tests, so looks like a degradation of some sort. Nevertheless, no cluster or database issues are being reported.

Mongo random sampling can have performance implications in some situations. So rather than rely on the backend I've updated the sampling method to use aggregation instead.

Seems to do the trick, but let me know if it happens again.

hurali97 commented 4 years ago

Thank you very much for such quick response.

serenag01 commented 4 years ago

https://poetrydb.org/author,random/Shelley;3

The above URL seems to still yield the application error

thundercomb commented 4 years ago

Thanks for spotting, @serenag01. It looks like sampling over 'long' poems (like Shelley's) causes a kind of timeout in the backend. I've now moved all sampling to the aggregate method, which is more efficient.

This should hopefully allow all random searches to return within a reasonable timeframe.