search

humanmade / altis-analytics

Analytics module for Altis
https://www.altis-dxp.com/resources/docs/analytics/
GNU General Public License v3.0
9 stars 3 forks source link

SPIKE: Altis Analytics Export #206

Closed robindevitt closed 2 years ago

robindevitt commented 2 years ago

Exporting large amounts of data with the following url /wp-json/analytics/v1/events/2021-12-09?format=json present's two issues:

  1. The API can be slow.
  2. The API sometimes fails.

Expected to see:

An export without any errors or failures.

Acceptance criteria:

shadyvb commented 2 years ago

@robindevitt Seems more of a spike than an issue ?

roborourke commented 2 years ago

We can close this out, I reproduced the issue and created some issues on the aws-analytics repo.

Ultimately ES is the bottleneck and it depends on the amount of data being requested, more data than ~60k rows of data fails consistently but it fails earlier than that inconsistently depending on how busy the site is.

See: