getparthenon / monorepo

Monorepo for Parthenon development
https://getparthenon.com
GNU General Public License v3.0
4 stars 1 forks source link

[Export] Data Exporter #7

Closed that-guy-iain closed 1 year ago

that-guy-iain commented 1 year ago

Description

Problem

It's common for companies to need to export data for their users to use via various formats. CSV/Excel/etc.

There are multiple ways of doing this. In foreground request process or background process.

One is providing the export file during a request and returning the file. This works at lower scales but often results in hacks as they grow. Generally either being increasing memory or timeouts. Both of which result in errors before being fixed and the same fix being done multiple times. The reality is, once it goes to a certain scale, it needs to be done in the background.

Once it's being done in the background, there are two options:

Solution

Provide an export system that supports multiple file types.

File Type support:

Export types:

Download page support:

Example

No response