Expose issues in Parquet format via datasette

[Ben-Hodgkiss]

Overview

Expose issues in Parquet format via datasette

Background Following the design proposal for an internal API, we would like to prove some technology choices which include the use of Fast API with DuckDB accessing Parquet on S3.

This work was identified during the spike on API design.

By exposing the issues in Parquet format via datasette, we will know whether Parquet will be an appropriate format for consuming via datasette as well as a new internal API.

A spike was done on using parquet in datasette by @ssadhu-sl here, which contains a fork of the datasette parquet plugin that we can start with

Tech Approach

• Add Parquet plugin to datasette
• Modify Parquet plugin as necessary to work alongside our exisitng datasette configuration

Acceptance Criteria/Tests

• Code merged within datasette repo
• Datasette exposes issues in Parquet format to production environment

---

Ticket Management - DELETE this section once completed

• Complete all relevant tags - make sure Infrastructure is tagged so it is picked up by our filters!
• Complete the time estimate field
• Make sure you have a PR link in the Overviewabove.
• If relevant, link to the relevant OKR as an attachment.
• Link to any tickets in other boards that are dependent on it.

[Ben-Hodgkiss]

@CarlosCoelhoSL - when picking this up, please liaise with @cpcundill about work he's already done on #102 as it may have done some of the ticket already. Also worth speaking with @eveleighoj to see if we can link the new "Expectations" work on this ticket into Parquet and Datasette at the same time.