BritishGeologicalSurvey / etlhelper

ETL Helper is a Python ETL library to simplify data transfer into and out of databases.
https://britishgeologicalsurvey.github.io/etlhelper/
GNU Lesser General Public License v3.0
105 stars 25 forks source link

Add tuple_row_factory and list_row_factory #132

Closed volcan01010 closed 1 year ago

volcan01010 commented 2 years ago

Summary

As an etlhelper user I want tuple and list row factories to allow data transfer in other native Python containers.

Description

The DBAPI default is for database rows to be returned as tuples. ETL Helper provides namedtuple and dictionary row factories to give more convenient access to data e.g. via column names. However, both of these methods require some transformation of data on the ETL Helper machine.

A tuple row factory would minimise data processing and therefore represents a possible speed-up on transfers. A list row factory gives a lightweight container that is also mutable. Mutable containers allow for simpler transform functions as they can be altered in place.

Mutable Immutable
Positional access list tuple
Name access dict namedtuple

Acceptance criteria