search

transferwise / pipelinewise-target-postgres

Singer.io Target for PostgreSQL - PipelineWise compatible
https://transferwise.github.io/pipelinewise/
Other
21 stars 39 forks source link

Limited column type support #56

Open adnan-shahpurwala opened 3 years ago

adnan-shahpurwala commented 3 years ago

Why can't we use the different datatypes supported by postgres? I'm trying to insert a column of type "arrays" but they always get changed to "jsonb"

Could it be as simple as modifying the function target_postgres.db_sync.column_type? If no, why?

adnan-shahpurwala commented 3 years ago

More context: This is my schema and dummy data:

schema = {
    'properties': {
        'id': {'type': 'string', 'key': True},
        'relative_position': {'type': 'array', 'items': {'type': 'number'}, 'key': False},
    }
}

# Rows/Docs to dump
rows = [
    {'id': 'a', 'relative_position': (0, 1, 2)},
    {'id': 'b', 'relative_position': (0, 1, 2)},
]

However, when I run the pipeline on dummy data, this is what I see in postgres:

image