questdb / questdb-connect

SQLAchemy and Apache Superset extensions for QuestDB
Apache License 2.0
14 stars 6 forks source link

VARCHAR type not supported #14

Closed antoniocaiazzo closed 1 month ago

antoniocaiazzo commented 3 months ago

Hi all,

I just started playing around with QuestDB and this connector, and I hit a wall with storing string types: I can't find the way to specify in the model that a field is a varchar instead of a string.

According to the documentation string types are deprecated in favour of varchar, but apart from this using varchar types unlocks various benefits, one of which very important is to be able to use that field in the dedup key columns.

Right now I'm getting:

psycopg2.DatabaseError: deduplicate key column can only be fixed size column [column=device_id, type=STRING]
LINE 1: ..."ts") PARTITION BY DAY WAL DEDUP UPSERT KEYS("ts","device_id...

This is my model:

class Readings(Base):
    __tablename__ = "readings"
    __table_args__ = (
        qdbc.QDBTableEngine(
            "readings",
            "ts",
            is_wal=True,
            dedup_upsert_keys=("ts", "device_id", "meter"),
        ),
    )

    ts = Column(qdbc.Timestamp, primary_key=True)
    device_id = Column(qdbc.String)
    meter = Column(qdbc.Symbol)
    payload = Column(qdbc.String)

Is there something I'm overlooking, or is this just not possible at the moment using class models over writing the CREATE TABLE by hand?

Thank you!

nwoolmer commented 2 months ago

Hi @antoniocaiazzo ,

We can do this in future. In the meantime, deduplication is now supported for STRING, VARCHAR and BINARY types, so this should resolve the original problem.