jayvynl / django-clickhouse-backend

Django clickhouse database backend.
MIT License
130 stars 21 forks source link

Support for AggregateFunction under AMT #73

Closed vigneshshettyin closed 10 months ago

vigneshshettyin commented 10 months ago
CREATE TABLE test_db.shard_table on cluster default
(
    `cmo_id` String,
    `year` UInt16,
    `code` LowCardinality(String),
    `clp_id` AggregateFunction(uniq, Int64),
    `plp_id` AggregateFunction(uniq, Int64)
)
ENGINE = AggregatingMergeTree
PARTITION BY year
ORDER BY (code, cmo_id, year)
SETTINGS index_granularity = 8192

Hi @jayvynl any method to define above table definition as a unmanaged model using the package?

jayvynl commented 10 months ago

Usually you don't want to use AggregationMergeTree directly. Create a view which group by the AMT, create unmanaged model of the view.

vigneshshettyin commented 10 months ago

Yes, I did that but I am getting below error while accessing it

django.db.utils.OperationalError: Code: 50. Unknown type AggregateFunction(uniq, Int64)

I have defined the model as below but unable to access objects

class TestMV(models.ClickhouseModel):
    id = StringField(primary_key=True)
    cmo_id = StringField(blank=True, null=True)
    code = StringField(blank=True, null=True)
    plp_agg = StringField(blank=True, null=True)
    clp_agg = StringField(blank=True, null=True)
    roll_up_date = StringField(blank=True, null=True)
    parent_id = StringField(blank=True, null=True)
    code_level = IntegerField()

    class Meta:
        managed = False
        db_table = 'test_data'

plp_agg and clp_agg are of type AggregateFunction(uniq, Int64)

Can you please check this on priority @jayvynl

jayvynl commented 10 months ago
CREATE  VIEW test_db.data_view on cluster default AS (
SELECT 
`cmo_id`, `year`, `code`,
uniq(`clp_id`), uniq(`plp_id`)
FROM test_db.shard_table
GROUP BY `cmo_id`, `year`, `code`
 )

Create unmanaged table map to this view.