mosajjal / dnsmonster

Passive DNS Capture and Monitoring Toolkit
https://dnsmonster.dev
GNU General Public License v3.0
316 stars 53 forks source link

Rework to modern Materialized Views #62

Closed mzealey closed 1 year ago

mzealey commented 1 year ago
netlify[bot] commented 1 year ago

Deploy Preview for unrivaled-llama-9deaa9 canceled.

Name Link
Latest commit 5b41c6b4d175a8809378c400e2d414250e3348ee
Latest deploy log https://app.netlify.com/sites/unrivaled-llama-9deaa9/deploys/6442925e3988f900083d10b4
mosajjal commented 1 year ago

looks good. I just tested it against some local data and it works. do you have any testing/benchmark on this? wondering if this is gonna affect the performance.

mzealey commented 1 year ago

There's no performance issue - it's just explicitly specifying the inner tablesOn 22 Apr 2023 08:03, Ali Mosajjal @.***> wrote: looks good. I just tested it against some local data and it works. do you have any testing/benchmark on this? wondering if this is gonna affect the performance.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

dnsstub commented 1 year ago

hi @mzealey

Move away from using inner MV's to separate MV and defined tables as per CH recommendations it seems very much fine

`Switch all SummingMergeTree to AggregatingMergeTree to allow for more functionality in future if needed can you please put some light over it because you are replacing every SummingMergeTree Instance with AggregatingMergeTree that doesnt seems to be efficient in every condition

`

mzealey commented 1 year ago

I've never seen a performance difference between Summing and Aggregating (with SimpleAggregationFunction), and Aggregating gives you so much more flexibility over extending function if you need.

dnsstub commented 1 year ago

i will post my comparisions here in 1 -2 days