ISG-ICS / cloudberry

Big Data Visualization
http://cloudberry.ics.uci.edu
91 stars 82 forks source link

[Frontend] Scatterplot map points may disappear after the last batch for some keywords #821

Closed baiqiushi closed 2 years ago

baiqiushi commented 3 years ago

Symptom

On OCHCA http://ochca.ics.uci.edu, use keyword passport covid and switch to pinmap; You will see the points are shown and then disappear soon;

Potential direction

The symptom can NOT be reproduced on PROD: https://cloudberry.ics.uci.edu/apps/twittermap; The differences between these two servers are the following:

PROD

berry.firstquery.gap = "2 days" berry.query.gap = "1 day"

 - Twittermap
```bash
# OCHCA
pinmap.samplingDayRange = 1460
pinmap.samplingLimit = 100000

# PROD
pinmap.samplingDayRange = 1460
pinmap.samplingLimit = 100000
pinmap.binaryTransfer = true

Based on these observations, there are possible two issues. One is some frontend logic assumes DRUM returns a certain number of batches and that might be violated due to the more aggressive DRUM setting on OCHCA server; The other is when binaryTransfer is enabled on OCHCA server, the symptom will appear even for covid keyword, which means the binary transfer feature for pinmap might have some bug too when certain behavior happens in DRUM;

baiqiushi commented 2 years ago

Solved by #818