search

vinceanalytics / vince

vince is a High Performance , API only , distributed, in-memory alternative to Google Analytics
https://www.vinceanalytics.com/
Apache License 2.0
87 stars 3 forks source link

BUG - Obtaining time series reduces stats #11

Open Ziedelth opened 1 week ago

Ziedelth commented 1 week ago

It's me again! When I make the query to retrieve the timeseries: {{ANALYTICS_URL}}/api/v1/stats/timeseries?site_id={{ANALYTICS_SITE_ID}}&period=6mo&metrics=visitors,visits,pageviews,views_per_visit,bounce_rate,visit_duration&property=source

I have the requested results:

{
    "results": [
        ...,
        {
            "timestamp": "2024-06-25T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 248,
                "views_per_visit": 2.883720930232558,
                "visit_duration": 14129.59387209302,
                "visitors": 15,
                "visits": 86
            }
        },
        {
            "timestamp": "2024-06-26T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 1650,
                "views_per_visit": 2.0728643216080402,
                "visit_duration": 4997.206679648245,
                "visitors": 24,
                "visits": 796
            }
        },
        {
            "timestamp": "2024-06-27T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 448,
                "views_per_visit": 2.1333333333333333,
                "visit_duration": 1284.948371428571,
                "visitors": 7,
                "visits": 210
            }
        }
    ]
}

Except that when I wait a few minutes and a few visits on the site (Googlebot), I see that they decrease in value:

{
    "results": [
        ...,
        {
            "timestamp": "2024-06-25T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 248,
                "views_per_visit": 3.1392405063291138,
                "visit_duration": 15381.583202531643,
                "visitors": 15,
                "visits": 79
            }
        },
        {
            "timestamp": "2024-06-26T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 1650,
                "views_per_visit": 2.0833333333333335,
                "visit_duration": 5022.445097222227,
                "visitors": 24,
                "visits": 792
            }
        },
        {
            "timestamp": "2024-06-27T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "pageviews": 448,
                "views_per_visit": 2.1853658536585368,
                "visit_duration": 1316.2885756097555,
                "visitors": 7,
                "visits": 205
            }
        }
    ]
}

Are the values lost?

Here are the server logs:

{"time":"2024-06-27T12:26:24.802203243Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9292-1719491184802.tmp"}
2024-06-27T12:26:25.004Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:28:37.519Z [INFO]  raft: starting snapshot up to: index=9302
{"time":"2024-06-27T12:28:37.519635037Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9302-1719491317519.tmp"}
2024-06-27T12:28:37.718Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:31:36.987Z [INFO]  raft: starting snapshot up to: index=9317
{"time":"2024-06-27T12:31:36.987815115Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9317-1719491496987.tmp"}
2024-06-27T12:31:37.160Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:34:53.737Z [INFO]  raft: starting snapshot up to: index=9323
{"time":"2024-06-27T12:34:53.737951082Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9323-1719491693737.tmp"}
2024-06-27T12:34:53.935Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:37:14.080Z [INFO]  raft: starting snapshot up to: index=9333
{"time":"2024-06-27T12:37:14.080494325Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9333-1719491834080.tmp"}
2024-06-27T12:37:14.278Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:39:48.275Z [INFO]  raft: starting snapshot up to: index=9335
{"time":"2024-06-27T12:39:48.275843028Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9335-1719491988275.tmp"}
2024-06-27T12:39:48.473Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
2024-06-27T12:42:53.113Z [INFO]  raft: starting snapshot up to: index=9341
{"time":"2024-06-27T12:42:53.113912014Z","level":"INFO","msg":"creating new snapshot","component":"snapshot","path":"/app/data/snapshots/snapshots/14-9341-1719492173113.tmp"}
2024-06-27T12:42:53.315Z [ERROR] raft: failed to take snapshot: error="failed to persist snapshot: arrow/ipc: tried to write record batch with different schema"
Ziedelth commented 1 week ago

After a few hours, the problem seems to be not only the reduction, but also the increase over the days that have already elapsed. I've just repeated the query with a different PC, and the results are no longer the same.

{
    "results": [
        {
            "timestamp": "2024-06-25T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "events": 248,
                "pageviews": 248,
                "views_per_visit": 2.339622641509434,
                "visit_duration": 11382.172471698112,
                "visitors": 15,
                "visits": 106
            }
        },
        {
            "timestamp": "2024-06-26T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "events": 1650,
                "pageviews": 1650,
                "views_per_visit": 2.6025236593059935,
                "visit_duration": 6267.2210410094685,
                "visitors": 24,
                "visits": 634
            }
        },
        {
            "timestamp": "2024-06-27T00:00:00Z",
            "values": {
                "bounce_rate": 1,
                "events": 975,
                "pageviews": 975,
                "views_per_visit": 1.8431001890359169,
                "visit_duration": 1573.7440737240088,
                "visitors": 18,
                "visits": 529
            }
        }
    ]
}