sparky8512 / starlink-grpc-tools

Random scripts and other bits for interacting with the SpaceX Starlink user terminal hardware
The Unlicense
482 stars 64 forks source link

Consecutive runs of dishHistoryInflux.py generate error. #12

Closed neurocis closed 3 years ago

neurocis commented 3 years ago

First run executes and populates as follows:

> SELECT * FROM "spacex.starlink.user_terminal.ping_stats"
name: spacex.starlink.user_terminal.ping_stats
time                count_full_obstructed_ping_drop count_full_ping_drop count_full_unscheduled_ping_drop count_obstructed count_unscheduled id                           samples total_obstructed_ping_drop total_ping_drop   total_unscheduled_ping_drop
----                ------------------------------- -------------------- -------------------------------- ---------------- ----------------- --                           ------- -------------------------- ---------------   ---------------------------
1610416033180088000 0                               0                    0                                0                0                 ut01000000-00000000-000xxxxx 3600    0                          5.000181436538696 0

Consecutive runs error with:

SpaceX.API.Device.Device is a service:
service Device {
  rpc Handle ( .SpaceX.API.Device.Request ) returns ( .SpaceX.API.Device.Response );
  rpc Stream ( stream .SpaceX.API.Device.ToDevice ) returns ( stream .SpaceX.API.Device.FromDevice );
}
Failed writing to InfluxDB database: 400: {"error":"partial write: field type conflict: input field \"count_full_obstructed_ping_drop\" on measurement \"spacex.starlink.user_terminal.ping_stats\" is type float, already exists as type integer dropped=1"}
sparky8512 commented 3 years ago

Hmm... I guess I need to make sure the stats that could be fractional always report as float, but count_full_obstructed_ping_drop shouldn't be fractional unless the pop_ping_drop_rate value in the raw data is greater than 1, which... I guess I shouldn't have assumed would never happen, but I'm not even sure what that means. More than 100% of the pings in that second were dropped?

sparky8512 commented 3 years ago

Could you run dishHistoryStats.py -v against that after verifying the problem is still reproducible? I think I can address the type conflict by manually forcing the data values, but I don't understand why that particular stat did what it did.

sparky8512 commented 3 years ago

Oh. Never mind. It's not greater than 1, it's being treated as a float because the variable it adds is sometimes a float. I guess I never ran into this because the history data from my dish is full of fractional ping drops so that variable would frequently be a float.

sparky8512 commented 3 years ago

Whoops, didn't mean to close this.

@neurocis : please retest with that change, but note that you may need to delete the existing data point from the database if any of the fields already got in with the wrong type. A good test will be to alternate between just an hour's worth of samples (the default) and all samples (dishHistoryInflux.py -a). If it looks good now, send this back to me, as I still need to update starlink_json.py for consistency.

sparky8512 commented 3 years ago

I'm pretty sure it's fixed now, as of that later change. I also checked the status Influx script and verified that it will not have the problem, since it just passes the grpc object attributes directly, and those are guaranteed to always be the same type (floats, bools, or strings).

Please re-open if it continues to be a problem with both the above changes.