Open jthomasmock opened 7 months ago
/**
* Possible values for TypeDisplay in ColumnSchema
*/
export enum ColumnSchemaTypeDisplay {
Number = 'number',
Boolean = 'boolean',
String = 'string',
Date = 'date',
Datetime = 'datetime',
Time = 'time',
Array = 'array',
Struct = 'struct',
Unknown = 'unknown'
}
@softwarenerd -- I've converted the headers above to type_display
enum.
I'm working on improvements in the backend protocol to better support these statistics right now.
I'm not sure it makes sense to compute number of unique values for arrays and structs for now -- there are varying degrees of ease of computing this in different backends, so I'll punt on that for now and we can address it later once we can investigate how to compute that consistently.
Sounds good! I also think it'd be interesting to hear from users on what types of metrics they'd like. I've indicated that we're holding off on the array/structs/unknowns for now
We can close this once #3021 is merged and validated.
@jthomasmock do we have a good test data that exercises all of the types and thus the column summary statistics? (including precision, null, empty, various types, etc)
we'd want QA to exhaustively cover these statistics to check their validity for the data set.
@jthomasmock do we have a good test data that exercises all of the types and thus the column summary statistics? (including precision, null, empty, various types, etc)
we'd want QA to exhaustively cover these statistics to check their validity for the data set.
I can work on this.
There are some example tests at: https://github.com/r-lib/pillar/blob/main/tests/testthat/test-format_decimal.R
@jthomasmock is there still work to do for Beta on this now that #3021 is merged and validated? (we do need tests but we can close this without them)
@jmcphers I think we are still missing date/datetime stats in: Positron Version: 2024.05.0 (Universal) build 1307
I could pick the backend side for those, I'm assuming @wesm is not working on it yet, right?
I'm working on the float formatting as we speak, so feel free to pick this up
The checkboxes above are the missing stats as of 2024-05-29. Boolean, date, datetime, factor/categorical, and unknown
When the Summary Panel is expanded, it will dynamically calculate and then reveal additional summary statistics for that specific column. This is a lazy operation in the backend as it would otherwise be costly for long/wide datasets.
Summary stats will be right aligned at the decimal place:
Completed:
Parent Categorical: https://github.com/posit-dev/positron/issues/3417
Number
Boolean
String
""
string - implicit missing)String sub-category: Categorical/Factor
Date or Datetime or time
Array -- holding off for now
Struct -- holding off for now
Unknown -- holding off for now