Closed tswast closed 1 week ago
I think approx_quantile
makes sense to expose (other backends like DuckDB also offer this).
@chloeh13q pointed out a bit of potential inconsistency, where we implement median for the BigQuery backend using the approx_quantile
method, but don't call it approx_median
; in that case, would it be more consistent to expose approx_
versions of both quantile and median?
I think it makes sense to revisit this next week once most of the maintainers are back from SciPy.
Tangential but ...
where we implement median for the BigQuery backend using the
approx_quantile
method, but don't call itapprox_median
...
@deepyaman Can you point to where this happens?
We do have an Approximate Median supported using Approximate quantiales: see https://github.com/ibis-project/ibis/blob/b44dac2a5d0346ed0f3dbdc05597104f64e40779/ibis/backends/bigquery/compiler.py#L148-L149
and the exact Median it's not supported: https://github.com/ibis-project/ibis/blob/b44dac2a5d0346ed0f3dbdc05597104f64e40779/ibis/backends/bigquery/compiler.py#L38C1-L43C20
@tswast Thanks for the issue!
I think it makes sense to add an Column.approx_quantile
method that mirrors the quantile
method that we have now.
Any takers from the BigFrames team?
Is your feature request related to a problem?
There isn't a great way to use BigQuery's APPROX_QUANTILES feature to get values besides the approximate median in Ibis.
What is the motivation behind your request?
To mimic pandas
describe
, BigQuery DataFrames uses APPROX_QUANTILES to get an approximate 25th percentile, median, and 75th percentile.Note: the percentiles are configurable in pandas, but unfortunately BigQuery SQL's # of bins approach makes it difficult to support arbitrary percentiles.
Describe the solution you'd like
From BigQuery DataFrames, perspective, it'd be great if there was an API to get evenly-spaced approximate quantiles, but perhaps a approx_quantile function that takes an integer from 0 - 100 would be more flexible? Or 0 to 1 to mimic pandas, but with the note that some backends like bigquery only support precision up to a certain point (maybe nearest 0.05 or nearest 0.01)?
What version of ibis are you running?
8.x, but working on 9.x upgrade
What backend(s) are you using, if any?
BigQuery
Code of Conduct