kahaaga
commented
5 years ago Thanks for bringing Measurements.jl to my attention; it had completely slipped my radar. I’ll make sure to include a link to it in the documentation in the next release!
From what I can see, the main difference between the packages is that Measurements.jl always assumes that the uncertainty is a standard deviation (so working with normal distributions), and that it focuses on the propagation of errors during computations using linear error propagation theory.
In practice, however, values are often not normally distributed. The purpose of UncertainData.jl is mainly to have a consistent way of treating uncertain values of all types, be it theoretical distributions with known parameters (e.g. a normal distribution), populations with weighted probabilities, or more complex empirical distributions that have to be approximated by a kernel density estimate. With UncertainData.jl, the user should be able to mix all types of uncertain values and resample the data (possibly subject to some sampling constraints) consistently, regardless of uncertainty type.
A user can use this to, for example, visualize uncertainties of different types in a consistent manner by applying the same constraint across observations (see the example below). Other use cases might be to compute some ensemble statistic of your uncertain dataset over a large number of independent realizations (which you get by resampling the dataset).
Thus, the main difference between this package and Measurements.jl is that that we focus on flexibility in data representation and easy resampling of different types of uncertain data. I'd use Measurements.jl if my data were normally distributed and my goal was to perform some computation and get exact uncertainties using linear error propagation theory. Measurements.jl also handles functional correlation and some linear algebra operations, which is very neat. If I'm interested in ensemble statistics over more complex uncertain datasets, I'd use the approach in this package. For statistical computations, the approach here is to just resample your data a bunch of times, then obtain statistics which themselves have some uncertainty. To compute some statistic over an uncertain value or dataset, the approach is basically this - which might or might not be sufficient for your needs, depending on how rigorous you want to be.
For example, I'm currently integrating this package with CausalityTools.jl to, in a consistent manner, measure information flow between time series where both time indices and data values are uncertain and follow different distributions (i.e. asking, for example, there asymmetric information flow between two time series even when I take into consideration dating errors and experimental error?).
I was motivated to create this package because in my field, working with paleoclimate and biological proxy records, data values often come from different sources and have uncertainties that follow all kinds of distributions (often multimodal). Also, measurements often have uncertainties both in the data index (time, for example) and in the data value, which have to be treated separately (functionality for handling this is coming in a future release). To avoid re-inventing the wheel for every new dataset I work with, I decided to make this package instead, so I can do data analysis with uncertain data in a consistent manner.
In the future, the aim is to improve the resampling part by integrating with Interpolations.jl, provide options for data imputation, support uncertain vectors (i.e. represented by multivariate distributions), make some plot recipes where you can directly use sampling constraints and make something that would allow conversion of data from spreadsheets.
In summary: with this package, you may save time when working with values having many different types of uncertainties, and uncertainty estimates on computations are obtained by resampling. Measurements.jl, on the other hand, propagates errors exactly and supports (I think) most or all base Julia math, and might thus be a better choice if your application calls for it, if your data justifies it (are normally distributed) and/or if uncertainties are correlated.
I hope this clarifies things. Comments and suggestions are most welcome!
Example 1
CIn this contrived example, I have a bunch of different observations from different distributions, some with known parameters, some without, and I want to plot realizations falling within the 30-th and 70th percentile ranges of the distributions.
using UncertainData
using Distributions, KernelDensity
using Plots
# Define some theoretical distributions
dμ, dσ = Uniform(-5, 5), Uniform(0.1, 3)
dα, dβ = Uniform(2, 3), Uniform(4, 9)
normally_distributed_vals = [UncertainValue(Normal, rand(dμ), rand(dσ)) for i = 1:10]
beta_distributed_vals = [UncertainValue(Beta, rand(dα), rand(dβ)) for i = 1:10]
# Create some samples obeying some complicated distribution and represent
# the samples by kernel density estimates.
M1 = MixtureModel([Normal(1, 0.5), Gamma(0.3, 5), Normal(2, 0.3)])
M2 = MixtureModel([Gamma(3, 4), Beta(3, 4), Frechet(4, 0.3)])
n = 1000
empirical_samples = [rand(M1, n), rand(M2, n)]
kde_dists = UncertainValue.(empirical_samples)
# Gather everything in an uncertain dataset
uvals = [normally_distributed_vals; beta_distributed_vals; kde_dists]
udata = UncertainDataset(uvals)
# Plot 1000 realizations of the uncertain dataset
plot(resample(udata, 1000), lc = :black, lα = 0.01, legend = false)
# Resample 1000 new realization, but restricting each observation to the 30th-to-70th
# percentile range of its support
plot!(resample(udata, TruncateQuantiles(0.3, 0.7), 1000),
lc = :red, lα = 0.01, legend = false)
Example 2
Another contrived example: Following the example above, let's say I had two uncertain datasets.
udata1 = UncertainDataset([beta_distributed_vals; normally_distributed_vals; kde_dists])
udata2 = UncertainDataset([normally_distributed_vals; beta_distributed_vals; kde_dists])I now want to compute the Pearson correlation between the datasets. To do this I can generate, say, 10000 realizations of each dataset and compute the correlation between each pair of realizations.
corrs = cor(udata1, udata2, 10000)We can take this further. Maybe I had good reason to not trust outliers in my dataset. In this case, I could for example restrict the values each observation cold take to the 25th-to-75th percentile range.
corrs = cor(udata1, udata2, 10000, TruncateQuantiles(0.25, 0.75))Maybe I want to disregard outliers for most observations, but not the five first:
constraints = [[NoConstraint() for i = 1:5]; [TruncateQuantiles(0.2, 0.8) for i = 1:length(udata)-5]]
constrained1 = constrain(udata1, constraints)
constrained2 = constrain(udata1, constraints)
corrs = cor(constrained1, constrained2, 10000)From this, I can estimate the correlation by the mean of the pairwise correlations.
μ, σ = mean(corrs), std(corrs)There are probably better ways of doing this, but the examples at least illustrate how the package works.
mkborregaard
rofinn
Just curious - what's the relationship/overlap/complementarity of this package and the uncertainty machinery in https://github.com/JuliaPhysics/Measurements.jl?