This commit adds synthesis of numerical column data. It uses the statistical library Accord.NET to generate a distribution from the histogram buckets.
Other additions:
Estimation of distribution type (not sure this is useful but I found it an interesting idea)
'Descriptive Stats': mean, quartiles, stddev etc. this came basically for free with the generated distribution. It might prove more accurate / robust than the stats we already generate so I included it, we can see going forward which is better.
Removed some unnecessary calls to Task.Run(). In JsonApiClient there were some calls to Task.Run(), I think thy existed purely to be able to the cancellation token. I think this is wasteful since in each case it was a relatively trivial operation (ie. no sigificant i/o) and Task.Run() actually spins up a background thread just for this. Also, if the underlying operation doesn't support the cancellation token, then adding it to the Task.Run() call has no effect except to block the creation of the Task if the token is activated. So the following are more or less equivalent:
await Task.Run(() => MyTask(), token);
// Roughly the same, except no background thread is created
token.ThrowIfCancellationRequested();
await MyTask();
- ~Added a basic cache to the ContextBuilder because the api calls to get the datasources were taking forever to parse. Now the context builder checks the cached tables (if present) and if there is a validation error it refreshes the cache and validates again.~ Actually this change caused issues with authentication, see commit 83149f49120a15129c2e223df9b61244be2b2fc1
- Added a badge to the README to indicate build status (I know it's kind of pointless but I was curious how to do this 🤷)
This commit adds synthesis of numerical column data. It uses the statistical library Accord.NET to generate a distribution from the histogram buckets.
Other additions:
Task.Run()
. InJsonApiClient
there were some calls to Task.Run(), I think thy existed purely to be able to the cancellation token. I think this is wasteful since in each case it was a relatively trivial operation (ie. no sigificant i/o) and Task.Run() actually spins up a background thread just for this. Also, if the underlying operation doesn't support the cancellation token, then adding it to the Task.Run() call has no effect except to block the creation of theTask
if the token is activated. So the following are more or less equivalent:// Roughly the same, except no background thread is created token.ThrowIfCancellationRequested(); await MyTask();