Integrate real datasets for testing purposes

chop-dbhi / origins

Origins is an open source bi-temporal database for storing and retrieving facts for slowly-changing data. It support "time travel" queries and has built-in change detection.

http://origins.link

BSD 2-Clause "Simplified" License

39 stars 5 forks source link

Integrate real datasets for testing purposes #126

Open bruth opened 9 years ago

bruth commented 9 years ago

The ideal features would be:

[ ] Based on a real domain or target use case
- Data monitoring
- Model integration
- https://github.com/chop-dbhi/data-models
- Genealogy
[ ] Includes retractions and explicit valid time values
[ ] Order of millions to billions of facts
[ ] Directory of individual CSV files with sequential filenames
- This ensures they are loaded/transacted sequentially
- Also enables loading a partial set for smaller tests
- Gzipped to save space

bruth commented 9 years ago

The PEDSnet concepts data is a good candidate for this. I am going to start building up a data set that spans some amount of time.