search

LinkNLearn / homedata

Sensor data from a home. More information in the ReadMe file.
2 stars 0 forks source link

Timebase for Entire Dataset #18

Open pwdel opened 5 years ago

pwdel commented 5 years ago

We need some kind of timebase for the entire dataset, which can work as the denominator for all possible timebases. One possibility is to just use, "second" measurements, however this could potentially create a huge overhead of data.

nicholaspoggioli commented 5 years ago

I'm not familiar with the timebase concept. Where can I learn more about it?

pwdel commented 5 years ago

Hey, sorry I have been away from this project. I'm going to make sure I get pull requests in my email. I have had to finish something else code related which has now been launched and you can see here if interested : www.confrnz.com

pwdel commented 5 years ago

@nicholaspoggioli This might be helpful to read

https://en.wikipedia.org/wiki/Unix_time

OK, so basically that is a standard time measurement system on the internet / unix / linux systems. The smallest unit of time is 1 second. The data coming from the data set that we're looking at does not occur every second, so do we need to mark data on a second by second basis? Or do we use larger statistical, "buckets" and compress everything?

I think since we are just starting out with this, we could just pick the timebase at 1 second and see whether it makes sense to compress is later.

If we were measuring something which required higher resolution than one second, for example heartbeat modulation, then we would have to do something fancy. I have seen a system that multiples unix time by a factor in order to get a higher resolution for measuring heart beat, and you have to back-calculated it into milliseconds - that's not the case here nor do I think we need that fine grain of resolution.

pwdel commented 5 years ago

Interesting article about timeseries data in python:

https://jakevdp.github.io/PythonDataScienceHandbook/03.11-working-with-time-series.html