search

LinkNLearn / homedata

Sensor data from a home. More information in the ReadMe file.
2 stars 0 forks source link

What is our data portability strategy? #3

Open pwdel opened 5 years ago

pwdel commented 5 years ago

There are many varieties of data model standardization which can enable the use of pre-built learning models. Rather than building a model completely from scratch, pick models that are pre-built and apply those.

pwdel commented 5 years ago

Interesting link to look at:

https://data.world

pwdel commented 5 years ago

Here's Google's formatting instructions:

https://cloud.google.com/ml-engine/docs/tensorflow/data-prep

Specifically for Tensorflow:

https://www.tensorflow.org/guide/datasets

pwdel commented 5 years ago

AWS File format for AWS ML

https://docs.aws.amazon.com/machine-learning/latest/dg/understanding-the-data-format-for-amazon-ml.html

pwdel commented 5 years ago

Azure Machine Learning:

https://docs.microsoft.com/en-us/azure/machine-learning/studio/import-data-from-local-file

Streaming data... Azure SQL database or SQL Server

pwdel commented 5 years ago

Question @nicholaspoggioli on porting data from an SQL dump into Json as you originally did. Thanks for adding the SQL code in your latest push. However, I'm failing to understand how you accomplished everything...I think I just don't understand how SQL works and have a huge knowledge gap. Did you literally just go in to a terminal and type SQL commands to extract that data? Or did you write code into a script somewhere and execute it? Or did you use some kind of previously built application? Or something else?

nicholaspoggioli commented 5 years ago

Imported the dump code into an SQL query in MySQL workbench. Edited it to produce tables with the data. Exported those tables as json files. This executed within mysql workbench

On Mon, Dec 17, 2018, 5:29 PM Patrick Delaney <notifications@github.com wrote:

Question @nicholaspoggioli https://github.com/nicholaspoggioli on porting data from an SQL dump into Json as you originally did. Thanks for adding the SQL code in your latest push. However, I'm failing to understand how you accomplished everything...I think I just don't understand how SQL works and have a huge knowledge gap. Did you literally just go in to a terminal and type SQL commands to extract that data? Or did you write code into a script somewhere and execute it? Or did you use some kind of previously built application? Or something else?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/LinkNLearn/homedata/issues/3#issuecomment-448038549, or mute the thread https://github.com/notifications/unsubscribe-auth/AYcJeEhMFLfhy-cr14R_e6zI_Y5jqt62ks5u6CjNgaJpZM4YVz2f .