Automate Reading and Formatting of Sensor Data

[jpshanno]

I work with sensors from a variety of instrument manufactures that all output different data formats. Some of the instruments export csv with or without extra header information, some export using another structured format such as XML or JSON, and some export a proprietary format. Incorporating new data into data archives or analyses becomes a multi-stop process. The first steps in getting at the data are often conversion (using the manufacturer's software) and formatting to match your desired data structure to allow merging datasets during analysis. A good solution for this would be a set of tools or functions that takes the raw data format as an input and outputs an analysis-ready dataset. This would let researchers and data managers quickly bring in new data in a single step. Ideally as many manufactures as possible would be available.

[adroghini]

@jpshanno As mentioned in other ticket, I have some HOBO temperature data to contribute. Not too messy, but there is some extra header information, and some info that is only stored in the file name. Also, the DateTime column header includes information about the timezone, which changes depending on the time of the year (DST or not) - it would be great to have some code that would query and use that timezone info to convert the data in that column to a POSIXct format (and preferably a UTC timezone).