Metropolitan-Council / tc.sensors

Package with functions to pull sensor data, sensor IDs, and sensor configuration for MnDOT metro district
https://metropolitan-council.github.io/tc.sensors
Other
1 stars 0 forks source link

pull_sensor: add in empty data for sensors that are missing all data #3

Closed ashleyasmus closed 4 years ago

ashleyasmus commented 4 years ago

When pull_sensor finds a sensor with no data at all, it just prints one empty row of data with date = "NA". Though it might lead to data storage/data size issues, it would be nice to have the option to get all those rows of missing data for easier aggregation down the road.

At the moment I do this outside of the pull_sensor function but it seems better to do this inside the function for efficiency? Maybe.

Something like:

# construct an empty data frame of hours, days and minutes
empty_dates <- expand.grid(date = date_range, 
                           hour = 0:23, 
                           min = seq(from = 0, to = 59.5, by = 0.5))
empty_dates <- data.table(empty_dates)

And then within the pull:

 for (i in 1:num_dates) {
    loops_ls[[i]] <- tc.sensors::pull_sensor(j, date_range[[i]])
    }

  loops_df <- data.table::rbindlist(loops_ls)
  loops_df <- merge.data.table(empty_dates, loops_df, all.x = T)
eroten commented 4 years ago

Issue closed with implementation of fill_gaps parameter in pull_sensor(). Default is TRUE.

59fac22b1eeab7780ee30e414daad9da84e0f08e