DOI-USGS / streamMetabolizer

streamMetabolizer uses inverse modeling to estimate aquatic metabolism (photosynthesis and respiration) from time series data on dissolved oxygen, water temperature, depth, and light.
http://usgs-r.github.io/streamMetabolizer/
Other
36 stars 22 forks source link

Include NA rows when there are missing data #74

Open Arroita opened 9 years ago

Arroita commented 9 years ago

Here what I do to make sure the time lag from row to row is constant. Let me know if you want me to modify it or check the data sets! Hope it helps.

#Just to use an example
library(powstreams)
t<-"nwis_07239450"
d<-c("depth_calcDischHarvey","doobs_nwis","dosat_calcGGbts","wtr_nwis","par_calcLat")
dat<-get_ts(d,t)

#Create reference date and time column. I used POSIXct format because this is the format of the data set
#For each stream starting and ending dates and frequence should be checked.
library(chron)
dtime<-seq(from=as.POSIXct("2007-10-01 05:00:00",tz="GMT"),to=as.POSIXct("2015-06-06 23:30:00",tz="GMT"),by="30 min")   

#I checked the dimensions for both and it seems in this particular case there are a lot of rows missing
Plotting the DateTime column works too
dim(dat)
length(dtime)
plot(dat$DateTime)

#Match DateTime column in your data set with the reference dtime column
depth<-dat$depth[match(dtime,dat$DateTime)]
doobs<-dat$doobs[match(dtime,dat$DateTime)]
#....

#I usually create a new data frame. It's also great to check it matches the number of rows you expect considering the number of days and frequency you have
dat<-data.frame(dtime,depth,doobs...)
aappling-usgs commented 8 years ago

Might also think about imputing data for these missing values. This new package could be relevant: https://matloff.wordpress.com/2015/09/16/new-r-softwaremethodology-for-handling-missing-dat/

aappling-usgs commented 8 years ago

Also check out http://blog.rstudio.org/2016/02/02/tidyr-0-4-0/ and the complete(), full_seq(), and nesting() functions for filling with NAs

aappling-usgs commented 8 years ago

other imputation packages: MissForest, MissMDA