we are no longer using the 'Prox2N' column and classifier algorithm to calculate partially inundated cases. Instead, abrupt shift detection (package: 'asdetect') is being used. Steps to run Prox2N have now all been removed.
'Current_Vel_Constant_Values.rda' contained equations to convert tilt to velocity. These have now been incorporated into the 'get.hydrodynamics' function, and so was deleted.
a new step to identify partially inundated cases has been added, so the function now contains several default parameters.
sampling rate is now calculated by subtracting the date of the second row from the first one. This is needed to select the appropriate calibration equation to convert tilt to velocity.
model to classify inundation status is now applied inside the pipe, rather than being assigned to objects (e.g., 'SVM.NF').
acceleration values are being truncated at the higher end so that any value > 0 g (i.e., when the Mini Buoy is dangling downwards) are converted to 0 g.
acceleration values are being truncated at the lower end, so values > -1 + sensitivity of the loggers are being converted to NAs.
1-minute rolling standard deviation in acceleration is being calculated for calculating wave orbital velocity later (only relevant for B4+).
mean of the rolling standard deviation is also now being calculated when aggregating.
acceleration is being converted to tilt (0 = horizontal, 90 = vertical).
a new step looks for NAs between Events. If the length of NA falls below the 'gaps' argument (say because of a misclassification that causes one event to split into two or more), then values are linearly interpolated between the two events. A cleaning step is applied to consecutively number Events once more.
an argument is now included to select the minimum duration of an event below which the event is convert to non inundated.
over a series of lines, the code to calculate partially inundated cases is applied. This includes subsampling the start and end of each flood event to search for partially inundated cases (the default of this set as half the median duration of inundation events), applying the as_detect() function, and overriding 'F' cases as 'P' if highlighted by the as_detect() function identifies them to be part for a rapid change indicating the Mini Buoy is being submerged.
if values are 90 degrees, they are automatically classified as fully inundated (as_detect() can be a little liberal when assigning P cases).
classify entire events as partially inundated cases if the Mini Buoy fails to cross a threshold value (default is 75 degrees)
current and wave orbital velocity calibrations have now been added for every second between 1 and 10 (B4 and B4+) or every 2 minutes between 2 and 10 minutes.
a new column is added stating whether or not the day has complete data
analysis functions
a total of 48 functions have been added, each to analyse the inundation/velocity data in a different way. Previously, these calculations were embedded the dplyr pipe, which was messy
results tables
function to summarise hydrodynamics has been updated (get.statistics() -> get.summary.statistics())
three new functions have been added to tabulate results aggregated by day, event, and tide
results text
get.stats.text() has been updated
statistics
added get.stats() for statistical comparison between target and reference data. STILL IN DEVELOPMENT. It will automatically identify potential outliers, apply U-tests per parameter, and report in the results table if there's a significant difference, and if the target values are higher or lower than the reference site.
FUN_GRAPHICS.R
added a function to plot ggplotly objects and a function to inspect the raw data. NOTE: the functions haven't been applied to any data, and it probably won't work anyway. Looking now into how to use ggplotly in shiny
added code to create a velocity stage plot is shown below, and is disabled for now.
added code to create violin plots for comparing inundation duration, current, and wave orbital velocity between target and reference datasets, and is disabled for now.
What needs to be done:
[ ] Cai - add median(table(data.classified$Event)) / 2 as a default for 'part' in the 'get.hydrodynamics' function
[x] Marie - populate tables in the App's front-end with calculations from the 'get.summary.statistics(), which has superseded 'get.statistics()'
[ ] Cai - combine summary, daily, event, and tidal results into one excel file to export
[ ] Cai - update get.stats() so outlier detection is working and switch to only using U-tests
[ ] Cai/Marie - how to use ggplotly in shiny
[ ] Marie - integrate the velocity stage and violin plots
[ ] Cai - add x-axis annotations to differentiate flood and ebb tide on the stage velocity plot
Details of what's changed:
FUN_HYDROLOGY.R get.hydrodynamics()
analysis functions
results tables
results text
statistics
FUN_GRAPHICS.R
What needs to be done: