Aggregate MTR

[peterdesmet]

The aggregation is:

• mean() for u, v, dens and ff
• mean.circular() for dd (converted to positive degrees)

For MTR however, we need:

1. to group by height and sum()
2. then group by time and mean()

But I don't know how to group_by(radar_id, height_bin), meanwhile keeping the datetime_bin to group_by(radar_id, height_bin, datetime_bin) in the next step. @CeciliaNilsson709 could you paste the code here that you used for that?

[CeciliaNilsson709]

I don't know of a good way of doing this all in one go (unless you want to average the other variables twice which is not very nice), so you will probably have to do the MTR aggregation by itself and then join it to the other aggregated variables. Thats what I do 🤷‍♀️. Here is how I aggregate the MTR:

flyway_night %>% group_by(radar_id, date_of_sunset, datetime) %>% summarize( sum_MTR = sum(mtr, na.rm = TRUE)) %>% group_by(radar_id, date_of_sunset) %>% summarize( average_MTR = mean(sum_MTR, na.rm = TRUE))-> nightly_MTR

"datetime" is the timestamp and you should use your "datetime_bin" instead of "date_of_sunset". This aggregates over all altitudes, if you want to do your height bins you can just add it after the "date_of_sunset/datetime_bin" in the first group_by() and then before "date_of_sunset/datetime_bin" in the second group_by(). Does that make sense?

[adokter]

@peterdesmet bioRad has the function vintegrate to calculate MTRs properly, see https://github.com/adokter/bioRad/blob/dadf73dadac9d2650724811e71c5175209afb382/R/bioRad.R#L603-L670. When integrating vp-processing into bioRad we really have to think carefully whether such aggregations need to be done on the text output generated by vp-processing. You have to be careful not to start bypassing a lot of the object-based functionality of bioRad, which I think is a pity. Maybe set up a skype meeting on this somewhere soon?

[adokter]

and this function calculates mtr's (by calling vintegrate) https://github.com/adokter/bioRad/blob/dadf73dadac9d2650724811e71c5175209afb382/R/bioRad.R#L603-L670

[peterdesmet]

When integrating vp-processing into bioRad we really have to think carefully whether such aggregations need to be done on the text output generated by vp-processing.

Yeah, rather than trying to solve this specific MTR aggregation issue, I'd like to rethink the whole vp-processing a bit, so all useful functionality is part of bioRad, but we still have a documented pipeline as .Rmd files (which has proofed useful). Basically a pipeline with the vignettes and vp-processing combined. I especially want to keep some resting points in that pipeline (i.e. a specific data output format) as that makes it easier to collaborate: i.e. I don't have to run the whole pipeline from vp -> csv to start aggregating data for visualizations: I can just use the csv). But we do run into odd solutions for e.g. MTR that bioRad would do better, so need to think about this further.

Won't have time for a Skype meeting soon, as I'm leaving for a conference in less than a week (to Ottawa: temptingly close to Cornell... in North American distances at least :smile:), but let's discuss this further when we can (via separate issue or email).