`process_aqs` data retention

[mitchellmanware]

@sigmafelix

When using the process_aqs function, why does the returned object (whether sf, SpatVector, or data.table) only return the data_field when mode = "sparse"?

> aqs <- process_aqs(
+   path = "./beethoven/rtorch/data/aqs/data_files/",
+   date = c("2018-01-01", "2018-01-10"),
+   mode = "full",
+   data_field = "Arithmetic.Mean",
+   return_format = "data.table"
+ )
> head(aqs)
          site_id       lon      lat       time
           <char>     <num>    <num>     <char>
1: 01073002388101 -86.81500 33.55306 2018-01-01
2: 01073100588101 -87.00361 33.33111 2018-01-01
3: 01073101088101 -86.54917 33.54528 2018-01-01
4: 01073200388101 -86.92417 33.49972 2018-01-01
5: 01073205988101 -86.84411 33.52143 2018-01-01
6: 01089001488101 -86.58816 34.68547 2018-01-01

[sigmafelix]

@mitchellmanware I added data_field argument mainly for tidying up beethoven input data. The main reason for letting data_field work in mode = "sparse" is that it is more straightforward to do that than adding logics for other modes. mode = "full" case can be simple because we can assign NA to data_field values when the data is not available in a location-date pair. However, mode = "location" is not that simple since the data are available spatiotemporally. I thought I would add a fun argument to define a summary function for this mode, but I reserved it for the simplicity of the function. I would be happy to discuss if it would be beneficial for users to support summary functions.

[mitchellmanware]

No that makes sense - thank you for the explanation. Updating process_aqs with the option to retain the measurement data is great, I can't believe it took us until now to add that functionality 😅

[sigmafelix]

@mitchellmanware Yes I was surprised as well when I started working on this update 😅. Better late than never, I think we're good to go. I will close the issue.