Updating mWater pull and merging with old field notes

[SamStruthers]

In an effort to better track sensor malfunctions in livestreaming data, I added a sensor malfunction section to our field notes. I/techs/others can submit a survey if they see data that looks erroneous which may not be caught by QAQC (drift, broken sensor, etc). By sourcing 'grab_malfunction_record.R', you get malfunction_records which has start/end_dt, sensor affected and notes, this can then be incorporated into the function kw_known_issues with some kind of map function in the future.

I also tried to match all the columns in sensor_notesand have looked at most data to make sure this was done correctly, have not yet run on entire script due to error with api pull (issue: #86 ). Once this is fixed, I can see if the field notes still work correctly downstream in pipeline.

Review requested:

• Thoughts on how old and new field notes are merged and any possible introduced errors
• Folder structure of mWater_pull_collate, some scripts will not be used in QAQC (spreadsheet, files_missing and download_pics) but I will use them on regular basis.
• Clarity/conciseness of clean_mwater_notes.R andgrab_sensor_notes.R code

[SamStruthers]

Fixed minor errors in the data manager tools scripts. Download pictures was mainly due to user error, solved mwater survey to reduce these errors in the future

[juandlt-csu]

This PR looks good to merge to me. BTW I and Katie have been working in the targets branch of this repository as we transition to targets-ifying the pipeline. That branch has the most up to date functions. Once this merges to main we will have to sync the targets branch up to this so that your changes are reflected there.

• Thoughts on field notes merging: I think what you came up with should be. It would be helpful to me if we met so that you could explain the joining process you are envisioning to me. Right now it is hard for me to visualize plugging this into the targets pipeline. This is mainly due to my lack of exposure to working with the mWater data though.
• Folder structure thoughts: I think that for now just putting functions in their own folder separate from scripts would be useful for integrating this stuff into the targets pipeline. I think that we will have to reorg this repo in the future so that it makes more sense when integrated with the FC system anyway, so don't worry about it too much right now I guess.
• Conciseness/clarity: I think that the scripts you specified are clear enough. Again my confusion is from lack of exposure to this workflow. I will say that I am unsure about sourcing clean_mwater_notes.R in all of the scripts, but I also just think that I don't see the vision like you do.

[SamStruthers]

Met with Juan about incorportating mWater notes into targets pipeline. For data push to FC system, I will add to the targets script to pull/clean mWater notes similarly to old field notes. One data is pushed to FC and we are trying to get the system running on a loop (2024) we will remove the pull in of old field notes and the mWater scripts will take their place in the Targets code