Join data from different localities

[Jules-]

I am using the myClim package to analyze the output of the 50 TOMST sensors I have installed in my thesis trial. With the documentation you provide I have been able to preliminary explore and analyze some data but I am stucked tryig to resolve some issues with the function join_mc(). The problem arises when I try to merge using this function the three time periods I have for each TOMST sensor. I have 50 files named 1A, 2A, ... 50A in three different directories, that I read using the mc_read_files into three different sets I name XXX, one for each time period. Then, when I try to join all of them, the output of the mc_join reads the same sensor (with different time periods) as three different sensors ( I attach a screenshot of a test with the sensor that I called 1A).

I am contacting you to see how I could solve this problem with the code, because I don't know how to solve it. It is an important part of my thesis to have a continuity of the time periods in the same set, with the later aim of plotting the same sensor in two periods at the same time.

I attach the sensor files for the 1A sensor as an example in the three time periods and the R code lines.

# prepare working environment
round_1 <- setwd("..../19 En-01 Jun 2023")
round_2 <- setwd("....../16 Sept2022-01 Dic 2022")
# I try with an example called "prueba" that I made only with the sensor 1A in a new folder
prueba <- setwd(".../Prueba")

# read all Tomst files from current directory without metadata
test <- mc_read_files(prueba, dataformat_name = "TOMST", 
                         recursive = F, 
                         silent = T,
                         clean = T)

# cleaning time series
test_tms <- mc_prep_clean(test)

join <- mc_join(data = test_tms, comp_sensors = c("TMS_T1", "TMS_T3"))

[Jules-]

Answer

The mc_join function joins loggers from the same locality together. The problem with your code is that you are using the mc_read_files function to read data, and each file is in its own locality. If you want to read multiple files into one locality, you must use the mc_read_data function. This function has a parameter called files_table, which specifies which file belongs to which locality. For example, you could have the files 1A.csv, 1A_2.csv, and 1A_3.csv in one locality called 1A. The files_table parameter can be a data.frame or a path to a CSV file with metadata.

library(stringr)
library(myClim)
path <- list.files(path = ".", pattern = "*.csv")
# path <- c("1A_2.csv", "1A_3.csv", "1A.csv")
files_table <- data.frame(path=path)
# extract locality_id with regular expression
files_table$locality_id <- str_extract(path, "([^_]+)_?.*\\.csv", group=1)
files_table$data_format <- "TOMST"
test <- mc_read_data(files_table)
join <- mc_join(test)

There is a small problem in your code with cleaning. If you have clean = TRUE in mc_read_*(), do not call mc_prep_clean(), because it removes information about duplicate and missing records.