Load_dam Error

[SRutjens]

Hi, I am trying to load my data from the DAM system and I get the same error message as displayed by Benjamin in issue #17. This is the error I get: data = load_dam(link_metadata)

Error in find_dam_first_last_lines(path, start_datetime, stop_datetime, : No data in selected date range

This is the script I am currently using:

data_dir="C:/Users/stijn/Documents/School/RFiles/Pilot"  
setwd(data_dir)
library(damr)
library(ggetho)

metadata=fread("metadata1Lux.csv")
link_metadata=link_dam_metadata(metadata, result_dir=data_dir)
data=load_dam(link_metadata)
summary(data)

I tested the software a couple of days earlier to test the rethomics package. I am new to R and wanted to get some hands-on experience by using the data of a short pilot experiment to test the package. During this pilot the guide helped me through the different functions wonderfully, including this particular step.

However, now I am using a new data set, which was started after the pilot and uses the same activity monitors and it leads to the error shown above. I have tried the suggestion to make the metadata file in R itself by using this code:

t <- rep("Monitor1.txt", times=32)
s <- rep("Monitor2.txt", times=32)
a <- c(t,s)
metadata <- data.table(file = a,region_id = rep(1:32), machine_name=c(rep(1, times=32),rep(2, times=32)), start_datetime = rep("2019-05-03 07:00:00", times = 64), stop_datetime = rep("2019-05-07 07:00:00", times = 64), treatment = c(rep("1 Lux", times=32),rep("control", times = 32)))

But to no avail. I noticed that the new data set files start at a high count in their first column.

1396    2 May 19    16:53:50    1   0   1   0   Ct  0   1   18  9   18  17  22  20  10  36  16  17  21  9   24  21  24  23  25  20  18  22  51  19  6   18  18  16  25  22  24  20  18  29
1397    2 May 19    16:53:52    1   0   1   0   Ct  0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   1   0   0

I suspect this is due to the activity monitor retaining the count from the pilot, which ended at 1030. I was wondering if this could be the cause of the error or if there is another cause that I have missed.

Here are the data and metadata used. The pilot data works fine as far as I know but the other data set gives the error. The pilot data: metadataPilot.txt Monitor1Pilot.txt

The data which gives the error: metadata1Lux.txt Monitor1.txt Monitor2.txt

Thank you in advance!

[qgeissmann]

Hi there, Thanks for the report! This is a very curious issue... I run your code on my machine, and I get data:

t <- rep("Monitor1.txt", times=32)
s <- rep("Monitor2.txt", times=32)
a <- c(t,s)
metadata <- data.table(file = a,region_id = rep(1:32), machine_name=c(rep(1, times=32),rep(2, times=32)), start_datetime = rep("2019-05-03 07:00:00", times = 64), stop_datetime = rep("2019-05-07 07:00:00", times = 64), treatment = c(rep("1 Lux", times=32),rep("control", times = 32)))
link_metadata=link_dam_metadata(metadata, result_dir=data_dir)
data=load_dam(link_metadata)
summary(data)

Gives me:

behavr table with:
 64 individuals
 7  metavariables
 2  variables
 3.6928e+04 measurements
 1  key (id)

Can you anyone else reproduce this bug?

@SRutjens, can you try to read just one file: damr:::read_dam_file(".../Monitor1.txt"). Do you get data? If so, can you paste the metadata you get? I suspect a time zone issue...?

[brelwill]

I'm getting the same result as you @qgeissmann. However, @SRutjens, it doesn't look like you're using the raw monitor file? You should be pulling data directly from the DAMSystem3Data folder for the most accurate counts. Just a thought, I usually get this error because my stop_datetime is too far in the future. Not sure whether you're using a .csv to run this, but when I convert the file, the time 7:00:00 turns into 19:00:00 for which you do not have data for 5/7/19. Also, the first column on the left is just an arbitrary count of monitor reads and is not read in the loading step, as far as I know. But the rows you listed here are outside the range being read, as indicated by your metadata, and should not affect your results.

[SRutjens]

Hi, @qgeissmann When I run damr:::read_dam_file(".../Monitor1.txt") I get the same error: Error in find_dam_first_last_lines(path, start_datetime, stop_datetime, : No data in selected date range

@brelwill That doesn't seem to be the case, because even when the range is set from 5/3/19 to 5/4/19 the same error persists

However when I tried the same code in rstudio.cloud as opposed to normal RStudio it seems to give the behavr table with the same values:

behavr table with:
 64 individuals
 7  metavariables
 2  variables
 3.6928e+04 measurements
 1  key (id)

Therefore I assume something is wrong with my RStudio. I'll try to reinstall RStudio and see if it fixes the problem.