hugomflavio / actel

Standardised analysis of acoustic telemetry data from fish moving through receiver arrays
https://hugomflavio.github.io/actel-website
26 stars 5 forks source link

Error: Not all values in the 'Release.date' column appear to be in a 'yyyy-mm-dd hh:mm' format (seconds are optional). Please double-check the biometrics. #94

Closed SimonDedman closed 9 months ago

SimonDedman commented 9 months ago

Hi mate, sorry to bother you with this since it feels likely that it'll be something dumb on my end, yet I can't find it.

Data: FWCdepredationStudy-Detections.csv spatial.csv deployments.csv biometrics.csv

Load & run code:

loadloc <- "/wherever"
biometrics <- readr::read_csv(file.path(loadloc, "biometrics.csv")) |> 
  dplyr::mutate(Sex = as.character("Female"))
spatial <- readr::read_csv(file.path(loadloc, "spatial.csv"))
deployments <- readr::read_csv(file.path(loadloc, "deployments.csv"))
detections <- readr::read_csv(file.path(loadloc, "FWCdepredationStudy-Detections.csv"))
class(biometrics$Release.date)
mypreload <- actel::preload(biometrics = biometrics,
                            spatial = spatial,
                            deployments = deployments,
                            detections = detections,
                            tz = "UTC")

Error: Not all values in the 'Release.date' column appear to be in a 'yyyy-mm-dd hh:mm' format (seconds are optional). Please double-check the biometrics.

In LibreOffice (Excel) and RStudio, biometrics dates look to be fine, all in yyyy-mm-dd hh:mm:ss format. It says seconds are optional: does option mean prohibited? Anything else look kooky to you? Thanks bud!

hugomflavio commented 9 months ago

Hey Simon!

Short version: The "00:00:00" on some timestamps is breaking it. But your biometrics file has other issues that will pop up later (see end of post).

Long version: This is an annoying pitfall of R... if the timestamp ends in 00:00:00, then R drops the time portion when converting the timestamp to a string. The underlying function that checks the biometrics input uses grepl to check for a timestamp format (because it was originally built to be checking a csv input). grepl forces the POSIX back to a character, with R "helpfully" shaving off the "00:00:00" part of it and breaking the check.

It would make sense for the preload route to check first if the timestamps are POSIX already, instead of trying to run all the checks blindly. I'll code that in. If you want to get it working right away, you can just change those stamps to e.g. 00:00:01 and it will bypass this issue.

This being said, you will then run into other issues. The first one that popped up was that there is no Signal column in the biometrics. Have a look here for the input requirements for actel: https://hugomflavio.github.io/actel-website/manual-data.html#biometrics-file

SimonDedman commented 9 months ago

Thanks bud, the 01 hack solved it.

Yep, Signal missing: to solve. I've not been given any info about this dataset and am groping around blindly which is somewhat a good exercise in forcing me to learn, but also inefficient.

Cheers matey, hope you're well!