search

geco-bern / FluxDataKit

LEMONTREE flux data kit
https://geco-bern.github.io/FluxDataKit
2 stars 11 forks source link

fdk_convert_lsm() fails with latest R version #58

Closed jaideep777 closed 11 months ago

jaideep777 commented 1 year ago

Probably because of changes in as.POSIXct(), the fdk_convert_lsm() function fails on R version 4.3.1.

The problem is in this line: dplyr::left_join(df[[1]], df[[2]]) it returns a very large df with this warning:

Detected an unexpected many-to-many relationship between `x` and `y`.
ℹ Row 1 of `x` matches multiple rows in `y`.
ℹ Row 1 of `y` matches multiple rows in `x`.
ℹ If a many-to-many relationship is expected, set `relationship = "many-to-many"`
  to silence this warning.

Possible fix:

Change lines 68 and 69 as follows:

line 68:    time_date <- as.POSIXct(time, origin = time_units, tz="GMT")
line 69:    # time_date <- as.POSIXct(time_date, tz = "GMT")
stineb commented 1 year ago

Can it be fixed by adding the by = "..." argument?

jaideep777 commented 1 year ago

I dont think so, because the column names are fine and matched correctly, but the values are not consistent across df1 and df2 because the previously computed time_date vector discards hhmm. The suggested fix basically includes hhmm in the time_date, so the joining entries have a 1-1 relationship.

stineb commented 12 months ago

This may be related. I get weird results when using the function fdk_convert_lsm() for a single site. The data frame all, which gets written into the file seems excessively large. When debugging:

Browse[2]> dim(all)
[1] 12623616       60

20 years of half-hourly data should generate about 350400 rows, not 12623616 row.