search

bhelsel / agcounts

R package for extracting actigraphy counts from accelerometer data.
Other
7 stars 5 forks source link

Mention rounding #24

Open muschellij2 opened 10 months ago

muschellij2 commented 10 months ago

As per the GT3X documentation, rounding half away from zero is the norm for rounding GT3X data as the last step (https://github.com/actigraph/GT3X-File-Format/blob/d4aa795a3d06cc8b22b4e3df645b597cdeee6d92/LogRecords/Activity.md?plain=1#L226). It should be noted somewhere that the normal round in R does not do this, but the ncar::Round function will (as well as many others).

This is important because you can get some slight differences from the readers:

library(agcounts)
library(ncar)
#> Loading required package: rtf
#> Loading required package: NonCompart
path = system.file("extdata/example.gt3x", package = "agcounts")

xyz = c("X", "Y", "Z")
x <- 
  agread(path, parser = "pygt3x")
y <-
  agread(path, parser = "read.gt3x")
dim(x)
#> [1] 18000     4
dim(y)
#> [1] 18000     4
x[165:170,]
#>                    time           X          Y        Z
#> 165 2023-06-13 08:34:01 -0.01171875 -0.1367188 1.027344
#> 166 2023-06-13 08:34:01 -0.02734375 -0.1562500 1.117188
#> 167 2023-06-13 08:34:01 -0.08593750 -0.1914062 1.117188
#> 168 2023-06-13 08:34:01 -0.09765625 -0.1835938 1.140625
#> 169 2023-06-13 08:34:01 -0.06250000 -0.1445312 1.097656
#> 170 2023-06-13 08:34:01 -0.03125000 -0.1171875 1.003906
y[165:170,]
#> Sampling Rate: 100Hz
#> Firmware Version: 1.7.2
#> Serial Number Prefix: TAS
#>                    time      X      Y     Z
#> 165 2023-06-13 08:34:01 -0.012 -0.137 1.027
#> 166 2023-06-13 08:34:01 -0.027 -0.156 1.117
#> 167 2023-06-13 08:34:01 -0.086 -0.191 1.117
#> 168 2023-06-13 08:34:01 -0.098 -0.184 1.141
#> 169 2023-06-13 08:34:01 -0.063 -0.145 1.098
#> 170 2023-06-13 08:34:01 -0.031 -0.117 1.004

max(abs(y[, xyz] - round(x[, xyz], 3)))
#> [1] 0.001
max(abs(y[, xyz] - ncar::Round(x[, xyz], 3)))
#> [1] 0

Created on 2023-09-05 with reprex v2.0.2

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.2.1 (2022-06-23) #> os macOS Big Sur ... 10.16 #> system x86_64, darwin17.0 #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz America/New_York #> date 2023-09-05 #> pandoc 3.1.5 @ /usr/local/bin/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> agcounts * 0.6.4 2023-07-26 [1] local #> bit 4.0.5 2022-11-15 [1] CRAN (R 4.2.0) #> bit64 4.0.5 2020-08-30 [1] CRAN (R 4.2.0) #> blob 1.2.4 2023-03-17 [1] CRAN (R 4.2.0) #> bslib 0.5.0 2023-06-09 [1] CRAN (R 4.2.0) #> cachem 1.0.8 2023-05-01 [1] CRAN (R 4.2.0) #> cli 3.6.1 2023-03-23 [1] CRAN (R 4.2.0) #> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.2.0) #> data.table 1.14.9 2023-08-21 [1] local #> DBI 1.1.3 2022-06-18 [1] CRAN (R 4.2.0) #> digest 0.6.33 2023-07-07 [1] CRAN (R 4.2.0) #> dplyr 1.1.2 2023-04-20 [1] CRAN (R 4.2.0) #> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.2.0) #> evaluate 0.21 2023-05-05 [1] CRAN (R 4.2.0) #> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.2.0) #> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.2.0) #> fs 1.6.3 2023-07-20 [1] CRAN (R 4.2.0) #> generics 0.1.3 2022-07-05 [1] CRAN (R 4.2.0) #> GGIR 2.9-0 2023-03-24 [1] CRAN (R 4.2.0) #> ggplot2 3.4.2 2023-04-03 [1] CRAN (R 4.2.0) #> glue 1.6.2 2022-02-24 [1] CRAN (R 4.2.0) #> gsignal 0.3-5 2022-05-15 [1] CRAN (R 4.2.0) #> gtable 0.3.3 2023-03-21 [1] CRAN (R 4.2.0) #> htmltools 0.5.5 2023-03-23 [1] CRAN (R 4.2.0) #> htmlwidgets 1.6.2 2023-03-17 [1] CRAN (R 4.2.0) #> httpuv 1.6.11 2023-05-11 [1] CRAN (R 4.2.0) #> jquerylib 0.1.4 2021-04-26 [1] CRAN (R 4.2.0) #> jsonlite 1.8.7 2023-06-29 [1] CRAN (R 4.2.0) #> knitr 1.43 2023-05-25 [1] CRAN (R 4.2.0) #> later 1.3.1 2023-05-02 [1] CRAN (R 4.2.0) #> lattice 0.21-8 2023-04-05 [1] CRAN (R 4.2.0) #> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.2.0) #> lubridate 1.9.2 2023-02-10 [1] CRAN (R 4.2.0) #> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.2.0) #> Matrix 1.6-0 2023-07-08 [1] CRAN (R 4.2.1) #> memoise 2.0.1 2021-11-26 [1] CRAN (R 4.2.0) #> mime 0.12 2021-09-28 [1] CRAN (R 4.2.0) #> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.2.0) #> ncar * 0.4.5 2022-02-23 [1] CRAN (R 4.2.0) #> NonCompart * 0.6.0 2022-07-16 [1] CRAN (R 4.2.0) #> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.2.0) #> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.2.0) #> png 0.1-8 2022-11-29 [1] CRAN (R 4.2.0) #> promises 1.2.0.1 2021-02-11 [1] CRAN (R 4.2.0) #> purrr 1.0.1 2023-01-10 [1] CRAN (R 4.2.0) #> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.2.0) #> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.2.0) #> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.2.0) #> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.2.0) #> R6 2.5.1 2021-08-19 [1] CRAN (R 4.2.0) #> Rcpp 1.0.11 2023-07-06 [1] CRAN (R 4.2.0) #> reactable 0.4.4 2023-03-12 [1] CRAN (R 4.2.0) #> read.gt3x 1.2.0 2022-06-30 [1] CRAN (R 4.2.0) #> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.2.0) #> reticulate 1.30 2023-06-09 [1] CRAN (R 4.2.0) #> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.2.0) #> rmarkdown 2.23 2023-07-01 [1] CRAN (R 4.2.0) #> RSQLite 2.3.1 2023-04-03 [1] CRAN (R 4.2.0) #> rstudioapi 0.15.0 2023-07-07 [1] CRAN (R 4.2.0) #> rtf * 0.4-14.1 2020-03-22 [1] CRAN (R 4.2.0) #> sass 0.4.7 2023-07-15 [1] CRAN (R 4.2.0) #> scales 1.2.1 2022-08-20 [1] CRAN (R 4.2.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.0) #> shiny 1.7.4.1 2023-07-06 [1] CRAN (R 4.2.0) #> stringi 1.7.12 2023-01-11 [1] CRAN (R 4.2.0) #> stringr 1.5.0 2022-12-02 [1] CRAN (R 4.2.0) #> styler 1.10.1 2023-06-05 [1] CRAN (R 4.2.0) #> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.2.0) #> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.2.0) #> timechange 0.2.0 2023-01-11 [1] CRAN (R 4.2.0) #> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.2.0) #> vctrs 0.6.3 2023-06-14 [1] CRAN (R 4.2.0) #> withr 2.5.0 2022-03-03 [1] CRAN (R 4.2.0) #> xfun 0.39 2023-04-20 [1] CRAN (R 4.2.0) #> xtable 1.8-4 2019-04-21 [1] CRAN (R 4.2.0) #> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.2.0) #> zoo 1.8-12 2023-04-13 [1] CRAN (R 4.2.0) #> #> [1] /Library/Frameworks/R.framework/Versions/4.2/Resources/library #> #> ─ Python configuration ─────────────────────────────────────────────────────── #> python: /Users/johnmuschelli/miniconda3/bin/python3 #> libpython: /Users/johnmuschelli/miniconda3/lib/libpython3.11.dylib #> pythonhome: /Users/johnmuschelli/miniconda3:/Users/johnmuschelli/miniconda3 #> version: 3.11.4 (main, Jul 5 2023, 08:41:25) [Clang 14.0.6 ] #> numpy: /Users/johnmuschelli/miniconda3/lib/python3.11/site-packages/numpy #> numpy_version: 1.23.5 #> pygt3x: /Users/johnmuschelli/miniconda3/lib/python3.11/site-packages/pygt3x #> #> NOTE: Python version was forced by RETICULATE_PYTHON #> #> ────────────────────────────────────────────────────────────────────────────── ```
bhelsel commented 9 months ago

@muschellij2 Thanks for mentioning this difference in the read.gt3x and pygt3x packages. It seems like this might be something that needs to be fixed with the read.gt3x R package since it is returning rounded decimal points. Any thoughts as to what part of the code is doing the rounding or any potential solutions?

muschellij2 commented 9 months ago

I think the results should be rounded to 3 decimals to replicate the output from actilife.

On Tue, Oct 3, 2023 at 5:51 PM Brian C. Helsel @.***> wrote:

@muschellij2 https://github.com/muschellij2 Thanks for mentioning this difference in the read.gt3x and pygt3x packages. It seems like this might be something that needs to be fixed with the read.gt3x R package since it is returning rounded decimal points. Any thoughts as to what part of the code is doing the rounding or any potential solutions?

— Reply to this email directly, view it on GitHub https://github.com/bhelsel/agcounts/issues/24#issuecomment-1745783447, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIGPLQ25SSZJOUQHCKAFYLX5SCH5AVCNFSM6AAAAAA4LXTPVGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBVG44DGNBUG4 . You are receiving this because you were mentioned.Message ID: @.***>

bhelsel commented 9 months ago

From what I've found, the pygt3x reader without rounding offers a closer estimate to the Actilife output. I'm not certain that ActiGraph rounds the raw data before calculating the activity counts. It seems like the rounding in read.gt3x mimics the Ncar::Round function.

library(agcounts)
library(dplyr)
#> 
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#> 
#>     filter, lag
#> The following objects are masked from 'package:base':
#> 
#>     intersect, setdiff, setequal, union

path <- system.file("extdata/example.gt3x", package = "agcounts")
agd_path <- system.file("extdata/example5sec.agd", package = "agcounts")

agd <- agcounts:::.read_agd(agd_path)$data

pygt3x <-
  agcounts::agread(path, parser = "pygt3x") %>%
  agcounts::calculate_counts(epoch = 5)

pygt3x_round <-
  agcounts::agread(path, parser = "pygt3x") %>%
  dplyr::mutate_at(.vars = 2:4, .funs = round, 3) %>%
  agcounts::calculate_counts(epoch = 5)

pygt3x_ncar_round <-
  agcounts::agread(path, parser = "pygt3x") %>%
  dplyr::mutate_at(.vars = 2:4, .funs = ncar::Round, 3) %>%
  agcounts::calculate_counts(epoch = 5)

colSums(abs(pygt3x[, 2:5] - agd[, 2:5]))
#>            Axis1            Axis2            Axis3 Vector.Magnitude 
#>                0                0                0                0
colSums(abs(pygt3x_round[, 2:5] - agd[, 2:5]))
#>            Axis1            Axis2            Axis3 Vector.Magnitude 
#>                7                4               13                9
colSums(abs(pygt3x_ncar_round[, 2:5] - agd[, 2:5]))
#>            Axis1            Axis2            Axis3 Vector.Magnitude 
#>                7                4               16               11

read.gt3x <- 
  agcounts::agread(path, parser = "read.gt3x") %>%
  agcounts::calculate_counts(epoch = 5)

colSums(abs(pygt3x_ncar_round[, 2:5] - read.gt3x[, 2:5]))
#>            Axis1            Axis2            Axis3 Vector.Magnitude 
#>                0                0                0                0

Created on 2023-10-04 with reprex v2.0.2

muschellij2 commented 9 months ago

Actigraph specification of GT3X format specifically says to round ACTIVITY packets: https://github.com/actigraph/GT3X-File-Format/blob/main/LogRecords/Activity2.md#scaling-activity-values and https://github.com/actigraph/GT3X-File-Format/blob/main/LogRecords/Activity.md#scaling-activity-values

bhelsel commented 9 months ago

The documentation doesn't change the fact that the pygt3x parser is getting a more consistent output compared to Actilife and is the method that ActiGraph recommended to read in the .gt3x raw files when their agcounts Python package was released. If users want a value rounded to 3 decimal points then it seems like the read.gt3x parser would be the best option since it rounds similar to Ncar::Round. Users could also round the values after reading them into R and before calculating the activity counts which yields the same output as the read.gt3x parser.

muschellij2 commented 9 months ago

That’s fair, but many will likely use base::round and get slightly wrong results. Either way, is this noted in the docs?

On Wed, Oct 4, 2023 at 9:50 AM Brian C. Helsel @.***> wrote:

The documentation doesn't change the fact that the pygt3x parser is getting a more consistent output compared to Actilife and is the method that ActiGraph recommended to read in the .gt3x raw files when their agcounts Python package was released. If users want a value rounded to 3 decimal points then it seems like the read.gt3x parser would be the best option since it rounds similar to Ncar::Round. Users could also round the values after reading them into R and before calculating the activity counts which yields the same output as the read.gt3x parser.

— Reply to this email directly, view it on GitHub https://github.com/bhelsel/agcounts/issues/24#issuecomment-1746921244, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAIGPLSM27NGGNLRXPGS4KDX5VSSPAVCNFSM6AAAAAA4LXTPVGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBWHEZDCMRUGQ . You are receiving this because you were mentioned.Message ID: @.***>

bhelsel commented 9 months ago

We list the preferred method as pygt3x file in the README.md file, but didn't say anything about the rounding problems. Do you want to propose something in the README.md and/or the agread function documentation? Otherwise, I can close this issue once I have a chance to update the documentation (probably within a few weeks).