search

KWB-R / wasserportal

R Package with Functions for Scraping Data of Wasserportal Berlin (https://wasserportal.berlin.de)
https://kwb-r.github.io/wasserportal/
MIT License
0 stars 0 forks source link

adapt single value query to start at 00:00 #48

Open wseis opened 3 months ago

wseis commented 3 months ago

within read_wasserportal(). When I want to get single values from yesterday, I write:

`

# Reading stations from Wasserportal
stations <- get_stations()

# Assigning crosstab variable
crosstab <- stations$crosstable

# Defining today
today <- Sys.Date()

# requesting data from Wasserportal
request <- wasserportal::read_wasserportal( station = "5803200",
                                            variables = "odf",
                                            from_date = today - 1,
                                            type = "single",
                                            stations_crosstable = crosstab)

` The resulting data start at 01:00 and increase by steps of 15 min. However, it should start at 00:00.

mrustl commented 3 months ago

Thanks @wseis for pointing this out. If you provide the parameter include_raw_time = TRUE, the columns timestamp_raw and timestamp_corr both show 00:00. As all datetime values of Wasserportal Berlin are provided in CET (see Zeitangaben https://wasserportal.berlin.de/erlaeuterungen.php), this leads to this offset during CEST season. So I don't know if filtering afterwards is sufficient (see below) or should this be solved directly within wasserportal::read_wasserportal() @hsonne ?

remotes::install_github("kwb-r/wasserportal")
#> Using github PAT from envvar GITHUB_PAT. Use `gitcreds::gitcreds_set()` and unset GITHUB_PAT in .Renviron (or elsewhere) if you want to use the more secure git credential store instead.
#> Skipping install of 'wasserportal' from a github remote, the SHA1 (40777f16) has not changed since last install.
#>   Use `force = TRUE` to force installation

# Reading stations from Wasserportal
stations <- wasserportal::get_stations()
#> Importing 10 station overviews from Wasserportal Berlin ... ok. (4.34 secs)

# Assigning crosstab variable
crosstab <- stations$crosstable

# Defining today
today <- Sys.Date()

# requesting data from Wasserportal
request <- wasserportal::read_wasserportal( station = "5803200",
                                            variables = "odf",
                                            from_date = today,
                                            type = "single",
                                            include_raw_time = TRUE,
                                            stations_crosstable = crosstab)
#> Reading 'variable_odf' for station 5803200 (station_5803200) ... ok. (1.12 secs)

knitr::kable(request[1:5,])
timestamp_raw timestamp_corr LocalDateTime UTCOffset surface_water.flow
04.06.2024 00:00 04.06.2024 00:00 2024-06-04 01:00:00 2 13.3
04.06.2024 00:15 04.06.2024 00:15 2024-06-04 01:15:00 2 13.9
04.06.2024 00:30 04.06.2024 00:30 2024-06-04 01:30:00 2 -777.0
04.06.2024 00:45 04.06.2024 00:45 2024-06-04 01:45:00 2 -777.0
04.06.2024 01:00 04.06.2024 01:00 2024-06-04 02:00:00 2 -777.0 

request_today <- wasserportal::read_wasserportal( station = "5803200",
                                            variables = "odf",
                                            from_date = today - 1,
                                            type = "single",
                                            include_raw_time = TRUE,
                                            stations_crosstable = crosstab)
#> Reading 'variable_odf' for station 5803200 (station_5803200) ... ok. (0.13 secs)

request_today <- dplyr::filter(request_today,
                LocalDateTime >= lubridate::as_datetime(today, tz = "CET"))

knitr::kable(request_today[1:5,])
timestamp_raw timestamp_corr LocalDateTime UTCOffset surface_water.flow
03.06.2024 23:00 03.06.2024 23:00 2024-06-04 00:00:00 2 14.2
03.06.2024 23:15 03.06.2024 23:15 2024-06-04 00:15:00 2 13.3
03.06.2024 23:30 03.06.2024 23:30 2024-06-04 00:30:00 2 12.8
03.06.2024 23:45 03.06.2024 23:45 2024-06-04 00:45:00 2 13.0
04.06.2024 00:00 04.06.2024 00:00 2024-06-04 01:00:00 2 13.3

Created on 2024-06-04 with reprex v2.1.0

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.3.2 (2023-10-31 ucrt) #> os Windows 10 x64 (build 19045) #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate German_Germany.utf8 #> ctype German_Germany.utf8 #> tz Europe/Berlin #> date 2024-06-04 #> pandoc 3.1.11 @ C:/Program Files/RStudio/resources/app/bin/quarto/bin/tools/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> ! package * version date (UTC) lib source #> D archive 1.1.7 2023-12-11 [1] RSPM (R 4.3.0) #> cli 3.6.2 2023-12-11 [1] RSPM (R 4.3.0) #> curl 5.2.1 2024-03-01 [1] RSPM (R 4.3.0) #> data.table 1.15.4 2024-03-30 [1] RSPM (R 4.3.0) #> digest 0.6.35 2024-03-11 [1] RSPM (R 4.3.0) #> dplyr 1.1.4 2023-11-17 [1] RSPM (R 4.3.0) #> evaluate 0.23 2023-11-01 [1] RSPM (R 4.3.0) #> fansi 1.0.6 2023-12-08 [1] RSPM (R 4.3.0) #> fastmap 1.1.1 2023-02-24 [1] RSPM (R 4.3.0) #> fs 1.6.4 2024-04-25 [1] RSPM (R 4.3.0) #> generics 0.1.3 2022-07-05 [1] RSPM (R 4.3.0) #> glue 1.7.0 2024-01-09 [1] RSPM (R 4.3.0) #> hms 1.1.3 2023-03-21 [1] RSPM (R 4.3.0) #> htmltools 0.5.8.1 2024-04-04 [1] RSPM (R 4.3.0) #> httr 1.4.7 2023-08-15 [1] RSPM (R 4.3.0) #> knitr 1.46 2024-04-06 [1] RSPM (R 4.3.0) #> kwb.datetime 0.5.0 2024-03-19 [1] Github (kwb-r/kwb.datetime@5f2b2c4) #> kwb.utils 0.15.0 2024-04-05 [1] Github (kwb-r/kwb.utils@4415aa2) #> lifecycle 1.0.4 2023-11-07 [1] RSPM (R 4.3.0) #> lubridate 1.9.3 2023-09-27 [1] RSPM (R 4.3.0) #> magrittr 2.0.3 2022-03-30 [1] RSPM (R 4.3.0) #> pillar 1.9.0 2023-03-22 [1] RSPM (R 4.3.0) #> pkgconfig 2.0.3 2019-09-22 [1] RSPM (R 4.3.0) #> purrr 1.0.2 2023-08-10 [1] RSPM (R 4.3.0) #> R6 2.5.1 2021-08-19 [1] RSPM (R 4.3.0) #> readr 2.1.5 2024-01-10 [1] RSPM (R 4.3.0) #> remotes 2.5.0 2024-03-17 [1] RSPM (R 4.3.0) #> reprex 2.1.0 2024-01-11 [1] RSPM (R 4.3.0) #> rlang 1.1.3 2024-01-10 [1] RSPM (R 4.3.0) #> rmarkdown 2.26 2024-03-05 [1] CRAN (R 4.3.3) #> rstudioapi 0.16.0 2024-03-24 [1] RSPM (R 4.3.0) #> rvest 1.0.4 2024-02-12 [1] RSPM (R 4.3.0) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.3) #> stringi 1.8.3 2023-12-11 [1] RSPM (R 4.3.0) #> stringr 1.5.1 2023-11-14 [1] RSPM (R 4.3.0) #> tibble 3.2.1 2023-03-20 [1] RSPM (R 4.3.0) #> tidyr 1.3.1 2024-01-24 [1] RSPM (R 4.3.0) #> tidyselect 1.2.1 2024-03-11 [1] RSPM (R 4.3.0) #> timechange 0.3.0 2024-01-18 [1] RSPM (R 4.3.0) #> tzdb 0.4.0 2023-05-12 [1] RSPM (R 4.3.0) #> utf8 1.2.4 2023-10-22 [1] RSPM (R 4.3.0) #> vctrs 0.6.5 2023-12-01 [1] RSPM (R 4.3.0) #> wasserportal 0.4.0 2024-06-04 [1] Github (kwb-r/wasserportal@40777f1) #> withr 3.0.0 2024-01-16 [1] RSPM (R 4.3.0) #> xfun 0.43 2024-03-25 [1] RSPM (R 4.3.0) #> xml2 1.3.6 2023-12-04 [1] RSPM (R 4.3.0) #> yaml 2.3.8 2023-12-11 [1] RSPM (R 4.3.0) #> #> [1] C:/Users/mrustl/AppData/Local/R/win-library/4.3 #> [2] C:/Program Files/R/R-4.3.2/library #> #> D ── DLL MD5 mismatch, broken installation. #> #> ────────────────────────────────────────────────────────────────────────────── ```