ropensci / GSODR

API Client for Global Surface Summary of the Day (GSOD) Weather Data Client in R
https://docs.ropensci.org/GSODR
Other
90 stars 15 forks source link

Hail flag no working #84

Closed dvictori closed 3 years ago

dvictori commented 3 years ago

I'd like to evaluate hail frequency so I'm interested in the I_HAIL flag. But it's always returning NA. Bellow, an exemple:

# check if Hail flag is working

library(data.table)
library(GSODR)

# first get station data using GSODR
stn <- '310540-99999'

data_gsodr <- get_GSOD(2001, stn)

# do we have hail? NO! All NA values
unique(data_gsodr$I_HAIL)
#> [1] NA

# now get data from server
data_server <- fread('https://www.ncei.noaa.gov/data/global-summary-of-the-day/access/2001/31054099999.csv')

# Hail is the third flag in FRSHTT. Thus anything that has xxx1xx is hail
# There are 3 days with hail obs for this station / year
hail <- as.integer(data_server$FRSHTT / 100) %% 10
data_server[hail == 1,]
#>        STATION       DATE LATITUDE LONGITUDE ELEVATION           NAME   TEMP
#> 1: 31054099999 2001-04-11 59.18333    135.15       212 UST JUDOMA, RS   19.5
#> 2: 31054099999 2001-06-03 59.18333    135.15       212 UST JUDOMA, RS   46.7
#> 3: 31054099999 2001-07-29 59.18333    135.15       212 UST JUDOMA, RS   74.0
#>    TEMP_ATTRIBUTES   DEWP DEWP_ATTRIBUTES    SLP SLP_ATTRIBUTES   STP
#> 1:               7    9.9               7 1007.7              7 999.9
#> 2:               7   44.1               6  998.3              7 999.9
#> 3:               8   56.2               7 1010.3              6 999.9
#>    STP_ATTRIBUTES VISIB VISIB_ATTRIBUTES  WDSP WDSP_ATTRIBUTES MXSPD  GUST
#> 1:              0  24.3                7   0.8               7   3.9 999.9
#> 2:              0  16.0                7   0.6               7   3.9 999.9
#> 3:              0  31.1                8   0.2               8   1.9 999.9
#>       MAX MAX_ATTRIBUTES    MIN MIN_ATTRIBUTES  PRCP PRCP_ATTRIBUTES  SNDP
#> 1:   36.5                   2.7              *  0.01               E 999.9
#> 2:   55.9                  36.5              *  0.43               F 999.9
#> 3:   92.3                  57.0                99.99                 999.9
#>    FRSHTT
#> 1:  11100
#> 2: 110100
#> 3:  10100

# Data for the same days has no hail information. No hail flag is present
data_gsodr[hail == 1,]
#>           STNID        NAME CTRY COUNTRY_NAME ISO2C ISO3C STATE LATITUDE
#> 1: 310540-99999 UST'-JUDOMA   RS       RUSSIA    RU   RUS         59.183
#> 2: 310540-99999 UST'-JUDOMA   RS       RUSSIA    RU   RUS         59.183
#> 3: 310540-99999 UST'-JUDOMA   RS       RUSSIA    RU   RUS         59.183
#>    LONGITUDE ELEVATION    BEGIN      END   YEARMODA YEAR MONTH DAY YDAY TEMP
#> 1:    135.15       212 19590101 20210116 2001-04-11 2001     4  11  101 -6.9
#> 2:    135.15       212 19590101 20210116 2001-06-03 2001     6   3  154  8.2
#> 3:    135.15       212 19590101 20210116 2001-07-29 2001     7  29  210 23.3
#>    TEMP_ATTRIBUTES  DEWP DEWP_ATTRIBUTES    SLP SLP_ATTRIBUTES STP
#> 1:               7 -12.3               7 1007.7              7  NA
#> 2:               7   6.7               6  998.3              7  NA
#> 3:               8  13.4               7 1010.3              6  NA
#>    STP_ATTRIBUTES VISIB VISIB_ATTRIBUTES WDSP WDSP_ATTRIBUTES MXSPD GUST  MAX
#> 1:              0  39.1                7  0.4               7     2   NA  2.5
#> 2:              0  25.7                7  0.3               7     2   NA 13.3
#> 3:              0  50.1                8  0.1               8     1   NA 33.5
#>    MAX_ATTRIBUTES   MIN MIN_ATTRIBUTES PRCP PRCP_ATTRIBUTES SNDP I_FOG
#> 1:           <NA> -16.3              *  0.3               E   NA    NA
#> 2:           <NA>   2.5              * 10.9               F   NA    NA
#> 3:           <NA>  13.9           <NA>   NA            <NA>   NA    NA
#>    I_RAIN_DRIZZLE I_SNOW_ICE I_HAIL I_THUNDER I_TORNADO_FUNNEL  EA  ES   RH
#> 1:             NA         NA     NA        NA               NA 0.2 0.4 65.4
#> 2:             NA         NA     NA        NA               NA 1.0 1.1 90.3
#> 3:             NA         NA     NA        NA               NA 1.5 2.9 53.7

sessionInfo()
#> R version 4.1.0 (2021-05-18)
#> Platform: x86_64-pc-linux-gnu (64-bit)
#> Running under: Ubuntu 20.04.2 LTS
#> 
#> Matrix products: default
#> BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
#> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3
#> 
#> locale:
#>  [1] LC_CTYPE=pt_BR.UTF-8       LC_NUMERIC=C              
#>  [3] LC_TIME=pt_BR.UTF-8        LC_COLLATE=en_US.UTF-8    
#>  [5] LC_MONETARY=pt_BR.UTF-8    LC_MESSAGES=en_US.UTF-8   
#>  [7] LC_PAPER=pt_BR.UTF-8       LC_NAME=C                 
#>  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
#> [11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C       
#> 
#> attached base packages:
#> [1] stats     graphics  grDevices utils     datasets  methods   base     
#> 
#> other attached packages:
#> [1] GSODR_3.1.0       data.table_1.14.0
#> 
#> loaded via a namespace (and not attached):
#>  [1] digest_0.6.27     withr_2.4.2       R6_2.5.0          magrittr_2.0.1   
#>  [5] reprex_2.0.0      evaluate_0.14     httr_1.4.2        highr_0.9        
#>  [9] stringi_1.6.2     rlang_0.4.11      cli_2.5.0         curl_4.3.1       
#> [13] rstudioapi_0.13   fs_1.5.0          rmarkdown_2.8     tools_4.1.0      
#> [17] bit64_4.0.5       stringr_1.4.0     glue_1.4.2        bit_4.0.4        
#> [21] xfun_0.23         yaml_2.2.1        compiler_4.1.0    htmltools_0.5.1.1
#> [25] knitr_1.33

Created on 2021-06-22 by the reprex package (v2.0.0)

adamhsparks commented 3 years ago

Thanks, I’ll check on this.

On 23 Jun 2021, at 03:26, dvictori @.***> wrote:

 I'd like to evaluate hail frequency so I'm interested in the I_HAIL flag. But it's always returning NA. Bellow, an exemple:

check if Hail flag is working

library(data.table) library(GSODR)

first get station data using GSODR

stn <- '310540-99999'

data_gsodr <- get_GSOD(2001, stn)

do we have hail? NO! All NA values

unique(data_gsodr$I_HAIL)

> [1] NA

now get data from server

data_server <- fread('https://www.ncei.noaa.gov/data/global-summary-of-the-day/access/2001/31054099999.csv')

Hail is the third flag in FRSHTT. Thus anything that has xxx1xx is hail

There are 3 days with hail obs for this station / year

hail <- as.integer(data_server$FRSHTT / 100) %% 10 data_server[hail == 1,]

> STATION DATE LATITUDE LONGITUDE ELEVATION NAME TEMP

> 1: 31054099999 2001-04-11 59.18333 135.15 212 UST JUDOMA, RS 19.5

> 2: 31054099999 2001-06-03 59.18333 135.15 212 UST JUDOMA, RS 46.7

> 3: 31054099999 2001-07-29 59.18333 135.15 212 UST JUDOMA, RS 74.0

> TEMP_ATTRIBUTES DEWP DEWP_ATTRIBUTES SLP SLP_ATTRIBUTES STP

> 1: 7 9.9 7 1007.7 7 999.9

> 2: 7 44.1 6 998.3 7 999.9

> 3: 8 56.2 7 1010.3 6 999.9

> STP_ATTRIBUTES VISIB VISIB_ATTRIBUTES WDSP WDSP_ATTRIBUTES MXSPD GUST

> 1: 0 24.3 7 0.8 7 3.9 999.9

> 2: 0 16.0 7 0.6 7 3.9 999.9

> 3: 0 31.1 8 0.2 8 1.9 999.9

> MAX MAX_ATTRIBUTES MIN MIN_ATTRIBUTES PRCP PRCP_ATTRIBUTES SNDP

> 1: 36.5 2.7 * 0.01 E 999.9

> 2: 55.9 36.5 * 0.43 F 999.9

> 3: 92.3 57.0 99.99 999.9

> FRSHTT

> 1: 11100

> 2: 110100

> 3: 10100

Data for the same days has no hail information. No hail flag is present

data_gsodr[hail == 1,]

> STNID NAME CTRY COUNTRY_NAME ISO2C ISO3C STATE LATITUDE

> 1: 310540-99999 UST'-JUDOMA RS RUSSIA RU RUS 59.183

> 2: 310540-99999 UST'-JUDOMA RS RUSSIA RU RUS 59.183

> 3: 310540-99999 UST'-JUDOMA RS RUSSIA RU RUS 59.183

> LONGITUDE ELEVATION BEGIN END YEARMODA YEAR MONTH DAY YDAY TEMP

> 1: 135.15 212 19590101 20210116 2001-04-11 2001 4 11 101 -6.9

> 2: 135.15 212 19590101 20210116 2001-06-03 2001 6 3 154 8.2

> 3: 135.15 212 19590101 20210116 2001-07-29 2001 7 29 210 23.3

> TEMP_ATTRIBUTES DEWP DEWP_ATTRIBUTES SLP SLP_ATTRIBUTES STP

> 1: 7 -12.3 7 1007.7 7 NA

> 2: 7 6.7 6 998.3 7 NA

> 3: 8 13.4 7 1010.3 6 NA

> STP_ATTRIBUTES VISIB VISIB_ATTRIBUTES WDSP WDSP_ATTRIBUTES MXSPD GUST MAX

> 1: 0 39.1 7 0.4 7 2 NA 2.5

> 2: 0 25.7 7 0.3 7 2 NA 13.3

> 3: 0 50.1 8 0.1 8 1 NA 33.5

> MAX_ATTRIBUTES MIN MIN_ATTRIBUTES PRCP PRCP_ATTRIBUTES SNDP I_FOG

> 1: -16.3 * 0.3 E NA NA

> 2: 2.5 * 10.9 F NA NA

> 3: 13.9 NA NA NA

> I_RAIN_DRIZZLE I_SNOW_ICE I_HAIL I_THUNDER I_TORNADO_FUNNEL EA ES RH

> 1: NA NA NA NA NA 0.2 0.4 65.4

> 2: NA NA NA NA NA 1.0 1.1 90.3

> 3: NA NA NA NA NA 1.5 2.9 53.7

sessionInfo()

> R version 4.1.0 (2021-05-18)

> Platform: x86_64-pc-linux-gnu (64-bit)

> Running under: Ubuntu 20.04.2 LTS

>

> Matrix products: default

> BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3

> LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/liblapack.so.3

>

> locale:

> [1] LC_CTYPE=pt_BR.UTF-8 LC_NUMERIC=C

> [3] LC_TIME=pt_BR.UTF-8 LC_COLLATE=en_US.UTF-8

> [5] LC_MONETARY=pt_BR.UTF-8 LC_MESSAGES=en_US.UTF-8

> [7] LC_PAPER=pt_BR.UTF-8 LC_NAME=C

> [9] LC_ADDRESS=C LC_TELEPHONE=C

> [11] LC_MEASUREMENT=pt_BR.UTF-8 LC_IDENTIFICATION=C

>

> attached base packages:

> [1] stats graphics grDevices utils datasets methods base

>

> other attached packages:

> [1] GSODR_3.1.0 data.table_1.14.0

>

> loaded via a namespace (and not attached):

> [1] digest_0.6.27 withr_2.4.2 R6_2.5.0 magrittr_2.0.1

> [5] reprex_2.0.0 evaluate_0.14 httr_1.4.2 highr_0.9

> [9] stringi_1.6.2 rlang_0.4.11 cli_2.5.0 curl_4.3.1

> [13] rstudioapi_0.13 fs_1.5.0 rmarkdown_2.8 tools_4.1.0

> [17] bit64_4.0.5 stringr_1.4.0 glue_1.4.2 bit_4.0.4

> [21] xfun_0.23 yaml_2.2.1 compiler_4.1.0 htmltools_0.5.1.1

> [25] knitr_1.33

Created on 2021-06-22 by the reprex package (v2.0.0)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

adamhsparks commented 3 years ago

Closed in https://github.com/ropensci/GSODR/commit/b00c42003887271ff0ab12f439e82218911ae85f