search

JuliaTime / TimeZones.jl

IANA time zone database access for the Julia programming language
Other
86 stars 51 forks source link

Rework tests to only perform a single fetch data from IANA servers #445

Open omus opened 10 months ago

omus commented 10 months ago

The current TimeZones.jl setup fetches a specific version of the tzdata from the IANA services for running tests. When running on CI this data is always fetched at least once. It was noticed in #441 that we perform a second fetch from the IANA services in tzdata/build.jl which hasn't been problematic but causes unnecessary load on their servers.

We may also want to look into using GHA caching to avoid downloading from their servers entirely.

omus commented 10 months ago

Definitely should use some GHA caching:

[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
ERROR: LoadError: InitError: RequestError: Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds while requesting https://data.iana.org/time-zones/releases/tzdata2016j.tar.gz

– https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554201#step:6:131

Looks like the tzdata_version check may also be an isssue:

[ Info: Downloading Windows to POSIX timezone ID XML version: release-43-1
[ Info: Compiling Windows time zone name translation
[ Info: Loading tzdata 2016j
[ Info: Downloading tzdata 2016j archive
[ Info: Decompressing tzdata 2016j region data
[ Info: Compiling tzdata 2016j region data
tzdata_version: Error During Test at D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:57
  Got exception outside of a @test
  RequestError: HTTP/1.1 200 OK (Operation too slow. Less than 1 bytes/sec transferred the last 20 seconds) while requesting https://data.iana.org/time-zones/releases/
  Stacktrace:
    [1] (::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool})(easy::Downloads.Curl.Easy)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:388
    [2] with_handle(f::Downloads.var"#9#18"{IOBuffer, Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, handle::Downloads.Curl.Easy)
      @ Downloads.Curl C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Curl\Curl.jl:90
    [3] #8
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:329 [inlined]
    [4] arg_write(f::Downloads.var"#8#17"{Base.DevNull, Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Bool, String, Bool, Bool}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
    [5] #7
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:328 [inlined]
    [6] arg_read
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:76 [inlined]
    [7] request(url::String; input::Nothing, output::IOBuffer, method::Nothing, headers::Vector{Pair{String, String}}, timeout::Float64, progress::Nothing, verbose::Bool, debug::Nothing, throw::Bool, downloader::Nothing)
      @ Downloads C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:327
    [8] request
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:295 [inlined]
    [9] #3
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:233 [inlined]
   [10] arg_write(f::Downloads.var"#3#4"{Nothing, Vector{Pair{String, String}}, Float64, Nothing, Bool, Nothing, Nothing, String}, arg::IOBuffer)
      @ ArgTools C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\ArgTools\src\ArgTools.jl:134
   [11] #download#2
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:232 [inlined]
   [12] download
      @ C:\hostedtoolcache\windows\julia\1.9.2\x64\share\julia\stdlib\v1.9\Downloads\src\Downloads.jl:221 [inlined]
   [13] tzdata_versions()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:56
   [14] tzdata_latest_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\download.jl:81
   [15] tzdata_version()
      @ TimeZones.TZData D:\a\TimeZones.jl\TimeZones.jl\src\tzdata\version.jl:92
   [16] (::var"#16#19")()
      @ Main D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:69
   [17] withenv(f::var"#16#19", keyvals::Pair{String, String})
      @ Base .\env.jl:197
   [18] macro expansion
      @ D:\a\TimeZones.jl\TimeZones.jl\test\tzdata\version.jl:68 [inlined]
   [19] macro expansion
...

– https://github.com/JuliaTime/TimeZones.jl/actions/runs/5946262840/job/16126554082#step:6:137