Timeouts downloading cabal.config's from stackage in azure ci

jneira commented 4 years ago

Hi! In hie we have to generate the hoogle database in our azure ci env and it has started to throw response time outs downloading cabal.config from stackage nightly or lts. The issue is related with the environment, locally i've got to download all files, although they can take several seconds. In past succesful builds i see in logs:

Downloading https://www.stackage.org/nightly/cabal.config... 4.88s
Downloading https://www.stackage.org/lts/cabal.config... 3.29s
Downloading https://raw.githubusercontent.com/haskell/haskell-platform/master/hptool/src/Releases2015.hs... 0.28s
Downloading https://hackage.haskell.org/packages/index.tar.gz... 6.98s
Downloading https://hackage.haskell.org/packages/hoogle.tar.gz... 3.85s

In failed ones :

Starting generate
Downloading https://www.stackage.org/nightly/cabal.config... 20.66s
Downloading https://www.stackage.org/lts/cabal.config... hoogle: HttpExceptionRequest Request {
  host                 = "www.stackage.org"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = "/lts/cabal.config"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 ResponseTimeout

when it get to downlaod nightly (in most of them the timeout is with nightly itself)

Locally i am getting just now:

Downloading https://www.stackage.org/lts/cabal.config... 1m01s
Downloading https://raw.githubusercontent.com/haskell/haskell-platform/master/hptool/src/Releases2015.hs... 0.55s
Downloading https://hackage.haskell.org/packages/index.tar.gz... 6.59s
Downloading https://hackage.haskell.org/packages/hoogle.tar.gz... 15.59s

but the timeout is not triggered.

Surely the root of the problem is not with hoogle itself but is there some way to increase the response timeout with some config and work around the issue?

ndmitchell commented 4 years ago

From the experiments I ran, when it was failing to get through, it was taking literally forever - a longer timeout wouldn't help. The issues started occurring a week or so ago and are being tracked at https://github.com/fpco/stackage-server/issues/291. That said, if you can find a setting that reduces it or makes things work even so, happy to apply it.

jneira commented 3 years ago

Closing as i am not using azure anymore and the reasons noted above

ndmitchell / hoogle

Timeouts downloading cabal.config's from stackage in azure ci #347