"Unexpected content-type" when uploading gzipped docs

[dpwiz]

When running cabal haddock --haddock-for-hackage it results in a .tar.gz file being created.

But trying to upload it (chrome, firefox) I'm getting the error:

Unexpected content-type For this resource the content-type was expected to be application/x-tar, rather than application/gzip

And have to undo the gzip wrapping the cabal did for me before uploading it again.

This is annoying and wasteful.

[andreasabel]

@dpwiz : Thanks for the bug report. Please provide full details that make the bug easily reproducible. (Think of a script that can be run to trigger the bug, without making permanent modifications to Hackage.)

[dpwiz]

cabal haddock --haddock-for-hackage hello

HACKAGE_API_KEY=...............................
PACKAGE_VER=hello-0.1.0.0
curl \
  --header "Authorization: X-ApiKey $HACKAGE_API_KEY" \
  -F_method=PUT \
  -F_transform=file2raw \
  -F_file=@dist-newstyle/$PACKAGE_VER-docs.tar.gz \
  https://hackage.haskell.org/package/$PACKAGE_VER/docs

On a related note, I'd prefer having cabal haddock --upload or something like that rather than doing all of this manually.

[andreasabel]

There is 'cabal upload -d' for that purpose. Does this work for you?

[dpwiz]

Not quite....

Uploading documentation
dist-newstyle/package-name-0.1.0.0-docs.tar.gz...
cabal: '/usr/bin/curl' exited with an error:
Warning: -:2: warning: '--user' uses unquoted whitespace in the line that may
Warning: cause side-effects!
curl: (56) Illegal or missing hexadecimal sequence in chunked-encoding

[andreasabel]

This looks like a bug in cabal. If possible, can you please search for this or similar in the cabal issues? I had a problem with curl and cabal once. Might also be OS specific.

[dpwiz]

Meanwhile, I'd be glad for Hackage to allow me upload gzipped tarballs from the browser (:

[andreasabel]

On macOS, I am getting e.g.

$ cabal upload -d /Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz -v
Uploading documentation
/Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz...
Running: /usr/local/opt/curl/bin/curl --config 
- 'https://hackage.haskell.org/package/HsYAML-aeson-0.2.0.1/candidate/docs' 
--request PUT 
--data-binary '@/Users/abel/bin/src/HsYAML-aeson/dist-newstyle/HsYAML-aeson-0.2.0.1-docs.tar.gz' 
--write-out '%{http_code}' 
--user-agent 'cabal-install/3.7 (osx; x86_64)' 
--silent --show-error --location 
--header 'Accept: text/plain' 
--header 'Content-Type: application/x-tar' 
--header 'Content-Encoding: gzip'
Documentation successfully uploaded for package candidate. You can now preview the result at
'https://hackage.haskell.org/package/HsYAML-aeson-0.2.0.1/candidate'. 
To upload non-candidate documentation, use 'cabal upload --publish'.

Maybe from running a similar command (with verbosity option -v) you can see how cabal is using curl and maybe you can succeed doing it similarly.

[dpwiz]

Looks like the browser uses content-type=gzip and enconding=whatever instead.

[andreasabel]

Could you fix your problem, @dpwiz ?

[dpwiz]

No. Using browser still gives "unexpected content-type". And cabal upload -d still gives me 401.

[andreasabel]

Please give some basic context: Version of

• OS
• curl
• cabal
• browser

What happens if you use curl in the way of https://github.com/haskell/hackage-server/issues/1070#issuecomment-1124320024 ?

[dpwiz]

• Linux (Ubuntu 22.04)
• curl 7.81.0
• cabal-install version 3.6.2.0
• Chrome 100, Firefox 100

[dpwiz]

Anyway, the details of my setup are irrelevant. The code states explicitly it will refuse compressed tarballs: https://github.com/haskell/hackage-server/blob/8ec64a1a30b6d53e4e86235c50e398ec803f50f1/src/Distribution/Server/Features/Documentation.hs#L312

(Yes, I can see curl instructions in there. I just like to have the browser upload form working, UX-wise.)

[andreasabel]

I can confirm (in one instance) that the web interface for documentation upload rejected a .tar.gz but accepted the gunzipped version .tar. (Firefox 100)

I have never used the web form before, I always use cabal upload -d. Weird about the 401. Can you upload candidate packages (cabal upload)?

[dpwiz]

Nah, cabal can't upload neither package nor docs.

[andreasabel]

Well, a 401 is about missing authorization. Are your auth settings up-to-date in .cabal/config?

hackage-username: ...
hackage-password: ...

[dpwiz]

There is a bug in its "curl" transport. I can upload docs for candidate, but it gets reset after publishing. And I can't upload docs to a non-candidate version even with "wget" transport.

[sgraf812]

@hasufell and me have been triggering this issue as well, and @hasufell opened https://github.com/haskell/cabal/issues/10252. It would be great to sort this out.

FWIW, this morning, the bug triggered quite consistently for me as I was trying to upload docs. After I set the hackage-* auth settings in my ~/.cabal/config, upload seemed to work fine.

It seems kind of intermittent for me.