Many "Error: Curl - Failure receiving data from server" when updating from 41370 to 41400.

lars-nvhgroup commented 7 months ago

When updating from 41370 to 41400 I receive, e.g: Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41390/pack-devpkg-llvm-from-40960.tar' and Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41390/pack-strace-from-41280.tar' among others. I receive these responses:

<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>update/41390/pack-devpkg-llvm-from-40960.tar</Key>
<RequestId>PKA0VY06GA4J1Q0B</RequestId>
<HostId>+gXeEUIDaikzFkJE76xHkk2CWAy0icBfiz675sB4mZWy4cf/qiXOceqwkIRVOVm6bVzAWHml/7tiGgV0GX+zeQ==</HostId>
</Error>

<Error>
<Code>NoSuchKey</Code>
<Message>The specified key does not exist.</Message>
<Key>update/41390/pack-strace-from-41280.tar</Key>
<RequestId>ZN0ART9K69J188TF</RequestId>
<HostId>z77gUIM9w3YkZP2Ue0YDd7Keu+AR1q0ugkdM0BOHTrUe4cIoPhcw2h6T/I2rfGoFYI7t1zpv4+Q=</HostId>
</Error>

fenrus75 commented 7 months ago

it looks like something in the CDN setup is not working well -- we're working on finding out what / resyncing our content out to the CDN

On Wed, Apr 3, 2024 at 11:36 AM Lars Hansson @.***> wrote:

When updating from 41370 to 41400 I receive, e.g: Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41390/pack-devpkg-llvm-from-40960.tar ' and Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41390/pack-strace-from-41280.tar ' among others. Looking up them I receive these responses:
NoSuchKey The specified key does not exist. update/41390/pack-devpkg-llvm-from-40960.tar PKA0VY06GA4J1Q0B +gXeEUIDaikzFkJE76xHkk2CWAy0icBfiz675sB4mZWy4cf/qiXOceqwkIRVOVm6bVzAWHml/7tiGgV0GX+zeQ== NoSuchKey The specified key does not exist. update/41390/pack-strace-from-41280.tar ZN0ART9K69J188TF z77gUIM9w3YkZP2Ue0YDd7Keu+AR1q0ugkdM0BOHTrUe4cIoPhcw2h6T/I2rfGoFYI7t1zpv4+Q=
— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FNYHZMB2MNIGRTDQO3Y3RD2NAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZDGNRUGQ4DANY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

fenrus75 commented 7 months ago

update on this

what seems to have happened is that for the 390 release, a few optional files were not created as part of the release

conflated with

something in our CDN chain changed from returning 404 http errors for non-existing files to returning a bunch of XML gunk without 404

swupd gives up on 404s for optional files and falls back to the non-optional side... ... but on invalid/aborted content it retries (unfortunately in this case, and we'll need to fix this)

we're now backfilling the missing files so we avoid the 404-is-not-there-mess.... ... and we'll fix swupd to deal with the new CDN twist

On Wed, Apr 3, 2024 at 11:58 AM Arjan van de Ven @.***> wrote:

it looks like something in the CDN setup is not working well -- we're working on finding out what / resyncing our content out to the CDN

On Wed, Apr 3, 2024 at 11:36 AM Lars Hansson @.***> wrote:

When updating from 41370 to 41400 I receive, e.g: Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41390/pack-devpkg-llvm-from-40960.tar ' and Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41390/pack-strace-from-41280.tar ' among others. Looking up them I receive these responses:
NoSuchKey The specified key does not exist. update/41390/pack-devpkg-llvm-from-40960.tar PKA0VY06GA4J1Q0B +gXeEUIDaikzFkJE76xHkk2CWAy0icBfiz675sB4mZWy4cf/qiXOceqwkIRVOVm6bVzAWHml/7tiGgV0GX+zeQ== NoSuchKey The specified key does not exist. update/41390/pack-strace-from-41280.tar ZN0ART9K69J188TF z77gUIM9w3YkZP2Ue0YDd7Keu+AR1q0ugkdM0BOHTrUe4cIoPhcw2h6T/I2rfGoFYI7t1zpv4+Q=
— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FNYHZMB2MNIGRTDQO3Y3RD2NAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZDGNRUGQ4DANY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

Aqua1ung commented 7 months ago

we're now backfilling the missing files

Waypipe seems to be missing still.

mfdemicco commented 7 months ago

Still broken for me:

Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 10 seconds before retrying the download Retry #1 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 20 seconds before retrying the download Retry #2 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 40 seconds before retrying the download etc.

fenrus75 commented 7 months ago

is this the only one? just wait a little (minute or two ) and it'll move on

On Fri, Apr 5, 2024 at 12:22 PM mfdemicco @.***> wrote:

Still broken for me:

Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 10 seconds before retrying the download Retry #1 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 20 seconds before retrying the download Retry #2 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 40 seconds before retrying the download etc.

— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088#issuecomment-2040495156, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FLZMCIOTXE2WYWJSOLY332YFAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQGQ4TKMJVGY . You are receiving this because you commented.Message ID: @.***>

mfdemicco commented 7 months ago

is this the only one? just wait a little (minute or two ) and it'll move on … On Fri, Apr 5, 2024 at 12:22 PM mfdemicco @.> wrote: Still broken for me: Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 10 seconds before retrying the download Retry #1 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 20 seconds before retrying the download Retry #2 downloading from https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar Error: Curl - Failure receiving data from server - ' https://cdn.download.clearlinux.org/update/41420/pack-NetworkManager-from-41210.tar' Waiting 40 seconds before retrying the download etc. — Reply to this email directly, view it on GitHub <#3088 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FLZMCIOTXE2WYWJSOLY332YFAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBQGQ4TKMJVGY . You are receiving this because you commented.Message ID: @.>

No. It's been going for at least an hour, maybe two:

scarolac commented 7 months ago

I disabled retries to see the full list. The update completes after it skips all of these. I removed excess text, kept only the addresses. Are any of them going to cause issues later or were they all optional?

Preparing to update from 41380 to 41450

fenrus75 commented 7 months ago

ald "pack--from.tar" are optional -- and the ones that are missing would otherwise be empty files ;(

I'll do another validation run -- we filled all from 370 to 430 but seems you're from 380 to 450 :)

On Tue, Apr 9, 2024 at 8:25 AM Chris Scarola @.***> wrote:

I disabled retries to see the full list. The update completes after it skips all of these. I removed excess text, kept only the addresses. Are any of them going to cause issues later or were they all optional?

Preparing to update from 41380 to 41450

' https://cdn.download.clearlinux.org/update/41450/Manifest-sysadmin-basic-delta-from-41380 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-pulseaudio-delta-from-41340 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-perl-basic-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-os-core-plus-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-libX11client-delta-from-41340 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-gstreamer-delta-from-41380 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-harfbuzz-lib-delta-from-41380 ' ...

... ' https://cdn.download.clearlinux.org/update/41450/pack-gstreamer-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-harfbuzz-lib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-lib-opengl-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-libX11client-from-41340.tar ' ' https://cdn.download.clearlinux.org/update/41420/pack-libglib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41420/pack-libunwind-lib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-os-core-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-os-core-plus-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41430/pack-os-core-update-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-perl-basic-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-pulseaudio-from-41340.tar ' ' https://cdn.download.clearlinux.org/update/41440/pack-python3-basic-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-sysadmin-basic-from-41380.tar ' 'https://cdn.download.clearlinux.org/update/41420/pack-zstd-from-41380.tar '

— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088#issuecomment-2045475808, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FIEE67SOMTCCU573HDY4QB5ZAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVGQ3TKOBQHA . You are receiving this because you commented.Message ID: @.***>

fenrus75 commented 7 months ago

(I did not realize we had released 380 -- I thought we went from 370 to 390)

On Tue, Apr 9, 2024 at 8:29 AM Arjan van de Ven @.***> wrote:

ald "pack--from.tar" are optional -- and the ones that are missing would otherwise be empty files ;(

I'll do another validation run -- we filled all from 370 to 430 but seems you're from 380 to 450 :)

On Tue, Apr 9, 2024 at 8:25 AM Chris Scarola @.***> wrote:

I disabled retries to see the full list. The update completes after it skips all of these. I removed excess text, kept only the addresses. Are any of them going to cause issues later or were they all optional?

Preparing to update from 41380 to 41450

' https://cdn.download.clearlinux.org/update/41450/Manifest-sysadmin-basic-delta-from-41380 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-pulseaudio-delta-from-41340 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-perl-basic-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-os-core-plus-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-libX11client-delta-from-41340 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-gstreamer-delta-from-41380 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370 ' ' https://cdn.download.clearlinux.org/update/41450/Manifest-harfbuzz-lib-delta-from-41380 ' ...

... ' https://cdn.download.clearlinux.org/update/41450/pack-gstreamer-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-harfbuzz-lib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-lib-opengl-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-libX11client-from-41340.tar ' ' https://cdn.download.clearlinux.org/update/41420/pack-libglib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41420/pack-libunwind-lib-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-os-core-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-os-core-plus-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41430/pack-os-core-update-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-perl-basic-from-41370.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-pulseaudio-from-41340.tar ' ' https://cdn.download.clearlinux.org/update/41440/pack-python3-basic-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41450/pack-sysadmin-basic-from-41380.tar ' ' https://cdn.download.clearlinux.org/update/41420/pack-zstd-from-41380.tar '

— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088#issuecomment-2045475808, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FIEE67SOMTCCU573HDY4QB5ZAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVGQ3TKOBQHA . You are receiving this because you commented.Message ID: @.***>

scarolac commented 7 months ago

I did 370 to 380 with no issue, after that is when the errors started popping

I tried doing 380 to 390, 400, 420 and all had similar issues until i just disabled retries to the full 450 to see what happens

fenrus75 commented 7 months ago

380->450 should now be complete (the last few bits are syncing to the CDN so it might be a few minutes to show up)

On Tue, Apr 9, 2024 at 8:34 AM Chris Scarola @.***> wrote:

I did 370 to 380 with no issue, after that is when the errors started popping

— Reply to this email directly, view it on GitHub https://github.com/clearlinux/distribution/issues/3088#issuecomment-2045498569, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ54FM2VCF7VGLIXXTPVIDY4QDCFAVCNFSM6AAAAABFV3PIQWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBVGQ4TQNJWHE . You are receiving this because you commented.Message ID: @.***>

K1ngfish3r commented 7 months ago

sorry guys

i@clr~ $ sudo swupd update
Password: 
Update started
Preparing to update from 41370 to 41520
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370'
Waiting 10 seconds before retrying the download
Retry #1 downloading from https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370'
Waiting 20 seconds before retrying the download
Retry #2 downloading from https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370'
Waiting 40 seconds before retrying the download
Retry #3 downloading from https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41370'
Warning: Maximum number of retries reached

then it goes to the next file and repeats

ruslanbay commented 5 months ago

Hi Can you please check if this is happening again? I can download a bundle manually, but swupd fails with the "Failure receiving data from server" error:

asreich commented 5 months ago

I'm having this issue with 41380 to 41520.

sudo swupd update
Update started
Preparing to update from 41380 to 41520
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41380'
Waiting 10 seconds before retrying the download
Retry #1 downloading from https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41380
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41520/Manifest-os-core-delta-from-41380'
Waiting 20 seconds before retrying the download

Edit: @fenrus75 Now it's trying a different intermediate version? 41370 to 41450?

sudo swupd update
Update started
Preparing to update from 41380 to 41520
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370'
Waiting 10 seconds before retrying the download
Retry #1 downloading from https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370'
Waiting 20 seconds before retrying the download
Retry #2 downloading from https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370
Error: Curl - Failure receiving data from server - 'https://cdn.download.clearlinux.org/update/41450/Manifest-lib-opengl-delta-from-41370'
Waiting 40 seconds before retrying the download

Edit 2: Tried direct from 41380 to 41450 as directed and it didn't work as described sudo swupd update --version 41450

Edit 3: 41380 to 41440 works because the pack-*.tar files are optional as described above. Find a version you can update to you that doesn't complain about "Manifest" files. sudo swupd update --version 41440 -r 0

clearlinux / distribution

Many "Error: Curl - Failure receiving data from server" when updating from 41370 to 41400. #3088