compile flag KERNEL_PROCURE_METHOD=download doesn't work

[eddieruan-alibaba]

When I compile 202311 branch with KERNEL_PROCURE_METHOD=download, the compilation fails at

get_url_version https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-5.10.0-23-2-common_5.10.179-3_all.deb?sv=2015-04-05&sr=b&sig=JmF0asLzRh6btfK4xxfVqX%2F5ylqaY4wLkMb5JwBJOb8%3D&se=2128-12-23T19%3A05%3A28Z&sp=r failed

@lguohan provided daily build log at https://sonic-build.azurewebsites.net/ui/sonic/pipelines

After comparing my log with daily log, I found that was due to that i was using KERNEL_PROCURE_METHOD=download while daily build would build linux kernel.

The detail analysis for this failure is the following. Need some one's help to check that server's authentication for downloading.

===============================

Thanks Guohan.

I saw the differences. The daily build will build kernel module, which goes via

https://github.com/sonic-net/sonic-linux-kernel/blob/59e5ea16e831d08e37f663327bf165255b60b98e/Makefile#L91

Obtaining the Debian kernel source

rm -rf $(BUILD_DIR) wget -O $(DSC_FILE) $(DSC_FILE_URL) wget -O $(ORIG_FILE) $(ORIG_FILE_URL) wget -O $(DEBIAN_FILE) $(DEBIAN_FILE_URL)

The compile log from https://sonic-build.azurewebsites.net/api/sonic/artifacts?branchName=202305&platform=broadcom&buildId=538643&target=target%2Fdebs%2Fbullseye%2Flinux-headers-5.10.0-23-2-common_5.10.179-3_all.deb.log showed as the following

Obtaining the Debian kernel source

rm -rf linux-5.10.179 wget -O linux_5.10.179-3.dsc ""https://sonicstorage.blob.core.windows.net/debian-security/pool/updates/main/l/linux"/linux_5.10.179-3.dsc" wget -O linux_5.10.179.orig.tar.xz ""https://sonicstorage.blob.core.windows.net/debian-security/pool/updates/main/l/linux"/linux_5.10.179.orig.tar.xz" wget -O linux_5.10.179-3.debian.tar.xz ""https://sonicstorage.blob.core.windows.net/debian-security/pool/updates/main/l/linux"/linux_5.10.179-3.debian.tar.xz"

In my case, I try to use prebuilt kernel module, which goes via https://github.com/sonic-net/sonic-linux-kernel/blob/59e5ea16e831d08e37f663327bf165255b60b98e/Makefile#L40

TBD, need upload the new kernel packages

LINUX_HEADER_COMMON_URL = "https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-$(KVERSION_SHORT)-common_$(KERNEL_VERSION)-$(KERNEL_SUBVERSION)_all.deb?sv=2015-04-05&sr=b&sig=JmF0asLzRh6btfK4xxfVqX%2F5ylqaY4wLkMb5JwBJOb8%3D&se=2128-12-23T19%3A05%3A28Z&sp=r" LINUX_HEADER_AMD64_URL = "https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-$(KVERSION_SHORT)-amd64_$(KERNEL_VERSION)-$(KERNEL_SUBVERSION)_amd64.deb?sv=2015-04-05&sr=b&sig=%2FD9a178J4L%2FN3Fi2uX%2FWJaddpYOZqGmQL4WAC7A7rbA%3D&se=2128-12-23T19%3A06%3A13Z&sp=r" LINUX_IMAGE_URL = "https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-image-$(KVERSION_SHORT)-amd64_$(KERNEL_VERSION)-$(KERNEL_SUBVERSION)_amd64.deb?sv=2015-04-05&sr=b&sig=oRGGO9xJ6jmF31KGy%2BwoqEYMuTfCDcfILKIJbbaRFkU%3D&se=2128-12-23T19%3A06%3A47Z&sp=r" $(addprefix $(DEST)/, $(MAIN_TARGET)): $(DEST)/% :

Obtaining the Debian kernel packages

rm -rf $(BUILD_DIR) wget --no-use-server-timestamps -O $(LINUX_HEADER_COMMON) $(LINUX_HEADER_COMMON_URL) wget --no-use-server-timestamps -O $(LINUX_HEADER_AMD64) $(LINUX_HEADER_AMD64_URL) wget --no-use-server-timestamps -O $(LINUX_IMAGE) $(LINUX_IMAGE_URL)

I ran the command manually. The failure is due to authentication. wget --no-use-server-timestamps -O linux-headers-5.10.0-23-2-common_5.10.179-3_all.deb "https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-5. 10.0-23-2-common_5.10.179-3_all.deb?sv=2015-04-05&sr=b&sig=JmF0asLzRh6btfK4xxfVqX%2F5ylqaY4wLkMb5JwBJOb8%3D&se=2128-12-23T19%3A05%3A28Z&sp=r"
--2024-05-05 07:55:09-- https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-5.10.0-23-2-common_5.10.179-3_all.deb?sv=2015-04-05&sr=b&sig=JmF0asLzRh6btfK4xxfVqX%2 F5ylqaY4wLkMb5JwBJOb8%3D&se=2128-12-23T19%3A05%3A28Z&sp=r
Resolving sonicstorage.blob.core.windows.net (sonicstorage.blob.core.windows.net)... 20.60.59.131
Connecting to sonicstorage.blob.core.windows.net (sonicstorage.blob.core.windows.net)|20.60.59.131|:443... connected.
HTTP request sent, awaiting response... 403 Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.
2024-05-05 07:55:10 ERROR 403: Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature..

Would these authentication string be still valid? It seems the timestamp indicated they were from 2015-04-05.

LINUX_HEADER_COMMON_URL = "https://sonicstorage.blob.core.windows.net/packages/kernel-public/linux-headers-$(KVERSION_SHORT)-common_$(KERNEL_VERSION)-$(KERNEL_SUBVERSION)_all.deb?sv=2015-04-05&sr=b&sig=JmF0asLzRh6btfK4xxfVqX%2F5ylqaY4wLkMb5JwBJOb8%3D&se=2128-12-23T19%3A05%3A28Z&sp=r"

Anyway, I will enable kernel compile as a workaround. But it would be great if we could skip them, since it takes almost 2 hours

Build end time: Sat May 4 10:46:07 UTC 2024 Elapsed time: 1h 43m 37s

[eddieruan-alibaba]

Email thread in sonic-dev https://lists.sonicfoundation.dev/g/sonic-dev/message/706

[neethajohn]

@saiarcot895, please take a look. method has been deprecated