Zeus branch compilation issue due to GitHub's deprecation of git:// protocol

[sergiompopoli]

As stated in Improving Git protocol security on GitHub, after March 15th, 2023 the git:// protocol is no longer supported. Because of this I was getting an error when bitbaking my Digi Embedded Yocto custom image (or any yocto image including meta-aws layer). In my particular case, the issue was with aws-iot-device-sdk-python-v1. Even if commit 55b20c1d78687fbd0232adc09f3fcc0aec364a3f addresses this issue, I was still getting errors. So I had to manually change file /usr/local/dey-3.0/sources/meta-aws/recipes-sdk/aws-iot-device-sdk-python-v1/aws-iot-device-sdk-python-v1.inc as follows:

Where it said: SRC_URI = "https://github.com/aws/aws-iot-device-sdk-python.git;branch=${BRANCH};name=aws-iot-device-sdk-python-v1"

I wrote: SRC_URI = "git://github.com/aws/aws-iot-device-sdk-python.git;protocol=https;branch=${BRANCH};name=aws-iot-device-sdk-python-v1" And the bitbake fetch operation finally worked. I advise you check other recipes that may suffer from this issue as well. I hope this helps others.

[nateglims]

Hi Sergio,

I am able to build the python device sdk v1 with the https:// url using poky's current zeus release. We will assess the URLs in this release, but the https:// url scheme should be preferred now with github.

[nateglims]

If you are able to share your layer SHAs, we can try to recreate issue.

[sergiompopoli]

Hello nateglims, Yes, I understand that it should work with the https://, in fact changing from the git:// to the https:// for GitHub repos is what solved many other issues. However, I have just reverted meta-aws to commit 1cea10e5f237f547d9b186f237c9936951f69384 (current zeus HEAD), bitbaked only aws-iot-device-sdk-python-v1, and I'm getting the following error:

bitbake aws-iot-device-sdk-python-v1
(...)
Build Configuration:
BB_VERSION           = "1.44.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "aarch64-dey-linux"
MACHINE              = "ccimx8mn-dvk"
DISTRO               = "dey"
DISTRO_VERSION       = "3.0-r3"
TUNE_FEATURES        = "aarch64"
TARGET_FPU           = ""
meta                 
meta-poky            
meta-yocto-bsp       = "HEAD:d88d62c20d7d8da85f02edb170dae0280624ad7e"
meta-oe              
meta-python          
meta-networking      
meta-webserver       = "zeus:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-qt5             = "zeus:91e6e3b1f5aebd17f895caac3dee32a14630da99"
meta-qt5-extra       = "zeus:8b53b092e0f0425f97197a698124191b26d4f280"
meta-multimedia      
meta-gnome           = "zeus:2b5dd1eb81cd08bc065bc76125f2856e9383e98b"
meta-aws             = "zeus:1cea10e5f237f547d9b186f237c9936951f69384"
meta-swupdate        = "HEAD:8ad6a75e1778cdda671b3d6aea2a9692613020ec"
meta-freescale       = "zeus:14f1a630a47375432f93c556927b879b51d84c4e"
meta-fsl-demos       = "zeus:ca27d12e4964d1336e662bcc60184bbff526c857"
meta-python2         = "zeus:4400f9155ec193d028208cf0c66aeed2ba2b00ab"
meta-webkit          = "zeus:ab4c99d825ed99901e7bd86870312f6c70865f2e"
meta-ml              = "zeus-5.4.70-2.3.1:75a2feb2d90cdab5361e544280d1f183f04b8911"
meta-digi-arm        
meta-digi-dey        = "meta-digi-overture:451aab2269f7291ed81e73ef162a87397c527b35"
meta-overture        = "main:9c849d8debda3b32c0db06f0771b5fad14df7d6a"

Initialising tasks: 100% |######################################################################################################################################################| Time: 0:00:00
Sstate summary: Wanted 6 Found 0 Missed 6 Current 200 (0% match, 97% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
ERROR: aws-iot-device-sdk-python-v1-git-r0 do_fetch: No checksum specified for /home/metabolomics/DEY/workspace/2023-03_08-ccimx8mn-dvk-zeus-0.3.2/../downloads/aws-iot-device-sdk-python.git, please add at least one to the recipe:
SRC_URI[aws-iot-device-sdk-python-v1.md5sum] = "b42d99138f3a404a96a2bc1e815d8835"
SRC_URI[aws-iot-device-sdk-python-v1.sha256sum] = "e29445cbfc724dda95d925537a8febcc7a87247a61b90583c35bb7631f1a29e0"
ERROR: aws-iot-device-sdk-python-v1-git-r0 do_fetch: Fetcher failure for URL: 'https://github.com/aws/aws-iot-device-sdk-python.git;branch=master;name=aws-iot-device-sdk-python-v1'. Missing SRC_URI checksum
ERROR: Logfile of failure stored in: /home/metabolomics/DEY/workspace/2023-03_08-ccimx8mn-dvk-zeus-0.3.2/tmp/work/aarch64-dey-linux/aws-iot-device-sdk-python-v1/git-r0/temp/log.do_fetch.50023
ERROR: Task (/usr/local/dey-3.0/sources/meta-aws/recipes-sdk/aws-iot-device-sdk-python-v1/aws-iot-device-sdk-python-v1_git.bb:do_fetch) failed with exit code '1'
NOTE: Tasks Summary: Attempted 884 tasks of which 883 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /usr/local/dey-3.0/sources/meta-aws/recipes-sdk/aws-iot-device-sdk-python-v1/aws-iot-device-sdk-python-v1_git.bb:do_fetch
Summary: There were 2 ERROR messages shown, returning a non-zero exit code.

As you can see, it seems to be an issue within the do_fetch step with the checksums. If we check the generated tmp/work/aarch64-dey-linux/aws-iot-device-sdk-python-v1/git-r0/temp/log.do_fetch.50023:

DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are []
NOTE: Installed into sysroot: []
NOTE: Skipping as already exists in sysroot: []
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_fetch
DEBUG: Executing python function base_do_fetch
ERROR: No checksum specified for /home/metabolomics/DEY/workspace/2023-03_08-ccimx8mn-dvk-zeus-0.3.2/../downloads/aws-iot-device-sdk-python.git, please add at least one to the recipe:
SRC_URI[aws-iot-device-sdk-python-v1.md5sum] = "b42d99138f3a404a96a2bc1e815d8835"
SRC_URI[aws-iot-device-sdk-python-v1.sha256sum] = "e29445cbfc724dda95d925537a8febcc7a87247a61b90583c35bb7631f1a29e0"
ERROR: Fetcher failure for URL: 'https://github.com/aws/aws-iot-device-sdk-python.git;branch=master;name=aws-iot-device-sdk-python-v1'. Missing SRC_URI checksum
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished

So, I only managed to get it to work once I changed aws-iot-device-sdk-python-v1.inc to : SRC_URI = "git://github.com/aws/aws-iot-device-sdk-python.git;protocol=https;branch=${BRANCH};name=aws-iot-device-sdk-python-v1"

[nateglims]

I was starting from an incorrect version 🤦

You are absolutely correct, it appears for zeus that the fix was applied differently and the git:// form is correct: https://git.yoctoproject.org/poky/commit/?h=zeus&id=daf096e295121ea49ebf21f8070e9a6e28f5d46c

I will revert and test the layer shortly.

Thanks!

[thomas-roos]

This pull request should solve this issue: https://github.com/aws4embeddedlinux/meta-aws/pull/3307