search

kernelci / kernelci-pipeline

Modular pipeline based on the KernelCI API
GNU Lesser General Public License v2.1
6 stars 18 forks source link

src/get_builds: service to fetch kernel build nodes #594

Open JenySadadia opened 2 months ago

JenySadadia commented 2 months ago

Related to https://github.com/kernelci/kernelci-api/pull/527

Add a service to poll for build events i.e. listen to successful kernel build nodes and fetch artifacts.

JenySadadia commented 2 months ago

Received event on staging where the node has a different owner.

05/10/2024 07:20:13 AM UTC [INFO] Listening for events... 
05/10/2024 07:20:13 AM UTC [INFO] Press Ctrl-C to stop.
05/10/2024 07:20:32 AM UTC [INFO] Build node received: {'id': '663dcac06d9eb92b38f2d846', 'kind': 'kbuild', 'name': 'kbuild-gcc-10-arm64-chromeos-mediatek', 'path': ['checkout', 'kbuild-gcc-10-arm64-chromeos-mediatek'], 'group': 'kbuild-gcc-10-arm64-chromeos-mediatek', 'parent': '663db9ab6d9eb92b38f2bfac', 'state': 'done', 'result': 'pass', 'artifacts': {'build_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_sh': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build.sh?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kimage_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kimage.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kimage_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kimage_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kselftest_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kselftest.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kselftest_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kselftest_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_dtbs_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_dtbs.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_dtbs_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_dtbs_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'metadata': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/metadata.json?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/0_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/0.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/1_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/1.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/2_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/2.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/3_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/3.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', '_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'kernel': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/Image?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'modules': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/modules.tar.xz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'kselftest_tar_gz': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/kselftest.tar.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D'}, 'data': {'kernel_revision': {'tree': 'kernelci', 'url': 'https://github.com/kernelci/linux.git', 'branch': 'staging-stable', 'commit': '76d3ceaa74811cee09b0df41a27a82f865fbf841', 'describe': 'staging-stable-20240510.2', 'version': {'version': 6, 'patchlevel': 1, 'sublevel': None, 'extra': '-15465-g76d3ceaa7481', 'name': None}, 'patchset': None}, 'arch': 'arm64', 'defconfig': 'cros://chromeos-6.1/arm64/chromiumos-mediatek.flavour.config', 'compiler': 'gcc-10', 'error_code': None, 'error_msg': None, 'fragments': ['arm64-chromebook', 'CONFIG_MODULE_COMPRESS=n', 'CONFIG_MODULE_COMPRESS_NONE=y'], 'config_full': 'cros://chromeos-6.1/arm64/chromiumos-mediatek.flavour.config+arm64-chromebook+CONFIG_MODULE_COMPRESS=n+CONFIG_MODULE_COMPRESS_NONE=y', 'platform': 'kubernetes', 'runtime': 'k8s-all', 'job_id': 'kci-663dba0e6d9eb92b38f2bfe7-kbuild-gcc-10-arm64-chro-wrnm0yd4', 'job_context': 'aks-kbuild-medium-1', 'kernel_type': 'image', 'regression': None}, 'debug': None, 'jobfilter': None, 'created': '2024-05-10T07:20:32.169000', 'updated': '2024-05-10T07:20:32.169000', 'timeout': '2024-05-10T13:20:32.169000', 'holdoff': None, 'owner': 'jeny', 'submitter': 'e05822fb07d1395ba4a40016491fa56b', 'user_groups': []}
05/10/2024 07:20:32 AM UTC [INFO] artifacts: {'build_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_sh': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build.sh?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kimage_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kimage.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kimage_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kimage_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kselftest_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kselftest.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_kselftest_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_kselftest_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_dtbs_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_dtbs.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'build_dtbs_stderr_log': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/build_dtbs_stderr.log.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'metadata': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/metadata.json?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/0_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/0.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/1_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/1.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/2_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/2.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'fragments/3_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/fragments/3.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', '_config': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/.config?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'kernel': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/Image?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'modules': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/modules.tar.xz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D', 'kselftest_tar_gz': 'https://kciapistagingstorage1.file.core.windows.net/staging/kbuild-gcc-10-arm64-chromeos-mediatek-663dba0e6d9eb92b38f2bfe7/kselftest.tar.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D'}
nuclearcat commented 4 weeks ago

Do we plan to merge this PR?

JenySadadia commented 4 weeks ago

I don't think we need to merge this. This was just a draft to demonstrate TI on fetching build nodes.