search

tlc-pack / tlcpack

https://tlcpack.ai/
Apache License 2.0
23 stars 30 forks source link

[CI] Add new PyPI packaging Jenkins pipeline #168

Closed Liam-Sturge closed 1 year ago

Liam-Sturge commented 1 year ago

This patch adds a new Jenkins pipeline for packaging. This is a migration of the current Github actions workflow of uploading a Tensor learning compiler binary distribution (tlcpack) to PyPI.

This new packaging pipeline implements a parallel step that creates and uploads AArch64 packages, which is something not present in the Github actions job it is intended to replace.

I have tested the pipeline here, with some minor modifications so that the packages were uploaded to my personal test.pypi account. The code submitted in this patch will attempt to upload to the apache-tvm production package.

cc @areusch @driazati @tqchen @leandron

tqchen commented 1 year ago

I still think it is valuable to maintain and use the GH action pipeline when possible(as they come for free) and use the new jenkins one for cases that are not covered by GH

leandron commented 1 year ago

I still think it is valuable to maintain and use the GH action pipeline when possible(as they come for free) and use the new jenkins one for cases that are not covered by GH

The issue is mostly about synchronisation of packages being published. If we have two separate pipelines (one in GitHub Actions and one in Jenkins), we're not guaranteed publishing the same reference package for all variants.

Apart from that, the proposed method also helps us with generating packages on demand as part of releases.

tqchen commented 1 year ago

Given it is natural for different platforms to have some different config and we do have different publication pipelines, and we are using different mechanisms for example in the case of windows/Mac, I don't think it would make too big of a difference as long as we ensure we point to the same hashtag. GH action can also be triggered on demand i believe

Liam-Sturge commented 1 year ago

Has this workflow been tested / if so can you link the passing runs and example artifacts somewhere?

Yes, I tested this out here: https://ci.tlcpack.ai/blue/organizations/jenkins/pypi-packaging%2Fregular-pypi-package-upload/detail/regular-pypi-package-upload/4/pipeline

The packages were successfully produced, but these were uploaded to my personal test PyPI space.

leandron commented 1 year ago

This is approved for quite a few days, merging it now.