search

apache / tvm

Open deep learning compiler stack for cpu, gpu and specialized accelerators
https://tvm.apache.org/
Apache License 2.0
11.82k stars 3.48k forks source link

[Release] v0.11.0 release schedule #13586

Open leandron opened 1 year ago

leandron commented 1 year ago

In order to continue with our agreed release cycle as per discussion in RFC#67 , I'd like to propose a schedule for our next TVM release: v0.11.0.

This schedule is tentative and may change as we progress through the process. In case dates change, this thread will be kept updated.

The proposed schedule is:

Note: for this specific release, given we'll have the end of the year period, I've extended the cherry-picking hard deadline in 1 week, so that we some more time to consider patches to be included.

Call for release managers: in case you want to be involved in upcoming releases, please manifest your interest in this thread and we'll try to organise.

See also:

cc @apache/tvm-committers @driazati @areusch @Mousius @tqchen @AndrewZhaoLuo

leandron commented 1 year ago

@apache/tvm-committers Just a reminder that today is the last day to have fixes and PRs ported to the v0.11.0 branch. Let me know.

areusch commented 1 year ago

we discussed the release in the community meeting this morning. here are notes:

leandron commented 1 year ago

I'm picking this up and facing an issue with protected branches when trying to cherry-pick #13624 into v0.11.0 - any idea how to solve that?

$ git push -f upstream v0.11.0
Enumerating objects: 42, done.
Counting objects: 100% (42/42), done.
Delta compression using up to 8 threads
Compressing objects: 100% (19/19), done.
Writing objects: 100% (23/23), 3.97 KiB | 1.32 MiB/s, done.
Total 23 (delta 18), reused 5 (delta 4), pack-reused 0
remote: Resolving deltas: 100% (18/18), completed with 18 local objects.
remote: error: GH006: Protected branch update failed for refs/heads/v0.11.0.
remote: error: At least 1 approving review is required by reviewers with write access. Commits must have valid signatures.
To github.com:apache/tvm.git
 ! [remote rejected]     v0.11.0 -> v0.11.0 (protected branch hook declined)
error: failed to push some refs to 'github.com:apache/tvm.git'

$ git push --delete upstream v0.11.0
remote: error: GH006: Protected branch update failed for refs/heads/v0.11.0.
remote: error: Cannot delete this protected branch
To github.com:apache/tvm.git
 ! [remote rejected]     v0.11.0 (protected branch hook declined)
error: failed to push some refs to 'github.com:apache/tvm.git'

cc @driazati

driazati commented 1 year ago

The release branches are protected from force pushes / deletes by GitHub, but cherry-picking shouldn't require rewriting history right? Are you doing something like this?

git fetch origin
git checkout v0.11.0
git cherry-pick 0eabbac2160a8a630e1994969f664ccf6233fc7e
git push origin
Johnson9009 commented 1 year ago

Can we ensure a date that this release will be completed? So the downstream organization can decide whether sync this release to our internal or not, thanks.

leandron commented 1 year ago

Can we ensure a date that this release will be completed? So the downstream organization can decide whether sync this release to our internal or not, thanks.

I'm working on this at the moment, and will update here as soon as it's ready for review/vote. thanks.

leandron commented 1 year ago

The release branches are protected from force pushes / deletes by GitHub, but cherry-picking shouldn't require rewriting history right? Are you doing something like this?

git fetch origin
git checkout v0.11.0
git cherry-pick 0eabbac2160a8a630e1994969f664ccf6233fc7e
git push origin

I was just trying to produce the tarball with the actual v0.11.0 with that cherry-picked commit, but I'm probably missing something:

% git remote -v
origin  git@github.com:leandron/tvm.git (fetch)
origin  git@github.com:leandron/tvm.git (push)
upstream    git@github.com:apache/tvm.git (fetch)
upstream    git@github.com:apache/tvm.git (push)

% git fetch upstream
% git checkout upstream/v0.11.0

% git reset --hard upstream/v0.11.0                       
HEAD is now at 1d9863470 [TIR] Fix PlanAndUpdateBufferAllocationLocation not visiting constant buffer (#13605)

% git cherry-pick 0eabbac2160a8a630e1994969f664ccf6233fc7e
[v0.11.0 2f999d62e] [BugFix][UMA] Protect target registration (#13624)
 Author: Balint Cristian <cristian.balint@gmail.com>
 Date: Fri Dec 16 15:23:26 2022 +0200
 4 files changed, 42 insertions(+), 16 deletions(-)

% git push upstream v0.11.0                        
Enumerating objects: 42, done.
Counting objects: 100% (42/42), done.
Delta compression using up to 8 threads
Compressing objects: 100% (19/19), done.
Writing objects: 100% (23/23), 3.93 KiB | 3.93 MiB/s, done.
Total 23 (delta 18), reused 5 (delta 4), pack-reused 0
remote: Resolving deltas: 100% (18/18), completed with 18 local objects.
remote: error: GH006: Protected branch update failed for refs/heads/v0.11.0.
remote: error: At least 1 approving review is required by reviewers with write access. Commits must have valid signatures.
To github.com:apache/tvm.git
 ! [remote rejected]     v0.11.0 -> v0.11.0 (protected branch hook declined)
error: failed to push some refs to 'github.com:apache/tvm.git'
%
manupak commented 1 year ago

Looks like the GH branch protection is kicking in , kind of like : https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches#require-pull-request-reviews-before-merging

Alternatively, you could do a PR to the branch (and merge it)?

areusch commented 1 year ago

i have asked asf-infra whether perhaps gitbox has misconfigured our branch protection. i'm not able to view that natively in the GH UI, i believe, so i'm left to deduce what i can from the asf.yaml and associated docs.

areusch commented 1 year ago

okay, traced this down with the help of asf infra. it appears we tried to configure this in .asf.yaml last Oct, but GH API didn't support configuring branch patterns. @driazati filed a ticket about this and they manually configured it for us. there do exist some .asf.yaml in fix_typo and alter/fix_gpu_other_build branches with the pattern indicated, but this is a red herring as according to asf infra, branch protection is only configured from canonical branches - that means:

it seems like the problem here is most likely the requirement of code review on those branches. shall we ask ASF infra to remove that requirement and see if that allows us to cherry-pick existing commits? we would need to probably add something about only pushing reviewed commits in that case; also need to address how to verify CI as well, but likely that's not much change from present as you'd want to look at the post-merge build.

driazati commented 1 year ago

Looks like the GH branch protection is kicking in , kind of like : https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/defining-the-mergeability-of-pull-requests/about-protected-branches#require-pull-request-reviews-before-merging

Alternatively, you could do a PR to the branch (and merge it)?

This should work for what we need to do and avoid the branch protection issues, has this been tried yet? We should keep the release branches protected IMO so they don't get updated without anyone knowing

HLearning commented 1 year ago

@leandron so...

junrushao commented 1 year ago

Any updates?

leandron commented 1 year ago

Hi. I'm sorry, this is a bit frustrating... you can still checkout the v0.11.0 branch to checkout the contents of the release. I'm working through some issues to be able to do the release, as you can see in the discussions above.

I'll update here when I'm able to push on v0.11.0 branch and the proceed with the release vote. In this meantime, I've shared the Release Notes, you can have a look. Is there anything specific you that is blocking you @HLearning?

tqchen commented 1 year ago

Seems like we can still do cherrypick PR and merge that to v0.11.0 instead of directly push. Happy to help on this and drive the release if help is needed

leandron commented 1 year ago

Seems like we can still do cherrypick PR and merge that to v0.11.0 instead of directly push. Happy to help on this and drive the release if help is needed

Yes, absolutely. I'm still getting those messages above, so I'm happy to accept help to have the change to land in the v0.11.0 branch.

tqchen commented 1 year ago

https://github.com/apache/tvm/pull/14057 is now merged with cherry pick.

leandron commented 1 year ago

14057 is now merged with cherry pick.

Thanks! Can you share the list of commands you ran to merge this? Perhaps we should adjust the Release guideline with that? You can see the commands I was running above, when I'm describing the issue I'm facing there.

tqchen commented 1 year ago

I just used git rebase to pick the commits onto top

git checkout [the-commit]
git rebase --onto [branch] HEAD~1

There might be other commands to cherry pick as well. In future perhaps we could try a staging branch before we finalize and put things under the release branch. Let us still if there are still blockers atm

HLearning commented 1 year ago

Hi. I'm sorry, this is a bit frustrating... you can still checkout the v0.11.0 branch to checkout the contents of the release. I'm working through some issues to be able to do the release, as you can see in the discussions above.

I'll update here when I'm able to push on v0.11.0 branch and the proceed with the release vote. In this meantime, I've shared the Release Notes, you can have a look. Is there anything specific you that is blocking you @HLearning?

I need a recent packaged version

leandron commented 1 year ago

The vote has opened for the release candidate: https://github.com/apache/tvm/issues/14129 - Thanks to @areusch and @driazati who helped sorting out the CI issues with the release branch.

leandron commented 1 year ago

I've just published v0.11.0 as latest release here in GitHub, and at this moment I'm dealing with announcing it at Apache and getting some help to update the website.

Here is the link for the GitHub releases: https://github.com/apache/tvm/releases

areusch commented 1 year ago

Unfortunately I encountered a problem when trying to publish the wheels for v0.11.0:

$ venv/bin/python3
Python 3.8.10 (default, Nov 14 2022, 12:59:47)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tvm
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/__init__.py", line 79, in <module>
    from . import micro
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/micro/__init__.py", line 23, in <module>
    from .model_library_format import (
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/micro/model_library_format.py", line 34, in <module>
    from ..relay.backend import executor_factory
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/__init__.py", line 29, in <module>
    from . import prelude
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/prelude.py", line 21, in <module>
    from tvm.relay.transform import ToANormalFormExpr
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/transform/__init__.py", line 20, in <module>
    from .transform import *
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/transform/transform.py", line 30, in <module>
    from ..backend.utils import mangle_module_name
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/backend/__init__.py", line 18, in <module>
    from . import te_compiler
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/relay/backend/te_compiler.py", line 27, in <module>
    from tvm.meta_schedule import is_meta_schedule_enabled
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/meta_schedule/__init__.py", line 18, in <module>
    from . import (
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/meta_schedule/builder/__init__.py", line 22, in <module>
    from .builder import Builder, BuilderInput, BuilderResult, PyBuilder, create
  File "/home/areusch/ws/tlcpack/venv/lib/python3.8/site-packages/tvm/meta_schedule/builder/builder.py", line 21, in <module>
    from typing_extensions import Literal
ModuleNotFoundError: No module named 'typing_extensions'

This is a hole in our CI testing/release process we should plug. We'll need to make a slight tweak to the python install_requires and release a v0.11.1. There aren't any functionality changes expected other than fixing the missing typing_extensions package, so folks don't need to block on this unless they depend on pip3 install apache-tvm.

14244 fixes this in the release branch.