Nightly Build for LMDeploy

InternLM / lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

https://lmdeploy.readthedocs.io/en/latest/

Apache License 2.0

3.11k stars 280 forks source link

Nightly Build for LMDeploy #1828

Open zhyncs opened 6 days ago

zhyncs commented 6 days ago

Motivation

Hi all

I referenced this https://github.com/InternLM/lmdeploy/blob/main/.github/workflows/cuda11.8-whl-release.yml and implemented a nightly build https://github.com/zhyncs/lmdeploy-build.

Currently, the release frequency of LMDeploy is once every 2 weeks to a month. During this process, if there are new features or bug fixes and users want to use the new whl package, they need to compile it themselves. This unnecessary complexity for users prompted me to do nightly builds in order to make it easier for users to install the latest version of whl. Cheers.

Current latest main https://github.com/zhyncs/lmdeploy-build/releases/tag/fd0cefb DeepSeek V2 by @grimoire https://github.com/grimoire/lmdeploy/tree/torch-deepseek-v2 https://github.com/zhyncs/lmdeploy-build/releases/tag/e21eaa0

Related resources

No response

Additional context

No response

zhyncs commented 6 days ago

Every day at 1:30 a.m., a new release for the day will be generated.

https://github.com/zhyncs/lmdeploy-build/blob/f30c351f815de5cee089df9ccf58f3edd870c502/.github/workflows/cuda11.8-whl-release.yml#L8-L9

josephrocca commented 6 days ago

Awesome! Thank you for this, it's very helpful. I'm not very familiar with Github Actions, but is it possible to also publish a nightly Docker image too? E.g. with tag like v0.4.2-nightly-e21eaa0 or something.

I was going to ask if you could release an alpha/prerelease before the next tagged version so I could test it before the official release, but it'd be even better if I could test an official Docker build after each important pull request so I can give feedback before the new changes make it into a new tagged version.

Also:

I think it'd also be great if the README had a link to the Docker hub page: https://hub.docker.com/r/openmmlab/lmdeploy/tags
It would be good to know what changes are included in the v0.4.2-post0, v0.4.2-post1, v0.4.2-post2 releases, since they don't have associated Github tagged releases. So it's hard to find what changes they include.

zhyncs commented 6 days ago

The frequency of Docker version updates is currently sufficient with one update per tag, in my opinion. You can use the latest Docker tag in combination with lmdeploy-build for the latest whl. If you are not a developer, this should be more than enough. Cheers.

josephrocca commented 6 days ago

Okay, that make sense, thanks! It would be good to be told a couple of days before the next tagged version will be released so I can test it before, just in case there are bugs that your team missed.