theroyallab / tabbyAPI

An OAI compatible exllamav2 API that's both lightweight and fast
GNU Affero General Public License v3.0
604 stars 75 forks source link

[REQUEST] Make docker build action faster #186

Open bdashore3 opened 2 months ago

bdashore3 commented 2 months ago

Problem

The docker action doesn't cache anything when building. This causes the build times to be over 10 minutes for every push.

Solution

Use an action to cache the intermediate build steps.

Alternatives

No response

Explanation

Doing this makes the image build build faster. This most likely requires a third-party build step.

Examples

No response

Additional context

No response

Acknowledgements

SecretiveShell commented 2 months ago

one of the biggest things slowing the build is the fact that the .dockerignore file has been moved to the docker folder, meaning that the entire git history and a bunch of other stuff not needed is copied into the container. running git mv on the file would probably cut off a minute or two from the build and reduce the overall file size

bdashore3 commented 2 months ago

Yep, merged that in from your PR. But the most important thing to significantly cut down on build times is to have a cache of some sort with actions. There's probably something out there.

SecretiveShell commented 2 months ago

https://docs.docker.com/build/cache/backends/gha/ seems to be a configuration option for the system we already use. @AmgadHasan can you provide any insights on this?

AmgadHasan commented 2 months ago

https://docs.docker.com/build/cache/backends/gha/ seems to be a configuration option for the system we already use. @AmgadHasan can you provide any insights on this?

There is a cache size limit of 10 GB per repo. TabbyAPI image is ~ 5GB in size so the cache limit will be reached quickly.

https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/caching-dependencies-to-speed-up-workflows#usage-limits-and-eviction-policy