search

huggingface / doc-builder

The package used to build the documentation of our Hugging Face repos
Apache License 2.0
92 stars 35 forks source link

Test build pr outside container #487

Closed Wauplin closed 8 months ago

Wauplin commented 8 months ago

This PR adds a new options custom_container to build PR in the CI in a custom container. By default, workflow runs inside "huggingface/transformers-doc-builder" which is the container needed for transformers and that has all dependencies like torch, tensorflow, etc. However, it takes quite some time (>2min) to initialize the container in the CI given its size. In theory, most libraries don't need this container so we should be able to bypass this step and run directly on ubuntu:latest.

This PR is related to https://github.com/huggingface/huggingface_hub/pull/2140 in which I tested the idea.

As discussed on slack (internal), plan is:

  1. we merge https://github.com/huggingface/doc-builder/pull/487 in its current state
  2. we test to disable custom_container (i.e. custom_container: "") on several repos, including huggingface_hub (already done), datasets, diffusers, hub-docs, etc.
  3. for the repos that needs the custom container, we set custom_container: "huggingface/transformers-doc-builder" in their CI. This should not change anything since it is the default value for now.
  4. Once we fixed the repos in which we think transformers image is needed, we make a new PR on doc-builder to change the default value of custom_container to ""

This way we should be able to make the transition between "all of them use transformers image" to "most of them use a bare ubuntu" without a breaking change in the process.

mishig25 commented 8 months ago

lgtm, why it is still in draft mode ?

Wauplin commented 8 months ago

lgtm, why it is still in draft mode ?

Lol, you're too fast. Just wanted to update description first :smile: Should be ready now :)