Scraping examples: works locally but not on docs.rs

[killercup]

I tried to add scraped examples to bevy's API docs in https://github.com/bevyengine/bevy/pull/9154 but it did not work on docs.rs.

If you run cargo +nightly doc -Zunstable-options -Zrustdoc-scrape-examples --open locally and search for just_pressed, you'll see scraped examples.

This repo is about finding out what's happening.

[killercup]

Bevy's crate is part of a workspace, where the main crate contains all the examples but is just a facade reexporting a bunch of bevy_* crates.

So in this repo, the layout is

test_61a7cd7f28e0
├── test_40a48f47864a
└── test_7018a98a70c0

[killercup]

Experiment one: Prove that there are no scraped examples by default.

This is version 0.1.0-nodocsrssettings.

Result: No examples scraped.

Locally: Examples scraped!

[killercup]

Experiment two: Add config described in rustdoc docs to top level crate.

This is version 0.1.0-withdocsrssettings.

Result: No examples scraped.

Locally: Examples scraped!

[killercup]

Experiment three: Add config described in rustdoc docs also to the two sub-crates.

This is version 0.1.1-withdocsrssettings.

Result: No examples scraped.

Locally: Examples scraped!

[killercup]

Experiment four: Add explicit config for examples

[[example]]
name = "status"
doc-scrape-examples = true

This is version 0.1.2-withdocsrssettings.

Result: No examples scraped.

Locally: Examples scraped!

Bonus: The build log says

[INFO] [stderr]     Scraping test_61a7cd7f28e0 v0.1.2-withdocsrssettings (/opt/rustwide/workdir)
[INFO] [stderr]  Documenting test_61a7cd7f28e0 v0.1.2-withdocsrssettings (/opt/rustwide/workdir)
[INFO] [stderr]     Finished dev [unoptimized + debuginfo] target(s) in 0.45s

[killercup]

To check that it doesn't work locally for sure, I ran this

cargo +nightly rustdoc --lib -Zrustdoc-map -Z unstable-options --config build.rustdocflags=[\"-Z\", \"unstable-options\", \"--emit=invocation-specific\", \"--resource-suffix\", \"-20231012-1.75.0-nightly-e20cb7702\", \"--static-root-path\", \"/-/rustdoc.static/\", \"--cap-lints\", \"warn\", \"--extern-html-root-takes-precedence\"] --offline -Zunstable-options --config=doc.extern-map.registries.crates-io=\"https://docs.rs/test_61a7cd7f28e0/0.1.2-withdocsrssettings/x86_64-unknown-linux-gnu\" -Zrustdoc-scrape-examples -j6  -Zunstable-options -Zrustdoc-scrape-examples

which is the sanitized build command from https://docs.rs/crate/test_61a7cd7f28e0/0.1.2-withdocsrssettings/builds/938672

[killercup]

The main difference I can think of between docs.rs and local is that docs.rs get .crate file which does not contain the workspace member code, while locally I have the repo checked out of course.

[killercup]

Two more tests:

• --no-deps: still works locally
• removing the path dependencies locally and using the published ones

[killercup]

Experiment five, after talking to @rust-lang/docs-rs team: Use crates.io deps only

This is version 0.1.3-withoutpathdeps.

Result: [No examples scraped.](https://docs.rs/test_61a7cd7f28e0/0.1.2-withdocsrssettings/test_61a7cd7f28e0/struct.Foo.html

Locally: No examples scraped! Finally.

[GuillaumeGomez]

Do you have the final working Cargo.toml by any chance? From our discussion, I remember that using path for dependency allowed to fix the bug locally but can't reproduce it. Here what I changed:

index d192fd6..22706ba 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -7,8 +7,8 @@ description = "testing scraping examples"
 license = "MIT"

 [dependencies]
-test_40a48f47864a = { version = "0.1.1-withdocsrssettings" }
-test_7018a98a70c0 = { version = "0.1.1-withdocsrssettings" }
+test_40a48f47864a = { path = "crates/test_40a48f47864a", version = "0.1.1-withdocsrssettings" }
+test_7018a98a70c0 = { path = "crates/test_7018a98a70c0", version = "0.1.1-withdocsrssettings" }

[killercup]

Anything up to and including this commit worked for me locally: https://github.com/killercup/test-rustdoc-example-scraping-across-workspace-on-docrs/commit/dfb942ce6837a946797708e8a36f6b3104bc6359

Let me try again with a newer nightly… Yup, still true with rustc 1.75.0-nightly (1c05d50c8 2023-10-21)

[GuillaumeGomez]

I have the following Cargo.toml:

[package]
name = "test_61a7cd7f28e0"
version = "0.1.3-withoutpathdeps"
edition = "2021"
authors = ["Pascal Hertleif"]
description = "testing scraping examples"
license = "MIT"

[dependencies]
test_40a48f47864a = { path = "crates/test_40a48f47864a", version = "0.1.1-withdocsrssettings" }
test_7018a98a70c0 = { path = "crates/test_7018a98a70c0", version = "0.1.1-withdocsrssettings" }

[package.metadata.docs.rs]
cargo-args = ["-Zunstable-options", "-Zrustdoc-scrape-examples"]

[[example]]
name = "one"
doc-scrape-examples = true

[[example]]
name = "two"
doc-scrape-examples = true

I have this version:

$ cargo --version
cargo 1.75.0-nightly (8eb8acbb1 2023-10-17)

But when generating docs with cargo doc -Zunstable-options -Zrustdoc-scrape-examples, I get:

What am I missing? ^^'

[killercup]

Aha, I see now: You're missing the workspace definition!

[workspace]
members = ["crates/*"]

Another interesting discovery :D

[GuillaumeGomez]

Bingo! That's a lot of requirements. ^^'

[GuillaumeGomez]

Noting for myself: I came back to this: with the [workspace] definition, whether it is from local crates with path or from crates.io, I have the scraped examples.

[epage]

I did some debugging on this.

Mind creating an issue against cargo?

When scraping examples from a package, cargo only asks for examples that use APIs from specific crates, presumably for performance reasons

        // Only scrape example for items from crates in the workspace, to reduce generated file size
        for pkg in cx.bcx.ws.members() {
            let names = pkg
                .targets()
                .iter()
                .map(|target| target.crate_name())
                .collect::<HashSet<_>>();
            for name in names {
                rustdoc.arg("--scrape-examples-target-crate").arg(name);
            }
        }

This is filtered to workspace members. Really what is needed is filtering for public dependencies which has recently been revived (rust-lang/rfcs#3516).

Path forward

• Long term: switch this to filtering on public dependencies
• Short term: decide whether to keep filtering by workspace members or take the perf hit and do no filtering. We should consult Will on that.

[GuillaumeGomez]

Doing it with the extra information you provided.