robbecker-wf
commented
5 years ago Is this related to https://github.com/dart-lang/dartdoc/issues/1827 ?
Open robbecker-wf opened 5 years ago
robbecker-wf
commented
5 years ago Is this related to https://github.com/dart-lang/dartdoc/issues/1827 ?
devoncarew
commented
5 years ago Thanks for the report!
Is this related to #1827?
I suspect unrelated, and it may not be related to the dart 1 / dart 2 changeover. We're now verifying more aspects of the generated code, so the slowdown is likely due to performing additional work. That said, 30x slower does sounds larger than expected.
robbecker-wf
commented
5 years ago Thanks for investigating @devoncarew .. any other details I can provide to help pin things down?
evanweible-wf
commented
5 years ago @devoncarew We're getting closer to having all of our packages switched over to dart2 at which point we'll have to move docs generation from dart 1 to 2. We'd like to avoid the large increase in generation times when that happens, so any help here would be appreciated. I also tried the --no-auto-include-dependencies flag but that seemed to have no effect (dartdoc v0.28.1+2).
robbecker-wf
commented
5 years ago Would this be related to having to build a fully resolved AST for all dependencies?
evanweible-wf
commented
5 years ago I would also be interested to know if --no-auto-include-dependencies is working as expected (or perhaps if I'm misunderstanding its intended purpose?) because I still see most of the time spent parsing files in ~/.pub-cache when running with that flag.
smaifullerton-wk
commented
5 years ago Any update on this? Finding that running dartdoc takes 27 min on a fairly small package because of ~/.pub-cache, so we are opting not to run it at all. Thanks!
evanweible-wf
commented
5 years ago @devoncarew @jcollins-g would it be possible to get any sort of update on this? Is dartdoc supposed to run on all dependencies by default? And if so, is there a way to only generate docs for the current package? I thought based on the name that --no-auto-include-dependencies would do that, but no such luck (https://github.com/dart-lang/dartdoc/issues/1943#issuecomment-494479752). I tried that again today on Dart 2.6.0-dev.8.1.
I also tried adding a dartdoc_options.yaml that only includes the root package, e.g.:
dartdoc:
include:
- w_transportBut that only sped up generation by a few seconds.
It would be helpful to know whether this slowdown is just the expected result given the amount of additional work dartdoc is doing now compared to on dart 1, or if this is unexpected.
When packages are published to pub, what command is run to generate those docs? Is it just dartdoc? Are there any analytics on how long doc generation takes for those public packages that we could use to compare to the generation times that we're seeing?
jcollins-g
commented
5 years ago It is an expected result; dartdoc attempts to compute a lot more, including support for external hyperlinks and hyperlinking packages together, and public interface canonicalization. This means dartdoc currently checks all dependent packages to determine where a specific element needs to be documented. It's complicated, because depending on flags Dartdoc might document an inherited or reexported element in many different places.
It shouldn't be as slow as it is, though -- there is a lot of room for improvement.
jcollins-g
commented
5 years ago the question of how dartdoc is run on the pub site is complicated, but here is the main entry point: https://github.com/dart-lang/pub-dev/blob/1148c9180efe14e05d2be09da5697848bee0af40/pkg/pub_dartdoc/bin/pub_dartdoc.dart
jcollins-g
commented
5 years ago FYI, --no-auto-include-dependencies is the default. Dartdoc still needs to read many dependencies even if it doesn't generate docs on them.
Here are some flags you can try to reduce generation time if you're in a CI or other environment where performance is important:
--no-generate-docs -- Do everything except write out the docs and validate them. Because Dartdoc's templating library is really slow, and the doc validator is painstakingly thorough, this can save a LOT of time. This might seem silly (why else run dartdoc?) but if you're not actually publishing the docs but just want early notification of new warnings and fatal errors I strongly recommend this flag. Implies --no-validate-docs.
--no-validate-docs -- This generates docs but doesn't run them through the "check every single link that dartdoc generated isn't broken" check. If you have a broken link validator on your website publish you don't need this unless debugging. When debugging, dartdoc can tell you exactly what line of original code the broken link came from but that's extremely expensive, and our link validator isn't optimized.
srawlins
commented
4 years ago I've reduced the time to generate docs, for dartdoc 0.32.3, but not by 32x. :/
kevmoo
commented
2 years ago Is there anything actionable here?
I agree we should be faster...and maybe set some SLO about speed. But I'm not sure how we act on this issue.
srawlins
commented
2 years ago I don't think there is anything actionable here; I especially don't think we can set a goal to be "as fast as Dart 1 was" (hard to measure, arbitrary).
I like the idea of setting an SLO to generate the flutter docs. Currently I think they take ~13 minutes still on an MBP. I'd love to get that down below 10 minutes. @jcollins-g also has plans for improving dartdoc reference resolution which removes the magical global references, and might dramatically speed up dartdoc generation. Not sure if those plans are tracked here...
TheFriendlyCoder
commented
6 months ago As a new user to Dart, I found it surprising and somewhat concerning when I discovered that running a simple dart doc or dartdoc operation on my small starter project, with about 5 or 6 class files, takes well over a minute (75-80 seconds on average) to generate API docs for it. I'm running a relatively new MacBook Pro (x86 architecture) with an 8 core i9 CPU, 16GB RAM, and all the other typical hardware so performance this abysmal seems downright unusable. The problem gets worse if I use an older computer that I have kicking around the house, which is much lower processing power (intel Atom quad-core system) where it takes many minutes for the documentation to generate.
Given that doc generators for pretty much any other language / toolchain that I've used (Python, Go, Rust, Java, etc.) take seconds if not milliseconds to run on much larger projects, this seems prohibitively slow.
Is there any ongoing investigation or work being done to try and improve this situation? This was the only relevant issue I could find in your Github project.
TheFriendlyCoder
commented
6 months ago For reference, I did read the comment thread here, and if I try some of the workarounds suggested throughout - many of which require the use of the dartdoc utility directly since not all of the options exist in the similarly named dart doc tool - shaved off a bit of time. With all the lookups, cutting the processing time roughly in half, but even that is crazy slow. Output from my time command is: dart pub global run dartdoc --no-generate-docs --no-auto-include-dependencies 35.78s user 2.45s system 146% cpu 26.180 total
srawlins
commented
6 months ago Is there any ongoing investigation or work being done to try and improve this situation? This was the only relevant issue I could find in your Github project.
We are actively working on removing the "universal reference scope" feature, via converting some code to use "doc imports." You can track https://github.com/dart-lang/sdk/issues/50702. I don't think there is a local issue to track removing "universal reference scope." But that should improve performance. Beyond that, there are no plans to investigate performance.
You can also look at the 'type-performance' label on the issue tracker: https://github.com/dart-lang/dartdoc/issues?q=is%3Aopen+is%3Aissue+label%3Atype-performance
TheFriendlyCoder
commented
6 months ago Another interesting test case that I did, I tried using the --include option to see if that would potentially improve performance in some way, but it did not. In fact, if I give this option a random / invalid value, the operation still took over 20 seconds to complete.
time dart pub global run dartdoc --no-generate-docs --no-auto-include-dependencies --no-validate-links --include fubar
dart pub global run dartdoc --no-generate-docs --no-auto-include-dependencies 20.84s user 1.84s system 162% cpu 13.947 totalTo be clear, this is a complete no-op operation because I've asked the tool to generate docs for a non-existent library, and it still takes nearly the same amount of time to complete as a full build operation. :(
TheFriendlyCoder
commented
6 months ago Is there any ongoing investigation or work being done to try and improve this situation? This was the only relevant issue I could find in your Github project.
We are actively working on removing the "universal reference scope" feature, via converting some code to use "doc imports." You can track dart-lang/sdk#50702. I don't think there is a local issue to track removing "universal reference scope." But that should improve performance. Beyond that, there are no plans to investigate performance.
You can also look at the 'type-performance' label on the issue tracker: https://github.com/dart-lang/dartdoc/issues?q=is%3Aopen+is%3Aissue+label%3Atype-performance
Thanks for the reply. I'll keep an eye on them.
TheFriendlyCoder
commented
6 months ago This is probably not the right place for this, but I figured I'd just throw it out there. From a back-end developer point of view, it would be nice even if I could just get the doc strings for my current project exported into a simple markdown or html format, without any cross referencing or cross linking with any external dependencies of any kind, so that I could get fast feedback on the output that is generated (excluding hyperlinks / reference links of course). This would be very helpful for quick feedback when you just want to see if the basic inline formatting you are putting in your doc strings is being parsed correctly.
I'm not sure if such a feature already exists, or if it is even possible. But anything that could be done to make doc generation - at least on small to medium size projects - run in seconds / milliseconds instead of minutes would be of tremendous value from a usability perspective.
TheFriendlyCoder
commented
6 months ago One other anecdote for comparison purposes: I've been using the Hugo static site generator for some doc generation and blogging stuff, and it can parse a markdown file and generate HTML content from it in something like 2ms. Some people use it to generate sites with thousands or tens of thousands of pages in a matter of seconds. When you have tools like this setting the bar so high, it makes the expectations for other similar tools to be at least in somewhat close proximity.
srawlins
commented
6 months ago Another flag that might improve performance (not sure though) is --no-link-to-remote.
TheFriendlyCoder
commented
6 months ago That doesn't seem to have an appreciable impact:
time dart pub global run dartdoc --no-generate-docs --no-auto-include-dependencies --no-validate-links --no-link-to-remote
Found 0 warnings and 0 errors.
dart pub global run dartdoc --no-generate-docs --no-auto-include-dependencies 32.59s user 2.51s system 157% cpu 22.265 total
Summary
Running dartdoc on a private package, I've noticed it takes 32x as long on dart 2. Under dart 1 it just parses the single library whereas under dart 2 it parses ALL of the packages before it even generates any docs.
15s - dartdoc on dart 1.24.3 (dartdoc version 0.9.14-dev) 8m 17s - dartdoc on dart 2.2.0 (dartdoc version 0.28.1+2)
Dart 1
Dart 2