search

dart-lang / pub

The pub command line tool
https://dart.dev/tools/pub/cmd
BSD 3-Clause "New" or "Revised" License
1.04k stars 226 forks source link

[API] full dependency graph in `package_config.json` #3795

Open dcharkes opened 1 year ago

dcharkes commented 1 year ago

We'd like to do build for native libraries, and these libraries might depend on each other. Therefore we need to know a dependency graph (or topological sort) to schedule building the libraries in the correct order.

For building native libraries the prototype has opted to rely on parsing .dart_tool/package_config.json rather than invoking package:pub programmatically. Currently, .dart_tool/package_config.json contains a resolved package version, but it does not contain a dependency graph between those packages. This means we cannot do the builds in the right order.

An alternative solution might be to call the pub solver programmatically instead of relying on the package config as the API layer between pub and the native asset builds.

Context:

@jonasfj @sigurdm We discussed this before, and I believe we preferred using JSON as the API rather than programmatic invocation, but I can't find any meeting notes/docs.

Note that if we decide to go for build_dependencies, we would need a different topological sort based on those dependencies rather than the "main" dependencies:

Without access to a dependency graph or topologic sort, packages doing native builds should not depend on any other package doing a native build. This effectively means we only support packages downloading dynamic libraries from from a CDN.

sigurdm commented 1 year ago

I think the best way currently is to run dart pub deps --json. You should be able to infer the graph from there pretty easily.

dcharkes commented 1 year ago

That requires us to do an extra process invocation, this could slow down things.

A process invocation is likely slower than just a programmatic source code invocation. In which case I'd need to be able to have the pub-solver as a dependency.

And a programmatic invocation is likely slower than reading a JSON file, because we run some of the pub code twice. Once on pub get and once on the programmatic invocation.

But I can use dart pub deps --json for the time being. 👍

jonasfj commented 1 year ago

But I can use dart pub deps --json for the time being.

I think that's a good way to get started.

We can always improve performance by:

But I would prefer not to unnecessarily increase API surface until we're sure this is the path we want to commit to.

dcharkes commented 1 year ago

Using the workaround adds a 250 ms slowdown to flutter app builds: