A clang-build tooling API

[NOhs]

So as discussed previously, I think it would make sense to not only have clang-build as a script/app that you can run, but something, you can also integrate into your project/workflow. The clang-build script itself should then use that API to provide the current functionality. I think we should play around with a simple first version, something like that:

Project discovery

• get_targets(folder_name) -> tree of targets

Project creation

• create_target?

Project inspection

• target class should have public fields:
  • Name
  • Type
  • List of targets it depends on
  • Attributes set in clang toml
  • Output name
  • Output path

Should this be read/write? Then we could potentially have a gui to set toml files.

Target compilation

• compile_targets(*targets), should have some way to report progress to the outside world, maybe we could even remove the hard-coded progress output in the actual target class definitions etc.

CLI interface

• Since some projects might not use Python, it might make sense to mirror these functions to the command line so that other programs can just parse the command line output, e.g. we could have something like:

clang-build --function get_targets "/home/my_project"

would print

target_0_name
    dependency_1_name
    dependency_2_name
        dependency_3_name
        dependency_3_name
target_1_name

with the names from the toml files etc, as usual.

and for all API functions this could automatically get wrapped (though we have to make sure that the documentation is still good).

So it is obvious that we have most of the stuff anyways in some form in our project, but I think cleaning it up to such a clean interface might help in the long run. Please feel free to edit as needed (I might have forgotten some things).

[NOhs]

Something we need to change in the target class imho is to keep the separation between target and dependency options, currently we merge everything which is not nice for browsing the project. I think internally we can merge, but need to keep two separate public variables that display what is coming from this target, and what is coming from dependencies.

[GPMueller]

Isn't this somewhat duplicate to #87 (or a subset)?

I agree that a target's own flags etc. and those of it's dependencies should be separated.

If I understand you correctly, you want to have a proper API, which would enable users to create projects, which others could not use as subprojects because clang-build would not know how to use the user-defined scripts? I would like to avoid this situation, if we can (see also my comment on #87)

[NOhs]

No, it is more that I would like clang-build to be easily integratable for example into software like vscode. For this we need to have an API that can be used to display a list of discovered targets, show their properties, select one and compile it etc.

[NOhs]

Kind of like front-end and back-end, and the CLI is just one front-end.

[GPMueller]

OK, I would also really like to see this kind of features. Is there any way to automatically generate a CLI from a given set of functions?

Does the following describe the kind of tooling API you wish to create?

The Python API would provide the ability to

• set environment (build type, etc.)
• create a project from a given folder or file
• list targets
• trigger build stages
• query build completion percentage, get messages, etc.

but not

• create a custom project or target (e.g. for building documentation)
• manipulate the build stages (e.g. to generate sources)
• change project- or target-specific variables

The latter parts would then be a separate build-config API, related to what is described in issue #87.

[NOhs]

The latter parts would require new features, whereas the former ones only require refactoring, I think (mostly). So yes, for a first step I am not considering the latter parts. Though it might not hurt to keep them in mind when refactoring. I actually already started in the new branch: api_first_try.

[GPMueller]

I believe we can add a first tooling API on PR #110 with little extra effort.

• project discovery: available via clang_build.project.Project.from_directory
• project creation: can create empty project and add targets
• project inspection: available and easily extended by more properties
• (maybe?) environment: TODO
• compilation and linking / build stages: TODO
  • (maybe?) ability to trigger a specific stage
  • completion percentages
  • indication of current stage
  • get build messages
• CLI: TODO

IMO compilation should probably be done via the project, not via directly calling target functions.

Note that we currently have the clang_build.target.TargetDescription class, which should have the necessary information about a given target for the tooling API. I believe the tooling API will not need to use the clang_build.target.Target class.

[GPMueller]

As suggested by @NOhs and demonstrated on the feature-child-process branch on the vscode-clang-build extension (requires the feature-py-api of clang-build corresponding to PR #110), a python -i child process can be used instead of requiring a CLI.

This allows us to use the Python API for tooling, meaning all the aforementioned points are covered by PR #110. While the API is certainly not yet ideal, we can create corresponding follow-up issues.