Python plugin system

[ginty]

This PR adds an initial Python-based plugin system.

• Origen applications can now function as plugins to other application without requiring any special setup, in short, an O2 plugin is identical to a top-level application
• An application can be built into a plugin package by running origen app package
• DUTs and (sub) blocks can now be imported from a plugin by prefixing the plugin name to the block path
• Added test_apps/python_plugin for testing out the plugin feature
• Added some initial documentation for the plugins feature

See here for the rendered docs - https://origen-sdk.org/o2/guides/plugins/introduction.html

What is currently missing:

Support for generating/compiling source files (patterns, flows, templates) provided by a plugin.

I never really liked the current plugin system from O1, but I guess we could fall back to that for O2 in the absence of any better ideas, however passing on this for now to see if we can think of something better.

I'm not sure if it's possible to support something like O1's:

origen.plugins # => [<App object>, <App object>]

I don't know if it's possible for Python packages to automatically introduce themselves into the environment, I think they might have to be initially imported by name by the application.

Some other miscellaneous additions that came along for the ride:

origen::utility::file_actions

These are a collection of (Rust) functions for inserting and removing lines from files, similar to the Thor::Actions used by the O1 code generators if you are familiar with them.

SubBlock API

Cleaned up the SubBlock API method to be SubBlock(<instance_name>, <block_path>, Options={}):

# old
SubBlock("core0", block_path="core")
SubBlock("core1", block_path="core", offset=0x1000_0000)

# new
SubBlock("core0", "core")
SubBlock("core1", "core", offset=0x1000_0000)

Versioning Changes

Python actually has it's own version format system which is not strictly semver compliant (grrrrr). The version numbers we have been using until now, e.g. 2.0.0-pre2 are semver compliant, but not Python (PEP440) compliant, the closest Python-compliant equivalent is 2.0.0.dev2.

The Rust components are remaining on semver but have now changed to 2.0.0-dev2 and internally versions are still mostly passed around by semver objects. However all version numbers exposed to the user are now converted to the Python format which means that they can remain oblivious to this point of contention.

I added some utility functions in origen::utility::version to convert string versions back and forth between the two formats.

origen build command

Improved how 'origen build' links in a locally compiled version of _origen.

This now creates a symlink (or copies the file on Windows) to the target build (debug or release as specified by the user) at o2/rust/pyapi/target/_origen.so.

When Origen is booted it detects if it is running in a development workspace and if so augments the Python load path to pick up the above file.

The 'origen build' command is now also made available to build the latest pyapi (and CLI) when an app is configured to use a local version of Origen.

Using a local version of Origen within an app

I found Poetry's use of a path reference to quite buggy when switching back and forth between a version and a path reference. The only way to cleanly make the switch was to re-build the app's environment.

I've encapsulated this behavior in the following command which can be used to cleanly switch an app over to a local version of Origen:

origen env setup --origen path/to/my/o2
origen build

The app will now automatically pick up the local build of Origen whenever it is invoked. To go back to a versioned reference to Origen:

origen env setup

Note that this is not required when using one of the O2 test apps since they are permanently configured to use the local Origen build.

origen exec command

Developers and users can mostly remain oblivious to the Poetry tool (Py equivalent of O1's Bundler) when using Origen, however occaisonally it needs to be invoked to run a tool within the context of an Origen application's package environment. Usually for O2 developers this is when running the Python tests via poetry run pytest.

All developers (and users) should now switch to using origen exec anytime they want to use poetry run, e.g. now use origen exec pytest instead.

This gives Origen the chance to setup the user's path to load the local build of Origen when this is run in an Origen development scenario (i.e. in an O2 workspace or an app configured to use a local Origen build).

It may be used for additional automatic environment augmentation in the future and using this instead of poetry run means that we are not coupled to Poetry and could change the underlying tool if a better one emerges in future (though Poetry does still seem to be the best Python package manager currently).

origen new command

origen new dut can now be run without a name argument to a create a default (un-named) dut within an application.

origen new block <name> can now be run within an app to create a new block. Derivative and nested blocks are fully supported too. Nested blocks are improved vs. O1 as they now fully support derivatives and are pretty much identical to top-level blocks with nesting mainly being a code organization concern.

origen env command

Consolidated the existing origen setup and origen update commands into sub-commands of origen env, i.e. you now run origen env setup and origen env update.

[ginty]

Forgot to say that I also removed the app's config/version.toml file. This was duplication with the app/plugin version also in pyproject.toml, so Origen will now read and update an app's version in pyproject.toml directly.