Metal integration - Githubissues

In keeping up with my usual, I said I'd not do any more impossible-to-digest-PRs just to do precisely that yet again. I started with what I thought would be pretty straightforward - build global commands into O2 and allow it to function partially as a "tool distribution platform" - idea being that any custom commands or scripts can leverage anything in O2 and distribute easily to any other O2 users, or globally in a site-managed installation.

As I did this, I started to prototype some more complex features, like the global app and command extensions. As I did this, I was having to rewrite more and more of the existing pieces to get the flexibility I wanted. I also started adding tests to make sure I wasn't going backwards then, as I started looking at corner cases, had to rewrite even more. Obviously, the changes themselves in this PR got away from me but positive side is I was able to make some decent progress, enough to warrant a push into master. A higher level summary of what's in this PR:

Main Updates: • Support loading Python plugins automatically upon bootup, similar to what O1 does. • Added more support for custom commands, including auto-loading from plugin sources. • Commands can also be added from auxiliary sources, that is, commands that don't reside in an app or plugin, added from origen.toml. • Support command extensions. Extensions can add options to existing commands and run callbacks before, after, and during clean up, for the target command. • Made python_no_app a workspace. Still no app, but now has a pyproject associated with it.

Smaller Updates: • Switched LDAP yet again as old-new one seems to be unavailable now. Still may have some intermittent connectivity issues, since I'm using a free version, but been working alright so far. • More config support (e.g. loading commands file, loading additional configs) • Bumped clap version (and whole host of changes w.r.t. to that and some look ahead for v4) • Add core_app at origen.core_app, allowing the origen namespace to also function as plugin, for adding blocks like SWD, Arm Debug, Memory Controller, etc. • Bump minimum poetry version • Reserved target, mode, no target, verbosity, verbosity keywords, and help options and shortcuts. There are not overridable and will generate soft-errors (won't kill the process) if encountered ○ Reserved even in no-app context to avoid confusion - just reserved them across the board. ○ Mode isn't available as 'm'. I have never used -m explicitly and don't wish to take up a permanent short spot that, in my experience, is seldom used. • Removed short name for verbosity keywords. Now a long name alias. So, '-k' is free and shortcut now is '--vk'. Otherwise, works the same. • The origen dev commands (build/fmt) are now under develop_origen, aliased to origen. E.g., origen origen build to get to the previous origen build command. • Made some of the CLI test methods available for users, under origen/helpers/regressions/cli. Stuff here may be useful for users to test their own CLI stuff with pytest.

Some caveats and my plans to address: • I removed commands that I haven't tested yet, outside of fmt and interactive. As I get back to doing more app-related stuff, I plan to refactor and add tests where I can. Most of the current commands have at least some basic tests. • There are some soft-errors if a py-project isn't available. This is a by-product of using global plugins, similar to what O1 can do. Other than some red printed the screen, it works fine. I need to revisit the release process though and see what hooks I need to add for a true "global installation with plugins". When I get to this, this issue should go away. • The "mode" option panics, purposefully because I haven't done any testing for it. However, I would like to bring up moving this into something else. mode seems woefully generic, at least for what we've been doing. I'd prefer something like "enable/disable production checks", possibly even allowing for levels of production checking. ○ I think there's also a "sim" mode floating around in O1 but, to my knowledge, this hasn't been used. With our new multi-target support, I'd rather this be moved into the sim target, or something like that. ○ The mode option is present since I already had the tests, but will be updated accordingly if we go in a different direction.

I have a FOR_PR tracker I've been using for things relevant to this PR. At this stage though, most are just cleanup, refactoring, or missing tests. None of these though I think would cause any significant rewrite.

Origen-SDK / o2

Metal integration #190