Remote app execution mode

[ginty]

As you know, O1 has two ways of executing:

• Globally, outside of an app workspace
• App mode, inside of an app workspace

It has already been discussed elsewhere that for O2 we will probably add a 3rd:

• Origen development, which would open up some commands related to building/deploying/testing Origen itself (more useful in O2 due to the multi-lang env)

And now I would like to discuss the addition of a 4th:

• Remote app mode

In this mode Origen would act like it was inside a particular application's workspace but while it is actually in some other directory outside of the application.

The relationship between the CLI and the application would be established by one of two ways:

• A file in the current directory or its ancestors, e.g. .origen_app which is either a sym link to the application directory or a file containing the path to it
• An environment variable, ORIGEN_APP_DIR

I think order of precedence for establishing what the current app is would probably be: Being in an app workspace -> ORIGEN_APP_DIR -> .origen_app and in some cases probably a warning would be emitted (e.g. when a .origen_app is found but is being overruled by the ENV variable).

The intended use case for this is when Origen is being used as a platform for creating commands and supporting functionality within a wider 3rd party tool environment. e.g. When it is used in design/verification environments where an Origen application is created and managed by one or two engs, it is then checked in to the wider environment and then all users get access to custom origen commands without really caring or being aware of how they were built and made available. @chrishume, thinking about your use cases here.

As part of this it will mean that Origen execution will need to be well de-coupled from whatever is the pwd, so it will also resolve weaknesses in O1 where some (many?) commands didn't work properly unless the pwd was the app's root dir.

I will probably start on this next week, so keen to hear any comments/suggestions/objections before then.

Thanks!

[coreyeng]

While were talking about the CLI, do you think it will remain in Rust long term? In O1, we had the ability to add/extend commands from the application or (eventually) plugins which isn't really possible considering the CLI is now compiled. We could wrap in some interface into the Rust side, but then we'll need to fill in a lot of the things clap does for us, like the command organization, and help dialog.

I wonder if it might be better to move the CLI to Python, which can dynamically build up commands and then have a bunch of helpers on the O2 side to decouple it as much as possible from the frontend. So hings like origen t will start on the Python side, but then immediately go into passing the command back to Rust, which would amount to running the same target code that is run today, just the starting point is in the frontend instead.

Another option would be roll our own dynamic-CLI, which I don't see that Rust has something currently. So, we might be filling in a gap there. This would keep the CLI in Rust, but the actual command 'building' would still be dynamic. Basically rewriting something Python can already do, in Rust.

Or is there another alternative I've not seen?

(I also know this is kinda hijacking this issue, but it relates to the CLI long term. I can open a separate issue to move the discussion there if you want)

[ginty]

Hi @coreyeng,

Definitely the CLI should stay in Rust, I'm 100% opposed to making Origen boot dependent on a working Python environment - that was painful and bad PR for O1 in many cases and nothing I've seen from Python suggests that it is going to be better at that than Ruby. There are also a number of potential use cases where Python shouldn't be required at all, format conversions being the obvious one, but also new app building and probably others will emerge as we add more standalone Rust functionality.

That said, I agree that we need to support app commands via Python, but I haven't thought through exactly how, but I would hope we can do it with as little effort as possible.

I did a quick Google and found this thread which suggests that Clap can at least be configured to allow you to proxy out to something like Python for command calls that don't correspond to a known Clap command - https://users.rust-lang.org/t/using-external-subcommands-with-clap/10753

I can definitely pick this up though and take a stab at something while working on this remote app feature.

[coreyeng]

Thanks! I was putting off doing much with #88 (or the CLI in general) because I wasn't sure how the it would look in the end. If we are keeping it in Rust that narrows it down quite a bit. I really like the clap crate too. Its really powerful and pretty easy to work with.

That external command looks promising though! I obviously missed that during my own research (admitting, I didn't put much time into it). I think we could, in one way or another, get everything I can think of out of that. We'd just need to do most of the error handling and help dialog ourselves. Though, we can probably write some macros to help with that. I'll take a closer look at this as well.

I guess an advantage with this too is we'd force an API on the user to extend the commands. So, no more usurping commands or having load order issues like we could have in O1.

[ginty]

Hi @coreyeng, OK cool and if you're interested in working on the app command system please just go ahead - I'm not too fussed and will find other things easily enough.

[coreyeng]

Hi @ginty, Okay! I look into that a bit. Big thing was knowing that we won't be moving the CLI into the frontend so I can go from there. Thanks!