search

Origen-SDK / o2

MIT License
4 stars 0 forks source link

remote file translator * IP-XACT works in a limited way #42

Closed info-rchitect closed 4 years ago

info-rchitect commented 4 years ago

Have not mastered BeautifulSoup yet but I think this should get the conversation started on how remote files should be handled. Here I have used direct delegation but do not translate anything by default. The user must call origen.app.translate to initiate the process.

info-rchitect commented 4 years ago

@ginty Should address blocks be modeled as sub_blocks?

info-rchitect commented 4 years ago

Unit tests are passing, I think it can be merged after we get a few reviews. There are enhancements to the PYAPI that are needed to truly create a full IP-XACT model. Until those are done, I don't think it makes sense to write the code that exports Python.

ginty commented 4 years ago

@ginty Should address blocks be modeled as sub_blocks?

No, they should be modeled as address blocks!

ginty commented 4 years ago

Thanks for picking this up @info-rchitect!

So this is going down the path of importing directly into a model, however should we support that paradigm at all?

My concern with it is that it encourages people to write it into their applications, making the import a run-time operation which works fine initially for a small amount of test data, but then things grow and eventually they complain that Origen is slow to start and/or run, and at that point the run-time import is so baked into their app it is difficult to change it.

I think we would be better off not offering that at all and instead this importer should focus only on taking the IP-XACT and writing out the equivalent Origen .py files.

That enforces it to be done offline and the resultant output will benefit from Origen's lazy loading to keep things snappy.

I also wonder if writing to some kind of binary format to facilitate loading it directly into Rust is something we may also consider in future.

info-rchitect commented 4 years ago

I definitely want to write py or serialize the data. Just need access to the full PYAPI to continue. Some attributes are not currently available, pls correct me if I am wrong.

info-rchitect commented 4 years ago

Can you elaborate on the performance concerns? The current implementation only runs when the user chooses to import, it does not do so at boot.

ginty commented 4 years ago

Can you elaborate on the performance concerns? The current implementation only runs when the user chooses to import, it does not do so at boot.

How quick do you think it would be to import a few thousand regs like this? I've no idea but I guess it would be noticeable, and in fact a lot of the complaints about slow boot time/target load time on O1 were from you due to doing similar things.

In NXP our auto apps also had similar complaints due to importing a large amount of data.

So my preference is that we prevent users from shooting themselves in the foot like this on O2.

Understood that that the API to target is not complete/frozen yet, but it should be in a few weeks, but there is nothing stopping you from importing the subset that is currently supported in the meantime.

ginty commented 4 years ago

I definitely want to write py or serialize the data. Just need access to the full PYAPI to continue. Some attributes are not currently available, pls correct me if I am wrong.

In fact you're using the same API whether you are invoking it at run-time like currently or else writing it to a file, so its the same either way and it should have no bearing on the implementation direction here.

info-rchitect commented 4 years ago

Ok if I serialize to rust, do you have any ideas on the format? Serializing to Python is easy, will start that today. Here is something I found: https://github.com/serde-rs/serde

info-rchitect commented 4 years ago

https://docs.rs/savefile/0.4.2/savefile/

ginty commented 4 years ago

I have no idea how to serialize to Rust, was just throwing that out there as our likely most performant future path.

I said binary since in principle I envisaged writing out to the Rust memory layout directly, then Rust just picks up a blob from a file and loads it into memory.

Definitely worth looking into that serde crate, I've seen that mentioned quite a lot, so its obviously good and it would be good for us to get an understanding of it. Savefile looks good too.

It would mean that importing in Rust instead of Python would probably make this route much easier of course.

ginty commented 4 years ago

Maybe a hybrid approach would work well:

Use your current approach of importing into a model at runtime (in Python) - but we obfuscate that enough to keep the underlying APIs well out of user space.

Then we add a dut.to_python() and dut.to_rust() methods, both of which could be implemented in Rust. Doing the Python writing in Rust would also be a good vehicle for you to learn it.

info-rchitect commented 4 years ago

I agree on the hybrid approach. Can you check discord as I had a question regarding the two loaders for sub blocks and everything else

info-rchitect commented 4 years ago

@ginty this now works with the latest master changes regarding determinism. I would vote to merge and open new PRs regarding export features and other modeling APIs.

info-rchitect commented 4 years ago

@ginty I take that back, it seems I picked up a lot of unrelated changes when I pulled from master just now. Can you see if this is desired?