search

m-labs / nmigen

A refreshed Python toolbox for building complex digital hardware. See https://gitlab.com/nmigen/nmigen
https://nmigen.org
Other
650 stars 55 forks source link

Bikeshed: Packaging existing RTL code with nmigen wrapper #258

Closed Fatsie closed 4 years ago

Fatsie commented 4 years ago

I would like to make nmigen wrappers around existing RTL code and package them for distribution with pip, conda and similar tools. Target is also that a user can use the wrapper class as any other submodule without need to take care of the RTL source code files.

An example alpha implementation (without packaging support) can be seen in ao68000.py used in my latest SnowWhite test tape-out.

In order for the user to not have to add the RTL files herself you can see that the files are added to the platform during elaboration. Users could have more than one instantiation of the wrapper class in their circuit. So in order to avoid errors because the files get added twice I use an empty .ao68000_added file as stamp. Maybe this could be supported natively by platform for example by a add_files_tag() or add_files_uuid() method. Let me know what you think. I'll try to implement what is proposed here.

Second part of the bikeshedding is the packaging itself; this is not nmigen specific so ignore if deemed not appropriate here. What is good pythonic way of distributing the RTL files with the nmigen wrapper ? I have been thinking of copying the RTL files along with the python code and use the module path to load the files from there. Another solution is to convert the RTL files into a python file where the files are presented as strings. Any comments ?

whitequark commented 4 years ago

Maybe this could be supported natively by platform for example by a add_files_tag() or add_files_uuid() method.

We could make it a no-op to add a file with the exact same name and content. After all, what's the harm of it? Also, I would say that you should (by convention0 place all of the files into a directory with the name of the current module; special variable __name__ contains it.

Second part of the bikeshedding is the packaging itself

I think you want to use pkg_resources.

whitequark commented 4 years ago

As an aside, there's currently no support for setting include path and definitions. You're going to hit at least the latter, possibly the former too. Some infrastructure would have to be added to support that. Definitions are fairly easy, but include paths would have to be checked to ensure they're relative.

Fatsie commented 4 years ago

Maybe this could be supported natively by platform for example by a add_files_tag() or add_files_uuid() method.

We could make it a no-op to add a file with the exact same name and content. After all, what's the harm of it? Also, I would say that you should (by convention0 place all of the files into a directory with the name of the current module; special variable __name__ contains it.

OK, given that file names are typically named after modules found in the source file, you likely still have a problem if two files have the same name in different modules.

As an aside, there's currently no support for setting include path and definitions. You're going to hit at least the latter, possibly the former too. Some infrastructure would have to be added to support that. Definitions are fairly easy, but include paths would have to be checked to ensure they're relative.

I have been looking mainly at open source FPGA retrocomputing code. This is a lot of VHDL and simple structured verilog code. So likely I don't need this support at first.

Also I would not mind that source code would need to be patched in order to be distributed with a nmigen wrapper like a lot of program source code need to be patched to distribute as rpm or deb.

whitequark commented 4 years ago

OK, given that file names are typically named after modules found in the source file, you likely still have a problem if two files have the same name in different modules.

That's true, but the purpose here is to make sure that, when looking at the build directory, it would be immediately clear where the Verilog/VHDL files come from.

whitequark commented 4 years ago

Seems like we have an agreement, so closing this issue. Feel free to comment if you have more questions.