Add support for use flags

[olofk]

Enable conditional expressions dependent on user-specified flags. Will use something similar to gentoo ebuild-style syntax. e.g xilinx ? (xilinx_memory.v) !xilinx (generic_memory.v)

Design considerations:

Ideally cores should be able to set requirements on use flags for their dependencies (local use flags), but this will require a new dependency manager as the current one isn't flexible enough. For that reason we will start with use flags set on the command-line that will affect every core in the dependency tree (global use flags). This way the flags will be evaulated before the satsolver starts

How should use flags be set?

• Command-specific switches e.g. fusesoc run --target=sim --i_am_a_flag ?
• A special flag switch e.g. `fusesoc run --target=sim --flags="+i_am_a_flag -this_flag_is_off" ?
• Something else?

How should flags be defined?

• A new flags section?
• As a new type of parameters?
• Skip this for now and just accept every flag set on the command line?

[a-gibson]

I've started looking at the feasibility of using FuseSoC and this feature is very important. Due to cascading dependencies, one of our libraries that is most depended on has multiple implementations (VHDL architectures) for the entities. For example, use this architecture for Virtex4 or this architecture for StratixV. Currently we selectively include the required files but can't quite replicate that behavior using the existing implicit tool_ or target_ flags.

Command-line switches is certainly a very helpful start. Need to be able to distinguish them from other flags that can be passed in (e.g. backendargs, so something like this may be suitable? fusesoc run --target=sim --flag="+i_am_a_flag" --flag="-this_flag_is_off" Each occurrence of --flag adds to a list which is easy to work through.

But in the long term in would be better to have this information in the project repository and not the IaC repository, so moving the information to the core file would make sense. I've only have a limited understanding of the underlying code, so don't have a full appreciation for how complicated this suggestion is, but would either of the following be feasible?

flags: # this is a new 'section' I've just made up
  family: virtex4
  vendor: xilinx

filesets:
  rtl:
    files:
      - ram.vhd # entity
      - "family_virtex4? (ram_virtex4.vhd)" # architecture

targets:
  default:
    filesets: [rtl]

or

flags: # this is a new 'section' I've just made up
  family: virtex4
  vendor: xilinx

filesets:
  rtl:
    files:
      - ram.vhd # entity

  rtlVirtex4:
    files:
      - ram_virtex4.vhd # architecture

targets:
  default:
    filesets: [rtl, "family_virtex4? (rtlVirtex4)"]

[olofk]

Well, the good news is that use flags is top priority for the next release, and having yet another user waiting for it makes it more likely to get started. The other good news is that an initial implementation that covers most of the use cases is quite near and matches quite well what you outline above.

My current thinking is to do this in four steps.

1. Add CLI support for setting flags. The plumbing is pretty much in place already so it's mostly about enabling access to them. As you note above they need to be distinguished from the target-specific compile/run-time options. I came up with six different syntaxes for this a couple of years ago. Should probably write them down somewhere to get some feedback. At this point there would be no checking of flags, so you can just set arbitrary ones. Both your examples above should then work (without the flags section)

2. Add support for setting flags with default values in core files. This will open up for a whole bunch of new policy decisions, so let's go into more detail when we get there, but I also envision some sort of flags section.

3. Add variable substitution, so that you can set mem_size=50 as a "flag" and all occurences of $mem_size (or %mem_size%, {mem_size}, I don't know) will be replaced.

4. Local use flags. This would mean that a core can add a dependency on another core and require a certain flag to be enabled or disabled. This will require a whole new dependency handler though, which is why I'm pushing this to the future. I believe however that the three former items can be implemented without adding any complexity the day when local flags are supported.

[olofk]

I just pushed a useflags branch. Please try it out to see if it works as intended. Note that it is likely to change. Just wanted to get something out while we're on the topic

[a-gibson]

Incorporating this into the next release would be fantastic!

Thanks for creating the branch. Of my two examples, the first didn't work and after looking at the diff of the branch this was expected because the documentation states files is a "List of File" and File is a simple string and not of type class String. But the second example worked, which is still very helpful! :-)

The four points you've listed all sound really useful and I recon I could use most, and possibly all, those features.

[olofk]

FYI, I pushed this to the master branch and also added support for setting use flags on individual files. An example of this in action can be seen here

[a-gibson]

Awesome! Very much appreciated!

[a-gibson]

I've just started looking at something and think Generators will be a cool solution. However, I need to pass information from a top-level core down through the hierarchy to a core that shouldn't know anything about what's above it. This core that's way down will invoke a Generator to do stuff but ideally it should be passed information from the top-level.

Could the capability of flags be expanded to support this? E.g. The top-level build might set some parameters like, Git hash, time of build, etc, and if a child core wants to, it can use those values. Might be beneficial for the syntax to support a default value in case a particular variable/value isn't given - quite useful when distinguishing between local user builds and CI builds.

[olofk]

Ok, I think we have a couple of different answers here

1. If you specify a use flag on the command line it's already available for all cores in the dependency tree.
2. I'm honestly not sure if flags are honored when the generated cores are parsed. The correct behaviour should be to take the flags into account there as well. Need to look up at some point, or maybe you know the answer already?
3. Flags are not sent to the generators. Looking at your use case however, I'm doing something a bit similar in the gitversion generator (fusesoc gen show gitversion will show you the usage) where the generator creates a bunch of defines that can be used in the design. You can find an example here and the corresponding RTL that uses these defines. Perhaps this doesn't help you at all, and you really need to pass this on the command-line. I'm not sure what would be the best way to achieve this. Use flags are really meant to be boolean (exists or not), but I have come across other situations, where it would be very useful with variable substitutions in the core files (like having a $BUILD_DATE in the core file that can get overwritten by a --build_date). Perhaps that's what you need? I think the main blocker for adding support for that is actually to find a syntax that doesn't clash with other things ($VAR is already used for environment variables, {VAR} might get in trouble with yaml syntax and so on)

And in any case, support for default values should be there. The current use flag support is very rudimentary, just to solve some use cases and see where to go next

[a-gibson]

Thanks for the comprehensive response!

1. I already use this feature to great effect already. :) I set some flags at the top-level and they affect which files are compiled, thus changing the resultant bitfile.

2. If I understand you correctly (and I feel I may not) I'm not sure either. I haven't tried but I can't foresee how to make a core file created by a Generator use a flag. I guess the code isn't there for a generator to create core files with syntax like "family_virtex4? (ram_virtex4.vhd)".

3. The gitversion example is pretty close to what I need. In fact my Generator does something very similar, but rather than create parameters it goes and edits the values given to VHDL constants (a bit of a hack because IIRC we were unable to set top-level generic values for some reason). Does the FuseSoC parameter feature work with VHDL? The example you show uses the Verilog pre-processor and VHDL doesn't have an equivalent. The closest thing in VHDL is a generic. So for ModelSim there is a -g option to vsim that lets you set generic values for a simulation. I've seen a webpage that says Quartus supports this by setting some TCL set_parameter -name generic_name generic_value (but I've yet to verify that still works). This could be a much neater solution! Does FuseSoC know how to link parameters with these tool specific options?

[olofk]

1. Great! Let's move on to 2 :)

2. You can for sure generate core files with the conditional syntax. The capi2.generator module is intended as a helper to remove some of the boring aspects of writing the core file, but you can just build it manually if you prefer. Or use the helper, but don't use add_files. And looking at the FuseSoC code, it will probably take the flags into account when parsing the generated core

3. FuseSoC supports setting top-level generics. Just set paramtype : generic and FuseSoC takes care of the rest. All the backends that support VHDL should have this implemented

[a-gibson]

Just had a chance to test this all out and as you suggest the backends support parameters. This investigation has reminded me of the issues we have to work around and it's the use of VHDL configuration files during synthesis. Quartus doesn't let you set values for generics in components the configuration specifies. But that's a different rabbit hole! The combination of generators and parameters should give me the flexibility to do what I need.

[imphil]

Use flags are around now. There's always room to improve them to more scenarios, but let's do so in individual issues.