Added script to create a reduced configuration.

[trholmes]

Should run on a full wires.dat file and output a reduced one.

[tomalin]

Document this script in README.md

[tomalin]

Add "#!/usr/bin/env python" to top of file, and declare file to be executable, like other .py scripts in this directory, so one doesn't need "python " prefix to run it.

[tomalin]

If I run this with "python makeReducedConfig.py" (after first running HourGlassConfig.py & WIres.py), it crashes with message that "modules.dat" is not found. Should it instead be reading processingmodules.dat?

[trholmes]

I added the line and made it executable.

This script uses the output files from running e.g. L1TrackNtupleMakercfg.py with bool writeConfig{true}; set in Settings.h. I'm not sure how those relate to the files that can be created with the scripts in this repo?

[tomalin]

I added the line and made it executable.

This script uses the output files from running e.g. L1TrackNtupleMakercfg.py with bool writeConfig{true}; set in Settings.h. I'm not sure how those relate to the files that can be created with the scripts in this repo?

I assume you are running Anders's new emulation code https://github.com/cms-L1TK/cmssw/pull/74 . I've just made comment there asking him to change names to those consistent with old python scripts, so suggest you do same to your script.

[trholmes]

No problem, taken care of.

[tomalin]

The script generator_hdl.py relies on the entries in the .dat files being ordered from beginning to end of chain. Can you preserve the order from the original files when you reduce them?

[trholmes]

I'm not totally sure what that would mean -- what happens when there are loops?

[tomalin]

An inefficient way of achieving it would be to loop through the entries in the original file, and for each entry in turn, to check if this entry is also in the reduced_file list. If it is, then print this entry out.

[pwittich]

@trholmes what is the status of the suggested modification by @tomalin ?

[trholmes]

@pwittich It's on my to do list but I'm a bit swamped right now with prep for moving to the US + end of semester -- I expect to be able to get to it shortly after I move, which is in 2 weeks.

[trholmes]

OK, updated and spot checked -- it should preserve the ordering now in the different .dat files.

[tomalin]

Running ./makeReducedConfig.py without options gives python error at L129 "Expected indented block".

[tomalin]

In README.md, please give example args for the -sector & -layers options. e.g. I guess "C" and "L1L2" would be valid options?

[tomalin]

As you created your branch a while ago, I suggest also updating it with changes to master. I think this requires (assuming you already checked out your branch): 1) Back up your code, in case git messes up 2) git pull 3) git rebase origin/master 4) Check that your local directory contains a merge of your changes and other peoples. 5) git diff --stat --cached origin/reduced_config (check what you will push) 6) git push --force-with-lease origin reduced_config

[trholmes]

Running ./makeReducedConfig.py without options gives python error at L129 "Expected indented block".

Ah sorry about that, I had some kind of weird tab conversion issue when I updated the file. Should be fixed now, along with your other suggestions.

[tomalin]

It would be good to automatically produce a .pdf file containing the design of the reduced chain. I think adding following code will do it:

At start of makeReducedConfig.py

from TrackletGraph import TrackletGraph import ROOT

At end of makeReducedConfig.py

Plot graph

tracklet = TrackletGraph.from_configs("%sprocessingmodules.dat"%args.output,"%smemorymodules.dat"%args.output,"%swires.dat"%args.output) pageWidth, pageHeight, dyBox, textSize = tracklet.draw_graph(tracklet.get_all_proc_modules()) ROOT.gROOT.SetBatch(True) ROOT.gROOT.LoadMacro('DrawTrackletProject.C') ROOT.DrawTrackletProject(pageWidth, pageHeight, dyBox, textSize);

[tomalin]

TrackletGraph.pdf made by my suggestion above shows that only 9 TE are connected to reduced chain. But reduced_processingmodules.dat shows 17 TE. I think is is because reduced_wires.dat shows that several TE have had all there output ports deleted TE_L1PHID13_L2PHIB11 . Is this a bug?

[trholmes]

Thanks Ian -- I was able to add this functionality after rebasing again, and I added a (admittedly somewhat case specific) protection that prevents these TEs from getting added. With this input config, I am now able to actually run L1TrackNtupleMaker with this input and see tracks with some small modifications that Anders and I added.

[tomalin]

I think this is now OK, though Anders should check it too. With 35 InputRouters & 32 MatchEngines, it is a fairly wide chain. I've verified that the top-level scripts produce a plausible PRMEMC chain with these reduced files (./generator_hdl.py --uut PR_L3PHIC -u 0 -d 2 -p reduced_processingmodules.dat -m reduced_memorymodules.dat -w reduced_wires.dat)

[pwittich]

I think this is now OK, though Anders should check it too. With 35 InputRouters & 32 MatchEngines, it is a fairly wide chain. I've verified that the top-level scripts produce a plausible PRMEMC chain with these reduced files (./generator_hdl.py --uut PR_L3PHIC -u 0 -d 2 -p reduced_processingmodules.dat -m reduced_memorymodules.dat -w reduced_wires.dat)

This is considerably bigger than I had expected. Can we prune it further? On the hardware this would mean loading 35 different links/input memories ....

[trholmes]

I think this is now OK, though Anders should check it too. With 35 InputRouters & 32 MatchEngines, it is a fairly wide chain. I've verified that the top-level scripts produce a plausible PRMEMC chain with these reduced files (./generator_hdl.py --uut PR_L3PHIC -u 0 -d 2 -p reduced_processingmodules.dat -m reduced_memorymodules.dat -w reduced_wires.dat)

This is considerably bigger than I had expected. Can we prune it further? On the hardware this would mean loading 35 different links/input memories ....

I've reduced it further by defaulting to only positive eta IRs -- you can see the resulting project attached. TrackletProject 2.pdf

[pwittich]

this looks like an improvement, thanks, @trholmes ! can someone generate the appropriate top-level FW from this?

[tomalin]

This now has 17 InputRouters & 32 MatchEngines, so is smaller, but still large. The scripts generate plausible VHDL from this input. (Though without modified top-level HLS with pruned inputs/outputs, it will not synthesise). The current PR seems good, unless Tova can figure out how to reduce the chain size further.

[trholmes]

I can always manually reduce which modules are included further, but we run the risk of not having a configuration that's likely to produce any tracks. Right now, when I run over 100 ttbar events, I get this:

tracks/event (pt > 2) = 1.28 tracks/event (pt > 3.0) = 0.68 tracks/event (pt > 10.0) = 0.06

For reference, before reducing to positive eta only, it was this: tracks/event (pt > 2) = 3.59 tracks/event (pt > 3.0) = 1.76 tracks/event (pt > 10.0) = 0.25

To me, this looks good for testing -- it wouldn't be so hard to hunt down good events to run on and you'd get a relatively steady stream of tracks once we started looping more events without having to fiddle around with the input too much.

[pwittich]

A possible alternative would be to create a different fake cabling scheme to reduce the number of input routers, or other configurations that are more optimized for this slice test. I am not convinced that this is a good direction to go though.

@trholmes does the readme contain instructions on how to use this, and can it also contain instructions on how to generate the HDL top-level?

[trholmes]

Yeah, I agree, I don't think making an optimized scheme is a good idea unless it's absolutely necessary to test.

The README explains how to run this script. RE generating the HDL top-level, maybe @tomalin can add that in?

[tomalin]

Yeah, I agree, I don't think making an optimized scheme is a good idea unless it's absolutely necessary to test.

The README explains how to run this script. RE generating the HDL top-level, maybe @tomalin can add that in?

I don't think I need to add anything else to README. The combined doc of your makeReducedCfg and the existing generate_hdl scripts makes clear how to create the top level from the reduced .dat files.