NVlabs / timeloop

Timeloop performs modeling, mapping and code-generation for tensor algebra workloads on various accelerator architectures.
https://timeloop.csail.mit.edu/
BSD 3-Clause "New" or "Revised" License
340 stars 104 forks source link

Reproducing the Results for Eyeriss-256 #74

Closed roggier closed 3 years ago

roggier commented 3 years ago

Hi,

for my master thesis I am currently trying to reproduce the results given in Figure 10 of the timeloop paper.

From my current understanding the files given in the configs/mapper folder named eyeriss-256.cfg should give an accurate energy estimation for a specific loop order and loop tiling when invoked with timeloop-mapper. However when applying the bounds of the alexnet first layer my energy estimates differ from what was presented in the paper. My other approach was using the architectures (.yaml) provided in the same folder and invoking the mapper as well as using the eyeriss-like design of the exercises and changing the number of PEs to 256.

is the comparison to the original eyeriss paper performed using the same dataflow for every layer or is it changing? And if Possible could you provide the .cfg file to reproduce the different layers

angshuman-parashar commented 3 years ago

The challenge here is that you're trying to match the behavior of a different analytical energy model, i.e., the model used in the Eyeriss ISCA paper. Carefully study the energy model in the Eyeriss ISCA paper. Next, you can either edit the pat-public code, or create an Accelergy plugin to reproduce those ratios in Timeloop. You may also have to select a different multicast model to precisely reproduce that behavior (see network-legacy.cpp:329-395).

Yes, you'll have to invoke the mapper separately for each layer. The optimal mapping for each layer may be different. The dataflow does not change -- the row-stationary dataflow is baked into the mapspace constraints, which the mapper will not violate as it walks the mapspace.

roggier commented 3 years ago

thanks for the answer

KelvinYang0320 commented 1 year ago

@roggier @angshuman-parashar I am trying to reproduce the results given in Figure 10. It seems that timeloop has changed a lot during these years. Is there an easy way to do that? Thank you! 😄

angshuman-parashar commented 1 year ago

Hmm... it should still be doable (or at least get reasonably close). Be sure to enable the EYERISS_HACK_MULTICAST in src/model/network-legacy.cpp. You'll also have to override the access energies in the architecture YAML. Sadly I do not remember the exact values, but we derived them from the Eyeriss paper.