How to interpret spatial_mapping and temporal_mapping in the output

This is the output from the default run. I wonder how to understand the output. It seems it only has mapping for inputs. Could you explain more about values in spatial mapping and temporal mapping? What about weights? How could this output file describe the overall execution process for all applications?

"inputs": {
   ...
      "spatial_mapping": {
            "spatial_mapping": {
                "O": [
                    [
                        [
                            "FX",
                            10.0
                        ],
                        [
                            "FY",
                            2.0
                        ]
                    ],
                    [
                        [
                            "K",
                            4.0
                        ]
                    ],
                    []
                ],
                "W": [
                    [],
                    [
                        [
                            "FX",
                            10.0
                        ],
                        [
                            "FY",
                            2.0
                        ],
                        [
                            "K",
                            4.0
                        ]
                    ],
                    []
                ],
                "I": [
                    [
                        [
                            "K",
                            4.0
                        ]
                    ],
                    [
                        [
                            "FX",
                            10.0
                        ],
                        [
                            "FY",
                            2.0
                        ]
                    ],
                    []
                ]
            }
        },
        "temporal_mapping": {
            "temporal_mapping": {
                "O": [
                    [],
                    [
                        [
                            "OY",
                            5
                        ],
                        [
                            "OY",
                            5
                        ],
                        [
                            "OX",
                            5
                        ],
                        [
                            "K",
                            4
                        ],
                        [
                            "K",
                            4
                        ],
                        [
                            "FY",
                            2
                        ]
                    ]
                ],
                "W": [
                    [
                        [
                            "OY",
                            5
                        ],
                        [
                            "OY",
                            5
                        ],
                        [
                            "OX",
                            5
                        ]
                    ],
                    [
                        [
                            "K",
                            4
                        ],
                        [
                            "K",
                            4
                        ],
                        [
                            "FY",
                            2
                        ]
                    ]
                ],
                "I": [
                    [],
                    [
                        [
                            "OY",
                            5
                        ],
                        [
                            "OY",
                            5
                        ],
                        [
                            "OX",
                            5
                        ],
                        [
                            "K",
                            4
                        ],
                        [
                            "K",
                            4
                        ],
                        [
                            "FY",
                            2
                        ]
                    ]
                ]
            }
}

Hi Z-KN,

That is good question!

Just for others' reference, I suppose the output is clipped from file outputs/Dimc-ds_cnn-layer_LayerNode_0_complete.json, by running the following command:

python main_onnx.py --model zigzag/inputs/examples/workload/mlperf_tiny/ds_cnn.onnx --accelerator zigzag.inputs.examples.hardware.Dimc --mapping zigzag.inputs.examples.mapping.default_imc

For your questions:

The spatial mapping and temporal mapping are internally generated by ZigZag-IMC, and the format here aligns with the public ZigZag. As the mapper runs before the cost model executes, the mapping is placed under the input category in the output JSON file.

Regarding the content of the spatial mapping, it mirrors the definition in the input file but is structured based on hardware levels rather than dimensions. For each layer operand (i.e., O, W, I), each bracket corresponds to a hardware level, starting from the compute unit level to the top memory level for that operand. The spatial mapping indicates the parallelism of the operand on each hardware level.

Take the "O" in the spatial mapping as an example:

From the first bracket to the last bracket, it represents: compute unit -> output registers -> SRAM. In the In-Memory Computing context, each compute unit represents a multiplier.
Within the first bracket, we have [[FX, 10], [FY, 2]], indicating that on the compute level, each output has a parallelism of 20. This means that each port of the upper level (i.e., register) is connected to 20 compute units, and the results from these compute units are accumulated to generate one output at the upper level.
Within the second bracket, we have [K, 4], representing output registers. This means each port of the upper level (SRAM) is connected to 4 registers for mapping the K dimension.

If any bracket is empty, it implies that each port of the upper level is connected to one port at the lower level. For example, the weight has 3 brackets, representing compute unit -> weight cells -> DRAM. Since the first bracket is empty, it means each weight cell is connected to a single compute unit.

Regarding the temporal mapping, each bracket corresponds to a memory level, starting from the bottom memory level to the top memory level related to that operand. Note the compute-unit level is excluded for the temporal mapping.

Take the "O" in the temporal mapping as an example:

From the first bracket to the last bracket, it represents: output registers -> SRAM. Within each bracket, the innermost loop is listed at the top, and the outermost loop is listed at the bottom.
The first bracket for "O" is empty, indicating no temporal loop within the output register level. Consequently, the output in the register will be updated every cycle.
The second bracket has multiple brackets, implying recycling operation within the SRAM level.

Therefore for "O", from top to down, the temporal mapping represents the for loop below:

--------SRAM level--------
for FY (0, 2):
for K (0, 4):
for K (0, 4):
  for OX (0, 5):
    for OY (0, 5):
      for OY (0, 5):
--------register level--------
         None
--------compute level--------
            mac computation

Take the "W" as an example:

From top to down, the temporal mapping represents the for loop below:

--------DRAM level--------
for FY (0, 2):
for K (0, 4):
for K (0, 4):
--------weight cell level--------
  for OX (0, 5):
    for OY (0, 5):
      for OY (0, 5):
--------compute level--------
            mac computation

Combining the temporal mapping size with the spatial mapping size, you can exactly get the same layer size as defined in the onnx file.

If you're unfamiliar with the loop representation based on memory level, please refer to the ZigZag paper for a detailed explanation.

Let me know if there is still any confusion.

Regards, Jiacong

Thank you for your detailed explanation. So in a high-level view, spatial mapping is mapping elements spatially to existing hardware at a time, while temporal mapping means updating elements time by time. Is that right?

Another question is, as

the mapping is placed under the input category

so mapping is also an input. The default IMC mapping is

mapping = {
    "default": {
        "core_allocation": 1,
        # "spatial_mapping": {"D1": ("OX", 25), "D2": (("FX", 3), ("FY", 3))},
        "memory_operand_links": {"O": "O", "W": "I2", "I": "I1"},
        "spatial_mapping_hint": {"D1": ["K", "OX"], "D2": ["C", "FX", "FY"]},
    }
}

Does it mean users need to specify its mapping as input? I thought Zigzag will search the best mapping given an architecture. So architecture is an input, while mapping is an output. Is it the case?

Regaridng your first question: yes, your understanding is totally correct!

Regarding your second question: The mapping file is necessary for ZigZag-IMC, as it not only defines the spatial mapping but also specifies the core allocation and the links between layer operands and memory operands. However, the spatial mapping definition is not necessary.

As for your requirement, ZigZag-IMC supports multiple use cases:

If your architecture only supports fixed spatial mapping, you can define the mapping using the spatial_mapping parameter, as the one commented in your code.
If your architecture dimension supports spatial mapping on multiple layer dimensions but not a fixed size, you can use the spatial_mapping_hint parameter to define the spatial mapping, as in the code you provided. Note that the spatial_mapping parameter should be commented out, as it has higher priority over spatial_mapping_hint.
If you want to fully explore the spatial mapping possibilities without constraints, you can either (1) include all layer dimensions within the spatial_mapping_hint, or (2) comment out both spatial_mapping and spatial_mapping_hint. The framework will then explore all possibilities internally.

KULeuven-MICAS / zigzag-imc

How to interpret spatial_mapping and temporal_mapping in the output #10

For your questions: