Xilinx / llvm-aie

Fork of LLVM to support AMD AIEngine processors
Other
107 stars 12 forks source link

[AIE2] New vector custom post combiners #195

Closed andcarminati closed 1 month ago

andcarminati commented 2 months ago

Optimize some vector operations (instrinsics) to COPYs. With this PR, we carry less subregister dependencies to the the selector and future passes. Also, we start to expose more selection-combine opportunities.

Covered situations: CONCAT- > UNMERGE CONCAT -> EXTRACT UPDATE -> UNMERGE

We have one case related to G_CONCAT_VECTORS that is a preparation for the next step: the splitting of memory operations.

andcarminati commented 2 months ago

QoR results:

|--------------------------|---------------|-----------------|--------------|--------------------------------------|-----------------|----------------|--------------|--------------|--------------|--------------|--------------------|--------------------|------------------------------------|------------------------------------|----------------|----------------------------|------------------------|--------------|-----------------|-----------------|-----------------|--------------|--------------|---------------------------|---------------------------|-----------------------|-----------------------|---------------------------|-----------------------|---------------|---------------|-------------------------|-------------------------|--------------------|-------------------|------------------|----------------------|--------------------|----------------------|----------------|------------------|--------------------|----------------|-----------------------------|-----------------------------|----------------|----------------|----------------------------------------------------------|----------------------------------------------|----------------------------------------------|------------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|--------------|--------------|-----------------------|-----------------------|------------------|------------------|-------------------------|-------------------------|--------------|--------------|------------------|------------------|--------------|--------------|----------------|----------------|---------------|---------------|--------------------------|--------------------------|--------------------|--------------------|--------------|-------------------------|-------------------------|---------------|---------------|----------------------|----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------|--------------------------------------|------------------------|----------------------|------------------|----------------------|------------------|----------------|-----------------|-----------------|-------------------------------|--------------|--------------|----------------------|------------------|--------------|--------------|------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|--------------|-------------------------|-------------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|----------------|----------------|-----------------------------|-----------------------------|--------------------|--------------------|--------------------|--------------------|-------------------------------|-------------------------------|------------------|------------------|-------------------------------|-------------------------------|-------------------------------|-----------------------------------|-------------------------------|------------------------------|------------------------------|------------------------------|--------------|--------------|--------------|--------------|-------------------|----------------------|--------------|--------------|--------------------|--------------|--------------|--------------|--------------|--------------------------------------|--------------------------------------|----------------|----------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|---------------------|-----------------|------------------------|------------------------|-----------------------|-----------------------|--------------------------------------|--------------|-----------------------|-----------------------|-------------------|-------------------|-------------------|-------------------|------------------|------------------|------------------|------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|------------------------------|--------------------------|-----------------------|-----------------------|------------------|------------------|------------------|------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|------------------|------------------|------------------|------------------|--------------|--------------|---------------------|--------------|--------------|-------------------|--------------|--------------|------------------|------------------|------------------|------------------|---------------|----------------|----------------|------------------|-------------------------|-------------------------|-------------------------|----------------|----------------|----------------|----------------|--------------|--------------|---------------|---------------|------------------|--------------|--------------|----------------|--------------|--------------|--------------------|----------------|--------------------------------------|--------------------------------------|-----------------------------|-----------------------------|-------------------------------------------|-----------------|-----------------|-------------------------------|-----------------------------|-------------------------|--------------|--------------|------------------|------------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|--------------|--------------|--------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|-----------------|------------------------|----------------|--------------------------------------|-------------------------------|------------------------------|----------------|-------------------|--------------|--------------|---------------|---------------|--------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------|---------------|---------------|--------------|------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
| Core_Compute_Cycle_Count | Shrink_aie2_1 | Elu_aie2_bf16_0 | Add2D_bf16_0 | PowAttributeBroadcasting_aie2_bf16_0 | ArgMin1d_bf16_0 | Softmax_bf16_0 | Abs_bf16_0   | Abs_int8_0   | Add2D_0      | Add2D_1      | Add2D_Standalone_0 | Add2D_Standalone_1 | AddAttributeBroadcasting_aie2_bf16 | AddAttributeBroadcasting_aie2_int8 | AddBf16_aie2_0 | AddBroadcastingBf16_aie2_0 | AddBroadcasting_aie2_0 | Add_aie2_0   | ArgMax1d_bf16_0 | ArgMax1d_int8_0 | ArgMin1d_int8_0 | AvgPool2D_0  | AvgPool2D_1  | AvgPool2D_aie2_bfloat16_0 | AvgPool2D_aie2_bfloat16_1 | AvgPool2D_aie2_int8_0 | AvgPool2D_aie2_int8_1 | BatchNorm1d_aie2_bfloat16 | BatchNorm1d_aie2_int8 | BatchNorm2D_0 | BatchNorm2D_1 | BilinearInterpolation_0 | BilinearInterpolation_1 | BitShift_AIE2_int8 | BitwiseNot_aie2_0 | BitwiseOr_int8_0 | BitwiseXor_aie2_int8 | Cast_aie2_bfloat16 | Cast_aie2_bfloat16_1 | Cast_aie2_int8 | Cast_aie2_int8_1 | Ceil_AIE2_bfloat16 | Ceil_AIE2_int8 | ChannelsFirstFlatten_bf16_0 | ChannelsFirstFlatten_int8_0 | Clip_aie2_bf16 | Clip_aie2_int8 | CompareOpsBroadcasting_K_EQ_GE_GT_LE_LT_CMP_GE_int8_aie2 | CompareOps_K_EQ_GE_GT_LE_LT_CMP_EQ_int8_aie2 | CompareOps_K_EQ_GE_GT_LE_LT_CMP_GE_int8_aie2 | CompareOps_K_EQ_GE_GT_LE_LT_CMP_GE_int8_aie2_ptr_interface | Conv1D_DW_AIE2_bf16_0 | Conv1D_DW_AIE2_bf16_1 | Conv1D_DW_AIE2_int8_0 | Conv1D_DW_AIE2_int8_1 | Conv2D_0     | Conv2D_1     | Conv2D_7x7s2_Layer1_0 | Conv2D_7x7s2_Layer1_1 | Conv2D_11x11s4_0 | Conv2D_11x11s4_1 | Conv2D_11x11s4_Layer1_0 | Conv2D_11x11s4_Layer1_1 | Conv2D_DW_0  | Conv2D_DW_1  | Conv2D_DW_bf16_0 | Conv2D_DW_bf16_1 | Conv2D_FC_0  | Conv2D_FC_1  | Conv2D_LReLU_0 | Conv2D_LReLU_1 | Conv2D_ReLU_0 | Conv2D_ReLU_1 | Conv2D_ReLU_Standalone_0 | Conv2D_ReLU_Standalone_1 | Conv2D_ReLU_int8_0 | Conv2D_ReLU_int8_1 | Conv2D_SV60  | Conv2D_Transpose_AIE2_0 | Conv2D_Transpose_AIE2_1 | Conv2D_bf16_0 | Conv2D_bf16_1 | Conv2D_mixed_batch_0 | Conv2D_mixed_batch_1 | DegroupG4_aie2_bf16_0 | DegroupG4_aie2_bf16_1 | DegroupG4_aie2_int8_0 | DegroupG4_aie2_int8_1 | DegroupG8_aie2_bf16_0 | DegroupG8_aie2_bf16_1 | DegroupG8_aie2_int8_0 | DegroupG8_aie2_int8_1 | DilatedConv2D_1 | DivAttributeBroadcasting_aie2_bf16_0 | DivBroadcasting_aie2_1 | EleMax_aie2_bfloat16 | EleMax_aie2_int8 | EleMin_aie2_bfloat16 | EleMin_aie2_int8 | ElemDiv_aie2_1 | Erf_aie2_bf16_0 | Erf_aie2_int8_0 | Erf_aie2_int8_0_ptr_interface | Exp_bf16_0   | Exp_bf16_1   | Expand_aie2_bfloat16 | Expand_aie2_int8 | Floor_aie2_0 | Floor_aie2_1 | FullyConnect_aie2_bf16 | FullyConnect_aie2_int8 | GEMM_bf16_0  | GEMM_bf16_1  | GEMM_int8_0  | GEMM_int8_1  | GEMV_0       | GEMV_1       | GeluTemplated_aie2_bf16 | GeluTemplated_aie2_int8 | GroupG4_aie2_bf16_0 | GroupG4_aie2_bf16_1 | GroupG4_aie2_int8_0 | GroupG4_aie2_int8_1 | GroupG8_aie2_bf16_0 | GroupG8_aie2_bf16_1 | GroupG8_aie2_int8_0 | GroupG8_aie2_int8_1 | Group_Conv2D_0 | Group_Conv2D_1 | HardSigmoidTemplated_bf16_0 | HardSigmoidTemplated_int8_0 | HardSigmoid_bf16_0 | HardSigmoid_bf16_1 | HardSigmoid_int8_0 | HardSigmoid_int8_1 | HardswishAsHardsigmoid_aie2_0 | HardswishAsHardsigmoid_aie2_1 | Hardswish_aie2_0 | Hardswish_aie2_1 | InstanceNormPart1_aie2_bf16_0 | InstanceNormPart1_aie2_int8_0 | InstanceNormPart2_aie2_int8_0 | InterpolateLinear1D_AIE2_bfloat16 | InterpolateLinear1D_AIE2_int8 | LayerNormC8Part1_aie2_bf16_0 | LayerNormC8Part1_aie2_int8_0 | LayerNormC8Part2_aie2_int8_0 | LayerNorm_0  | LayerNorm_1  | Log_bf16_0   | Log_int8_0   | LogicalNot_aie2_0 | LogicalXor_aie2_int8 | MaxPool2D_0  | MaxPool2D_1  | Mish_aie2_bfloat16 | Mul2D_0      | Mul2D_1      | Mul2d_bf16_0 | Mul2d_bf16_1 | MulAttributeBroadcasting_aie2_bf16_0 | MulAttributeBroadcasting_aie2_int8_0 | MulBf16_aie2_0 | MulBroadcastingBf16_aie2_0 | MulBroadcasting_aie2_0 | Mul_aie2_0   | Neg_aie2_0   | Neg_aie2_1   | Pad2D_0      | Pad2D_1      | Pad3D_AIE2_bfloat16 | Pad3D_AIE2_int8 | PixelShuffle_aie2_bf16 | PixelShuffle_aie2_int8 | PixelUnshuffle_bf16_0 | PixelUnshuffle_int8_0 | PowAttributeBroadcasting_aie2_int8_0 | Pow_int8_0   | Range_bfloat16_aie2_0 | Range_bfloat16_aie2_1 | Range_int8_aie2_0 | Range_int8_aie2_1 | Reciprocal_aie2_0 | Reciprocal_aie2_1 | ReduceMax_bf16_0 | ReduceMax_bf16_1 | ReduceMax_int8_0 | ReduceMax_int8_1 | ReduceMeanAxis_1_aie2_int8 | ReduceMeanAxis_2_aie2_int8 | ReduceMeanAxis_3_aie2_int8 | ReduceMeanAxis_4_aie2_int8 | ReduceMeanAxis_5_aie2_int8 | ReduceMeanAxis_6_aie2_int8 | ReduceMeanAxis_7_aie2_bf16 | ReduceMeanAxis_7_aie2_int8 | ReduceMeanNoc8_AIE2_bfloat16 | ReduceMeanNoc8_AIE2_int8 | ReduceMin1D_aie2_bf16 | ReduceMin1D_aie2_int8 | ReduceMin_bf16_0 | ReduceMin_bf16_1 | ReduceMin_int8_0 | ReduceMin_int8_1 | ReduceSumAxis_1_aie2_int8 | ReduceSumAxis_2_aie2_int8 | ReduceSumAxis_3_aie2_int8 | ReduceSumAxis_4_aie2_int8 | ReduceSumAxis_5_aie2_int8 | ReduceSumAxis_6_aie2_int8 | ReduceSumAxis_7_aie2_bf16 | ReduceSumAxis_7_aie2_int8 | ReduceSum_bf16_0 | ReduceSum_bf16_1 | ReduceSum_int8_0 | ReduceSum_int8_1 | Requantize_0 | Requantize_1 | Rescale_aie2_int8_0 | Round_aie2_0 | Round_aie2_1 | Rsqrt_aie2_bf16_0 | Scale_Add_0  | Scale_Add_1  | Scale_Add_bf16_0 | Scale_Add_bf16_1 | Select_aie2_bf16 | Select_aie2_int8 | Shrink_aie2_0 | SiLU_aie2_bf16 | SiLU_aie2_int8 | SiLU_aie2_int8_1 | SigmoidTemplated_bf16_0 | SigmoidTemplated_int8_0 | SigmoidTemplated_int8_1 | Sigmoid_bf16_0 | Sigmoid_bf16_1 | Sigmoid_int8_0 | Sigmoid_int8_1 | Sign_int8_0  | Sign_int8_1  | Sin_aie2_bf16 | Sin_aie2_int8 | Slice_bfloat16_0 | Slice_int8_0 | Softmax_1    | Softmax_bf16_1 | Sqrt_bf16_0  | Sqrt_bf16_1  | Squeeze_bfloat16_0 | Squeeze_int8_0 | SubAttributeBroadcasting_aie2_bf16_0 | SubAttributeBroadcasting_aie2_int8_0 | SubBroadcasting_aie2_bf16_0 | SubBroadcasting_aie2_int8_0 | SubBroadcasting_aie2_int8_0_ptr_interface | Sub_aie2_bf16_0 | Sub_aie2_int8_0 | Sub_aie2_int8_0_ptr_interface | TanhTemplated_aie2_bfloat16 | TanhTemplated_aie2_int8 | Tanh_int8_0  | Tanh_int8_1  | Tile_aie2_bf16_0 | Tile_aie2_int8_1 | Topk1D_bf16_0 | Topk1D_bf16_1 | Topk1D_int8_0 | Topk1D_int8_1 | Topk2D_bf16_0 | Topk2D_bf16_1 | Topk2D_int8_0 | Topk2D_int8_1 | Transpose_aie2_bf16_021 | Transpose_aie2_bf16_021_pad | Transpose_aie2_bf16_102 | Transpose_aie2_bf16_102_pad | Transpose_aie2_bf16_120 | Transpose_aie2_bf16_201 | Transpose_aie2_bf16_201_pad | Transpose_aie2_bf16_210 | Transpose_aie2_bf16_210_pad | Transpose_aie2_int8_021 | Transpose_aie2_int8_021_pad | Transpose_aie2_int8_102 | Transpose_aie2_int8_102_pad | Transpose_aie2_int8_120 | Transpose_aie2_int8_120_pad | Transpose_aie2_int8_201 | Transpose_aie2_int8_201_pad | Transpose_aie2_int8_210 | Transpose_aie2_int8_210_pad | bfloat16     | int8         | Pow_bf16_0   | Sign_bf16_0  | ReduceSumAxis_5_aie2_bf16 | ReduceSumAxis_3_aie2_bf16 | ReduceSumAxis_6_aie2_bf16 | ReduceMeanAxis_3_aie2_bf16 | ReduceMeanAxis_6_aie2_bf16 | ReduceMeanAxis_5_aie2_bf16 | Sign_bf16_1  | ReduceSumAxis_4_aie2_bf16 | ReduceSumAxis_2_aie2_bf16 | ReduceSumAxis_1_aie2_bf16 | ReduceMeanAxis_2_aie2_bf16 | ReduceMeanAxis_4_aie2_bf16 | ReduceMeanAxis_1_aie2_bf16 | Elu_aie2_int8_0 | DivBroadcasting_aie2_0 | ElemDiv_aie2_0 | DivAttributeBroadcasting_aie2_int8_0 | InstanceNormPart2_aie2_bf16_0 | LayerNormC8Part2_aie2_bf16_0 | Mish_aie2_int8 | Rsqrt_aie2_int8_0 | Tanh_0       | Tanh_1       | Sqrt_int8_0   | Sqrt_int8_1   | CompareOps_K_EQ_GE_GT_LE_LT_CMP_EQ_bfloat16_aie2 | CompareOpsBroadcasting_K_EQ_GE_GT_LE_LT_CMP_GE_bfloat16_aie2 | CompareOps_K_EQ_GE_GT_LE_LT_CMP_GE_bfloat16_aie2 | GELU_0        | GELU_1        | Averege diff | Diff stdev | Quantile #1 | Quantile #2 | Quantile #3 | Quantile #4 | Quantile #5 | Quantile #6 | Quantile #7 | Quantile #8 | Quantile #9 |
|--------------------------|---------------|-----------------|--------------|--------------------------------------|-----------------|----------------|--------------|--------------|--------------|--------------|--------------------|--------------------|------------------------------------|------------------------------------|----------------|----------------------------|------------------------|--------------|-----------------|-----------------|-----------------|--------------|--------------|---------------------------|---------------------------|-----------------------|-----------------------|---------------------------|-----------------------|---------------|---------------|-------------------------|-------------------------|--------------------|-------------------|------------------|----------------------|--------------------|----------------------|----------------|------------------|--------------------|----------------|-----------------------------|-----------------------------|----------------|----------------|----------------------------------------------------------|----------------------------------------------|----------------------------------------------|------------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|--------------|--------------|-----------------------|-----------------------|------------------|------------------|-------------------------|-------------------------|--------------|--------------|------------------|------------------|--------------|--------------|----------------|----------------|---------------|---------------|--------------------------|--------------------------|--------------------|--------------------|--------------|-------------------------|-------------------------|---------------|---------------|----------------------|----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------|--------------------------------------|------------------------|----------------------|------------------|----------------------|------------------|----------------|-----------------|-----------------|-------------------------------|--------------|--------------|----------------------|------------------|--------------|--------------|------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|--------------|-------------------------|-------------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|----------------|----------------|-----------------------------|-----------------------------|--------------------|--------------------|--------------------|--------------------|-------------------------------|-------------------------------|------------------|------------------|-------------------------------|-------------------------------|-------------------------------|-----------------------------------|-------------------------------|------------------------------|------------------------------|------------------------------|--------------|--------------|--------------|--------------|-------------------|----------------------|--------------|--------------|--------------------|--------------|--------------|--------------|--------------|--------------------------------------|--------------------------------------|----------------|----------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|---------------------|-----------------|------------------------|------------------------|-----------------------|-----------------------|--------------------------------------|--------------|-----------------------|-----------------------|-------------------|-------------------|-------------------|-------------------|------------------|------------------|------------------|------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|------------------------------|--------------------------|-----------------------|-----------------------|------------------|------------------|------------------|------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|------------------|------------------|------------------|------------------|--------------|--------------|---------------------|--------------|--------------|-------------------|--------------|--------------|------------------|------------------|------------------|------------------|---------------|----------------|----------------|------------------|-------------------------|-------------------------|-------------------------|----------------|----------------|----------------|----------------|--------------|--------------|---------------|---------------|------------------|--------------|--------------|----------------|--------------|--------------|--------------------|----------------|--------------------------------------|--------------------------------------|-----------------------------|-----------------------------|-------------------------------------------|-----------------|-----------------|-------------------------------|-----------------------------|-------------------------|--------------|--------------|------------------|------------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|--------------|--------------|--------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|-----------------|------------------------|----------------|--------------------------------------|-------------------------------|------------------------------|----------------|-------------------|--------------|--------------|---------------|---------------|--------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------|---------------|---------------|--------------|------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
| Before                   |           735 |            1463 |          256 |                                40463 |             383 |           7631 |          407 |          510 |          215 |          466 |                320 |                480 |                                760 |                                922 |            673 |                        727 |                    890 |          832 |             354 |             410 |             303 |         1073 |          785 |                      3297 |                      2281 |                  1073 |                   785 |                       719 |                   520 |           398 |           738 |                     731 |                     385 |               2008 |               233 |              482 |                  715 |               1930 |                 1930 |            794 |              794 |               1381 |            447 |                       13603 |                       13995 |            228 |            276 |                                                      989 |                                          994 |                                          989 |                                                        989 |                  3477 |                  4036 |                  1543 |                  1769 |         8218 |         2510 |                  5926 |                  1649 |             5462 |             5468 |                    4315 |                    3015 |         2962 |          874 |             1217 |             3934 |         2803 |         1220 |           2213 |           5301 |          1316 |         28202 |                     1316 |                     2569 |              10445 |                970 |          898 |                   39085 |                   10470 |         25551 |         40034 |                11176 |                22846 |                   606 |                  1025 |                   367 |                   577 |                   750 |                  1152 |                   439 |                   640 |            5797 |                                 5820 |                   1563 |                  333 |              236 |                  333 |              236 |           1502 |            2896 |            2554 |                          2533 |         7333 |         1472 |                 1968 |             1931 |          301 |          896 |                   1047 |                    836 |         3626 |         7669 |         3066 |        36335 |          474 |          392 |                    1389 |                    1214 |                 502 |                1540 |                 319 |                 835 |                1033 |                1667 |                 562 |                 898 |           3998 |           4521 |                         647 |                         257 |               1031 |                707 |                459 |                471 |                          1368 |                          1915 |             1368 |             1910 |                          2931 |                         11178 |                         12336 |                             15243 |                         11972 |                         9066 |                         7896 |                        11518 |        19682 |        16577 |         4976 |         1617 |               225 |                  528 |          798 |          590 |               5684 |          459 |          459 |          455 |          303 |                                 1416 |                                  612 |           1077 |                       1132 |                    475 |          420 |          435 |          343 |          676 |         1732 |                9304 |            9723 |                   8574 |                   8574 |                 17152 |                 17152 |                                 4312 |         4312 |                  3963 |                  2659 |              1206 |              1794 |              1348 |              2529 |             7296 |             9571 |            14708 |            19513 |                       7335 |                       7390 |                       2933 |                       7357 |                       2949 |                       2929 |                       6937 |                       2122 |                        22344 |                    62115 |                   194 |                   169 |             7296 |            18739 |             8996 |            19267 |                      7223 |                      7263 |                      2897 |                      7288 |                      2899 |                      2875 |                      6897 |                      2079 |            19807 |            12335 |            20171 |            11603 |         1538 |          834 |                 326 |          367 |         1670 |              3603 |          434 |          434 |             1745 |             1745 |              337 |              213 |           672 |           3546 |           2969 |             2967 |                    2136 |                    1468 |                    1468 |           2011 |           1327 |             91 |            110 |          416 |          122 |          3041 |           841 |              947 |         1547 |          502 |           1645 |        29776 |         3792 |                207 |            207 |                                  760 |                                  922 |                         705 |                         868 |                                       868 |             651 |             810 |                           810 |                        2524 |                     310 |          353 |          425 |             4272 |             2619 |          1219 |           171 |           838 |           120 |         34471 |           305 |         30790 |           255 |                    1856 |                        2340 |                    1157 |                        1142 |                    1856 |                    1871 |                        1767 |                    1868 |                        1868 |                    2685 |                        3614 |                    1153 |                        1093 |                    2686 |                        2686 |                    2700 |                        2544 |                    2694 |                        2538 |         1055 |          862 |        36879 |         1048 |                      7680 |                      7678 |                      7664 |                       7768 |                       7754 |                       7748 |          208 |                     12827 |                     12817 |                     12790 |                      13325 |                      13297 |                      13291 |             622 |                   2234 |           2176 |                                 8506 |                         10868 |                        12436 |          10412 |              2649 |         2982 |         3942 |         21973 |         21973 |                                             1876 |                                                         1874 |                                             1871 |          2930 |          3874 |              |            |             |             |             |             |             |             |             |             |             |
|--------------------------|---------------|-----------------|--------------|--------------------------------------|-----------------|----------------|--------------|--------------|--------------|--------------|--------------------|--------------------|------------------------------------|------------------------------------|----------------|----------------------------|------------------------|--------------|-----------------|-----------------|-----------------|--------------|--------------|---------------------------|---------------------------|-----------------------|-----------------------|---------------------------|-----------------------|---------------|---------------|-------------------------|-------------------------|--------------------|-------------------|------------------|----------------------|--------------------|----------------------|----------------|------------------|--------------------|----------------|-----------------------------|-----------------------------|----------------|----------------|----------------------------------------------------------|----------------------------------------------|----------------------------------------------|------------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|--------------|--------------|-----------------------|-----------------------|------------------|------------------|-------------------------|-------------------------|--------------|--------------|------------------|------------------|--------------|--------------|----------------|----------------|---------------|---------------|--------------------------|--------------------------|--------------------|--------------------|--------------|-------------------------|-------------------------|---------------|---------------|----------------------|----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------|--------------------------------------|------------------------|----------------------|------------------|----------------------|------------------|----------------|-----------------|-----------------|-------------------------------|--------------|--------------|----------------------|------------------|--------------|--------------|------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|--------------|-------------------------|-------------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|----------------|----------------|-----------------------------|-----------------------------|--------------------|--------------------|--------------------|--------------------|-------------------------------|-------------------------------|------------------|------------------|-------------------------------|-------------------------------|-------------------------------|-----------------------------------|-------------------------------|------------------------------|------------------------------|------------------------------|--------------|--------------|--------------|--------------|-------------------|----------------------|--------------|--------------|--------------------|--------------|--------------|--------------|--------------|--------------------------------------|--------------------------------------|----------------|----------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|---------------------|-----------------|------------------------|------------------------|-----------------------|-----------------------|--------------------------------------|--------------|-----------------------|-----------------------|-------------------|-------------------|-------------------|-------------------|------------------|------------------|------------------|------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|------------------------------|--------------------------|-----------------------|-----------------------|------------------|------------------|------------------|------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|------------------|------------------|------------------|------------------|--------------|--------------|---------------------|--------------|--------------|-------------------|--------------|--------------|------------------|------------------|------------------|------------------|---------------|----------------|----------------|------------------|-------------------------|-------------------------|-------------------------|----------------|----------------|----------------|----------------|--------------|--------------|---------------|---------------|------------------|--------------|--------------|----------------|--------------|--------------|--------------------|----------------|--------------------------------------|--------------------------------------|-----------------------------|-----------------------------|-------------------------------------------|-----------------|-----------------|-------------------------------|-----------------------------|-------------------------|--------------|--------------|------------------|------------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|--------------|--------------|--------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|-----------------|------------------------|----------------|--------------------------------------|-------------------------------|------------------------------|----------------|-------------------|--------------|--------------|---------------|---------------|--------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------|---------------|---------------|--------------|------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
| After                    |           759 |            1488 |          258 |                                40589 |             384 |           7632 |          407 |          510 |          215 |          466 |                320 |                480 |                                760 |                                922 |            673 |                        727 |                    890 |          832 |             354 |             410 |             303 |         1073 |          785 |                      3297 |                      2281 |                  1073 |                   785 |                       719 |                   520 |           398 |           738 |                     731 |                     385 |               2008 |               233 |              482 |                  715 |               1930 |                 1930 |            794 |              794 |               1381 |            447 |                       13603 |                       13995 |            228 |            276 |                                                      989 |                                          994 |                                          989 |                                                        989 |                  3477 |                  4036 |                  1543 |                  1769 |         8218 |         2510 |                  5926 |                  1649 |             5462 |             5468 |                    4315 |                    3015 |         2962 |          874 |             1217 |             3934 |         2803 |         1220 |           2213 |           5301 |          1316 |         28202 |                     1316 |                     2569 |              10445 |                970 |          898 |                   39085 |                   10470 |         25551 |         40034 |                11176 |                22846 |                   606 |                  1025 |                   367 |                   577 |                   750 |                  1152 |                   439 |                   640 |            5797 |                                 5820 |                   1563 |                  333 |              236 |                  333 |              236 |           1502 |            2896 |            2554 |                          2533 |         7333 |         1472 |                 1968 |             1931 |          301 |          896 |                   1047 |                    836 |         3626 |         7669 |         3066 |        36335 |          474 |          392 |                    1389 |                    1214 |                 502 |                1540 |                 319 |                 835 |                1033 |                1667 |                 562 |                 898 |           3998 |           4521 |                         647 |                         257 |               1031 |                707 |                459 |                471 |                          1368 |                          1915 |             1368 |             1910 |                          2931 |                         11178 |                         12336 |                             15243 |                         11972 |                         9066 |                         7896 |                        11518 |        19682 |        16577 |         4976 |         1617 |               225 |                  528 |          798 |          590 |               5684 |          459 |          459 |          455 |          303 |                                 1416 |                                  612 |           1077 |                       1132 |                    475 |          420 |          435 |          343 |          676 |         1732 |                9304 |            9723 |                   8574 |                   8574 |                 17152 |                 17152 |                                 4312 |         4312 |                  3963 |                  2659 |              1206 |              1794 |              1348 |              2529 |             7296 |             9571 |            14708 |            19513 |                       7335 |                       7390 |                       2933 |                       7357 |                       2949 |                       2929 |                       6937 |                       2122 |                        22344 |                    62115 |                   194 |                   169 |             7296 |            18739 |             8996 |            19267 |                      7223 |                      7263 |                      2897 |                      7288 |                      2899 |                      2875 |                      6897 |                      2079 |            19807 |            12335 |            20171 |            11603 |         1538 |          834 |                 326 |          367 |         1670 |              3603 |          434 |          434 |             1745 |             1745 |              337 |              213 |           672 |           3546 |           2969 |             2967 |                    2136 |                    1468 |                    1468 |           2011 |           1327 |             91 |            110 |          416 |          122 |          3041 |           841 |              947 |         1547 |          502 |           1645 |        29776 |         3792 |                207 |            207 |                                  760 |                                  922 |                         705 |                         868 |                                       868 |             651 |             810 |                           810 |                        2524 |                     310 |          353 |          425 |             4272 |             2619 |          1219 |           171 |           838 |           120 |         34471 |           305 |         30790 |           255 |                    1856 |                        2340 |                    1157 |                        1142 |                    1856 |                    1871 |                        1767 |                    1868 |                        1868 |                    2685 |                        3614 |                    1153 |                        1093 |                    2686 |                        2686 |                    2700 |                        2544 |                    2694 |                        2538 |         1055 |          862 |        36877 |         1047 |                      7672 |                      7670 |                      7656 |                       7752 |                       7738 |                       7732 |          207 |                     12763 |                     12753 |                     12726 |                      13197 |                      13169 |                      13163 |             610 |                   2186 |           2128 |                                 8314 |                         10380 |                        11604 |           9580 |              2425 |         2694 |         3558 |         19670 |         19670 |                                             1670 |                                                         1668 |                                             1665 |          2594 |          3426 | -0.41%       |       1.94 | -0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      |
|--------------------------|---------------|-----------------|--------------|--------------------------------------|-----------------|----------------|--------------|--------------|--------------|--------------|--------------------|--------------------|------------------------------------|------------------------------------|----------------|----------------------------|------------------------|--------------|-----------------|-----------------|-----------------|--------------|--------------|---------------------------|---------------------------|-----------------------|-----------------------|---------------------------|-----------------------|---------------|---------------|-------------------------|-------------------------|--------------------|-------------------|------------------|----------------------|--------------------|----------------------|----------------|------------------|--------------------|----------------|-----------------------------|-----------------------------|----------------|----------------|----------------------------------------------------------|----------------------------------------------|----------------------------------------------|------------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|--------------|--------------|-----------------------|-----------------------|------------------|------------------|-------------------------|-------------------------|--------------|--------------|------------------|------------------|--------------|--------------|----------------|----------------|---------------|---------------|--------------------------|--------------------------|--------------------|--------------------|--------------|-------------------------|-------------------------|---------------|---------------|----------------------|----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------|--------------------------------------|------------------------|----------------------|------------------|----------------------|------------------|----------------|-----------------|-----------------|-------------------------------|--------------|--------------|----------------------|------------------|--------------|--------------|------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|--------------|-------------------------|-------------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|----------------|----------------|-----------------------------|-----------------------------|--------------------|--------------------|--------------------|--------------------|-------------------------------|-------------------------------|------------------|------------------|-------------------------------|-------------------------------|-------------------------------|-----------------------------------|-------------------------------|------------------------------|------------------------------|------------------------------|--------------|--------------|--------------|--------------|-------------------|----------------------|--------------|--------------|--------------------|--------------|--------------|--------------|--------------|--------------------------------------|--------------------------------------|----------------|----------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|---------------------|-----------------|------------------------|------------------------|-----------------------|-----------------------|--------------------------------------|--------------|-----------------------|-----------------------|-------------------|-------------------|-------------------|-------------------|------------------|------------------|------------------|------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|------------------------------|--------------------------|-----------------------|-----------------------|------------------|------------------|------------------|------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|------------------|------------------|------------------|------------------|--------------|--------------|---------------------|--------------|--------------|-------------------|--------------|--------------|------------------|------------------|------------------|------------------|---------------|----------------|----------------|------------------|-------------------------|-------------------------|-------------------------|----------------|----------------|----------------|----------------|--------------|--------------|---------------|---------------|------------------|--------------|--------------|----------------|--------------|--------------|--------------------|----------------|--------------------------------------|--------------------------------------|-----------------------------|-----------------------------|-------------------------------------------|-----------------|-----------------|-------------------------------|-----------------------------|-------------------------|--------------|--------------|------------------|------------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|--------------|--------------|--------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|-----------------|------------------------|----------------|--------------------------------------|-------------------------------|------------------------------|----------------|-------------------|--------------|--------------|---------------|---------------|--------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------|---------------|---------------|--------------|------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
| Total diff               | REGR(+3.27%)  | REGR(+1.71%)    | REGR(+0.78%) | REGR(+0.31%)                         | REGR(+0.26%)    | SAME(+0.01%)   | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)       | SAME(+0.00%)       | SAME(+0.00%)                       | SAME(+0.00%)                       | SAME(+0.00%)   | SAME(+0.00%)               | SAME(+0.00%)           | SAME(+0.00%) | SAME(+0.00%)    | SAME(+0.00%)    | SAME(+0.00%)    | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)              | SAME(+0.00%)          | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)       | SAME(+0.00%)      | SAME(+0.00%)     | SAME(+0.00%)         | SAME(+0.00%)       | SAME(+0.00%)         | SAME(+0.00%)   | SAME(+0.00%)     | SAME(+0.00%)       | SAME(+0.00%)   | SAME(+0.00%)                | SAME(+0.00%)                | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)                                             | SAME(+0.00%)                                 | SAME(+0.00%)                                 | SAME(+0.00%)                                               | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)             | SAME(+0.00%)             | SAME(+0.00%)       | SAME(+0.00%)       | SAME(+0.00%) | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)         | SAME(+0.00%)         | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)    | SAME(+0.00%)                         | SAME(+0.00%)           | SAME(+0.00%)         | SAME(+0.00%)     | SAME(+0.00%)         | SAME(+0.00%)     | SAME(+0.00%)   | SAME(+0.00%)    | SAME(+0.00%)    | SAME(+0.00%)                  | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)         | SAME(+0.00%)     | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)           | SAME(+0.00%)           | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)        | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)                | SAME(+0.00%)                | SAME(+0.00%)       | SAME(+0.00%)       | SAME(+0.00%)       | SAME(+0.00%)       | SAME(+0.00%)                  | SAME(+0.00%)                  | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)                  | SAME(+0.00%)                  | SAME(+0.00%)                  | SAME(+0.00%)                      | SAME(+0.00%)                  | SAME(+0.00%)                 | SAME(+0.00%)                 | SAME(+0.00%)                 | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)      | SAME(+0.00%)         | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)       | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)                         | SAME(+0.00%)                         | SAME(+0.00%)   | SAME(+0.00%)               | SAME(+0.00%)           | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)        | SAME(+0.00%)    | SAME(+0.00%)           | SAME(+0.00%)           | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)                         | SAME(+0.00%) | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)      | SAME(+0.00%)      | SAME(+0.00%)      | SAME(+0.00%)      | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)               | SAME(+0.00%)                 | SAME(+0.00%)             | SAME(+0.00%)          | SAME(+0.00%)          | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)              | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)        | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)      | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)  | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)     | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%)   | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)     | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)   | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)       | SAME(+0.00%)   | SAME(+0.00%)                         | SAME(+0.00%)                         | SAME(+0.00%)                | SAME(+0.00%)                | SAME(+0.00%)                              | SAME(+0.00%)    | SAME(+0.00%)    | SAME(+0.00%)                  | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%) | SAME(+0.00%) | SAME(+0.00%)     | SAME(+0.00%)     | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)  | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%)            | SAME(+0.00%)                | SAME(+0.00%) | SAME(+0.00%) | SAME(-0.01%) | SAME(-0.10%) | IMPR(-0.10%)              | IMPR(-0.10%)              | IMPR(-0.10%)              | IMPR(-0.21%)               | IMPR(-0.21%)               | IMPR(-0.21%)               | IMPR(-0.48%) | IMPR(-0.50%)              | IMPR(-0.50%)              | IMPR(-0.50%)              | IMPR(-0.96%)               | IMPR(-0.96%)               | IMPR(-0.96%)               | IMPR(-1.93%)    | IMPR(-2.15%)           | IMPR(-2.21%)   | IMPR(-2.26%)                         | IMPR(-4.49%)                  | IMPR(-6.69%)                 | IMPR(-7.99%)   | IMPR(-8.46%)      | IMPR(-9.66%) | IMPR(-9.74%) | IMPR(-10.48%) | IMPR(-10.48%) | IMPR(-10.98%)                                    | IMPR(-10.99%)                                                | IMPR(-11.01%)                                    | IMPR(-11.47%) | IMPR(-11.56%) | -0.41%       |       1.94 | -0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      | +0.00%      |
|--------------------------|---------------|-----------------|--------------|--------------------------------------|-----------------|----------------|--------------|--------------|--------------|--------------|--------------------|--------------------|------------------------------------|------------------------------------|----------------|----------------------------|------------------------|--------------|-----------------|-----------------|-----------------|--------------|--------------|---------------------------|---------------------------|-----------------------|-----------------------|---------------------------|-----------------------|---------------|---------------|-------------------------|-------------------------|--------------------|-------------------|------------------|----------------------|--------------------|----------------------|----------------|------------------|--------------------|----------------|-----------------------------|-----------------------------|----------------|----------------|----------------------------------------------------------|----------------------------------------------|----------------------------------------------|------------------------------------------------------------|-----------------------|-----------------------|-----------------------|-----------------------|--------------|--------------|-----------------------|-----------------------|------------------|------------------|-------------------------|-------------------------|--------------|--------------|------------------|------------------|--------------|--------------|----------------|----------------|---------------|---------------|--------------------------|--------------------------|--------------------|--------------------|--------------|-------------------------|-------------------------|---------------|---------------|----------------------|----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------------|-----------------|--------------------------------------|------------------------|----------------------|------------------|----------------------|------------------|----------------|-----------------|-----------------|-------------------------------|--------------|--------------|----------------------|------------------|--------------|--------------|------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|--------------|-------------------------|-------------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|---------------------|----------------|----------------|-----------------------------|-----------------------------|--------------------|--------------------|--------------------|--------------------|-------------------------------|-------------------------------|------------------|------------------|-------------------------------|-------------------------------|-------------------------------|-----------------------------------|-------------------------------|------------------------------|------------------------------|------------------------------|--------------|--------------|--------------|--------------|-------------------|----------------------|--------------|--------------|--------------------|--------------|--------------|--------------|--------------|--------------------------------------|--------------------------------------|----------------|----------------------------|------------------------|--------------|--------------|--------------|--------------|--------------|---------------------|-----------------|------------------------|------------------------|-----------------------|-----------------------|--------------------------------------|--------------|-----------------------|-----------------------|-------------------|-------------------|-------------------|-------------------|------------------|------------------|------------------|------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|----------------------------|------------------------------|--------------------------|-----------------------|-----------------------|------------------|------------------|------------------|------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|---------------------------|------------------|------------------|------------------|------------------|--------------|--------------|---------------------|--------------|--------------|-------------------|--------------|--------------|------------------|------------------|------------------|------------------|---------------|----------------|----------------|------------------|-------------------------|-------------------------|-------------------------|----------------|----------------|----------------|----------------|--------------|--------------|---------------|---------------|------------------|--------------|--------------|----------------|--------------|--------------|--------------------|----------------|--------------------------------------|--------------------------------------|-----------------------------|-----------------------------|-------------------------------------------|-----------------|-----------------|-------------------------------|-----------------------------|-------------------------|--------------|--------------|------------------|------------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|---------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|-------------------------|-----------------------------|--------------|--------------|--------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|--------------|---------------------------|---------------------------|---------------------------|----------------------------|----------------------------|----------------------------|-----------------|------------------------|----------------|--------------------------------------|-------------------------------|------------------------------|----------------|-------------------|--------------|--------------|---------------|---------------|--------------------------------------------------|--------------------------------------------------------------|--------------------------------------------------|---------------|---------------|--------------|------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|