Question re: behavior of `--bind-to socket` when number of processes exceeds available sockets

darrylabbate commented 1 year ago

I have a question about a difference I noticed between 4.1.x and 5.x/main. The 5.x behavior makes sense to me, but I'm not sure if it's intentional or not. It may be an unexpected change for some users.

I noticed Open MPI@main complains about --bind-to socket when requesting more processes than available sockets:

# main branch
[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket hostname
--------------------------------------------------------------------------
Binding is performed to the first available specified object type
within the object where the process was mapped. In other words,
binding can only be done to the mapped object or to a resource
located beneath that object.

The specified binding lies above the mapping object type:

  Mapping level: BYCORE
  Binding level: PACKAGE

Please correct the map/bind directives and try again.
--------------------------------------------------------------------------

However, this works with 4.1.x:

# 4.1.5
[ec2-user@c7g-st-n64-1 ~]$ /opt/amazon/openmpi/bin/mpirun --report-bindings -n 2 --bind-to socket hostname
c7g-st-n64-1
[c7g-st-n64-1:06130] MCW rank 0 is not bound (or bound to all available processors)
[c7g-st-n64-1:06130] MCW rank 1 is not bound (or bound to all available processors)
c7g-st-n64-1

Specifying --map-by ppr:2:node seems to trigger an error in PRRTE:

# main branch
[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by ppr:2:node hostname
--------------------------------------------------------------------------
While computing bindings, we found no available cpus on
the following node:

  Node:  c7g-st-n64-1

Please check your allocation.
--------------------------------------------------------------------------

...yet it works fine with 4.1.x:

# 4.1.5
[ec2-user@c7g-st-n64-1 ~]$ /opt/amazon/openmpi/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by ppr:2:node hostname
c7g-st-n64-1
[c7g-st-n64-1:07180] MCW rank 0 is not bound (or bound to all available processors)
[c7g-st-n64-1:07180] MCW rank 1 is not bound (or bound to all available processors)
c7g-st-n64-1

I can, however, --oversubscribe with no issue:

# main branch
[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --oversubscribe hostname
[c7g-st-n64-1:07244] Rank 0 is not bound (or bound to all available processors)
[c7g-st-n64-1:07244] Rank 1 is not bound (or bound to all available processors)
c7g-st-n64-1
c7g-st-n64-1

For sake of comparison, I ran the above commands with an identical setup on a host with 2 sockets

--bind-to socket,main (same error as host with 1 socket)

[ec2-user@c5n-st-n72-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket hostname
--------------------------------------------------------------------------
Binding is performed to the first available specified object type
within the object where the process was mapped. In other words,
binding can only be done to the mapped object or to a resource
located beneath that object.

The specified binding lies above the mapping object type:

  Mapping level: BYCORE
  Binding level: PACKAGE

Please correct the map/bind directives and try again.
--------------------------------------------------------------------------

--bind-to socket --map-by ppr:2:node,main (successful, unlike the host with 1 socket)

[ec2-user@c5n-st-n72-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by ppr:2:node hostname
[c5n-st-n72-1:60273] Rank 0 bound to package[0][core:0-17]
[c5n-st-n72-1:60273] Rank 1 bound to package[1][core:18-35]
c5n-st-n72-1
c5n-st-n72-1

--oversubscribe,main (same as host with 1 socket)

[ec2-user@c5n-st-n72-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --oversubscribe hostname
[c5n-st-n72-1:60354] Rank 0 is not bound (or bound to all available processors)
[c5n-st-n72-1:60354] Rank 1 is not bound (or bound to all available processors)
c5n-st-n72-1
c5n-st-n72-1

--bind-to socket,4.1.5

[ec2-user@c5n-st-n72-1 ~]$ /opt/amazon/openmpi/bin/mpirun --report-bindings -n 2 --bind-to socket hostname
c5n-st-n72-1
c5n-st-n72-1

--bind-to socket --map-by ppr:2:node,4.1.5 (same as host with 1 socket)

[ec2-user@c5n-st-n72-1 ~]$ /opt/amazon/openmpi/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by ppr:2:node hostname
c5n-st-n72-1
c5n-st-n72-1

--oversubscribe,4.1.5

[ec2-user@c5n-st-n72-1 ~]$ /opt/amazon/openmpi/bin/mpirun --report-bindings -n 2 --oversubscribe hostname
c5n-st-n72-1
c5n-st-n72-1

lscpu output for host with 1 socket (c7g.16xlarge EC2 instance)

Architecture:        aarch64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  1
Core(s) per socket:  64
Socket(s):           1
NUMA node(s):        1
Vendor ID:           ARM
Model:               1
Stepping:            r1p1
BogoMIPS:            2100.00
L1d cache:           64K
L1i cache:           64K
L2 cache:            1024K
L3 cache:            32768K
NUMA node0 CPU(s):   0-63
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma lrcpc dcpop sha3 sm3 sm4 asimddp sha512 sve asimdfhm dit uscat ilrcpc flagm ssbs paca pacg dcpodp svei8mm svebf16 i8mm bf16 dgh rng

lscpu output for host with 2 sockets (c5n.18xlarge EC2 instance)

Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              72
On-line CPU(s) list: 0-71
Thread(s) per core:  2
Core(s) per socket:  18
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               85
Model name:          Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
Stepping:            4
CPU MHz:             3400.526
BogoMIPS:            5999.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            1024K
L3 cache:            25344K
NUMA node0 CPU(s):   0-17,36-53
NUMA node1 CPU(s):   18-35,54-71
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq monitor ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves ida arat pku ospke

rhc54 commented 1 year ago

It's intentional. For two processes, we default to --map-by core --bind-to core. If you override the binding to be by socket, then this will error out as the socket layer is above that of the core. It confuses the accounting system, which is required to track which cores are actually occupied. If you want to bind to the socket, then you need to map by at least the socket level.

rhc54 commented 1 year ago

FWIW, I'm not ruling out the possibility of restoring the ability to "bind upward" from where you map. I had to revise the mapping accounting system to support detailed management of cores and threads. One consequence of that was the revised binding rule.

I think there is a way around that rule, but it would take some effort to do and to fully debug all the unintended consequences of the change. For now, that's outside the time available to me, so it is "work for a rainy day" at some future point.

rhc54 commented 1 year ago

BTW: you can recover the OMPI v4.x behavior by simply adding --map-by slot to your mpirun cmd line

darrylabbate commented 1 year ago

BTW: you can recover the OMPI v4.x behavior by simply adding --map-by slot to your mpirun cmd line

That doesn't seem to work in conjunction with --bind-to socket

[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by slot hostname
--------------------------------------------------------------------------
While computing bindings, we found no available cpus on
the following node:

  Node:  c7g-st-n64-1

Please check your allocation.
--------------------------------------------------------------------------

--map-by slot on its own still binds both processes to their own core

[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --map-by slot hostname
[c7g-st-n64-1:59837] Rank 0 bound to package[0][core:0]
[c7g-st-n64-1:59837] Rank 1 bound to package[0][core:1]
c7g-st-n64-1
c7g-st-n64-1

I tried --map-by socket on its own and it seems to work similarly to 4.1.x, but the binding output indicates each process is bound differently

[ec2-user@c7g-st-n64-1 ~]$ ~/.local/bin/mpirun --report-bindings -n 2 --map-by socket hostname
[c7g-st-n64-1:59953] Rank 0 bound to package[0][core:0-63]
[c7g-st-n64-1:59953] Rank 1 is not bound (or bound to all available processors)
c7g-st-n64-1
c7g-st-n64-1

rhc54 commented 1 year ago

Oh - this is the oddball Amazon topology again. I fixed the map/bind problem for it, but it isn't in OMPI main or OMPI v5 yet. If you are using the main branch, you can go to 3rd-party/prrte, checkout the master branch and do a git pull to obtain the changes. Probably need to do the same for the 3rd-party/openpmix submodule.

darrylabbate commented 1 year ago

Hmm, I pulled the latest commits for both openpmix and prrte, but the above commands all yield the same output

--map-by slot:oversubscribe on its own works, but you may as well use --oversubscribe in this case

[ec2-user@c7g-st-n64-1 prrte]$ ~/.local/bin/mpirun --report-bindings -n 2 --map-by slot:oversubscribe hostname
[c7g-st-n64-1:55403] Rank 0 is not bound (or bound to all available processors)
[c7g-st-n64-1:55403] Rank 1 is not bound (or bound to all available processors)
c7g-st-n64-1
c7g-st-n64-1

Side note: I did find a segmentation fault playing around with some of the options:

[ec2-user@c7g-st-n64-1 prrte]$ ~/.local/bin/mpirun --report-bindings -n 2 --bind-to socket --map-by ppr:2 hostname
Segmentation fault

rhc54 commented 1 year ago

Odd - works fine for me. Can you provide your topology (the xml output from lstopo)? Maybe this is yet another oddball one.

darrylabbate commented 1 year ago

Sure

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE topology SYSTEM "hwloc.dtd">
<topology>
  <object type="Machine" os_index="0" cpuset="0xffffffff,0xffffffff" complete_cpuset="0xffffffff,0xffffffff" online_cpuset="0xffffffff,0xffffffff" allowed_cpuset="0xffffffff,0xffffffff" local_memory="132659404800">
    <page_type size="4096" count="28782558"/>
    <page_type size="65536" count="0"/>
    <page_type size="2097152" count="7041"/>
    <page_type size="33554432" count="0"/>
    <page_type size="1073741824" count="0"/>
    <info name="DMIProductName" value="c7g.16xlarge"/>
    <info name="DMIProductVersion" value=""/>
    <info name="DMIBoardVendor" value="Amazon EC2"/>
    <info name="DMIBoardName" value=""/>
    <info name="DMIBoardVersion" value=""/>
    <info name="DMIChassisVendor" value="Amazon EC2"/>
    <info name="DMIChassisType" value="1"/>
    <info name="DMIChassisVersion" value=""/>
    <info name="DMIChassisAssetTag" value="Amazon EC2"/>
    <info name="DMIBIOSVendor" value="Amazon EC2"/>
    <info name="DMIBIOSVersion" value="1.0"/>
    <info name="DMIBIOSDate" value="11/1/2018"/>
    <info name="DMISysVendor" value="Amazon EC2"/>
    <info name="Backend" value="Linux"/>
    <info name="LinuxCgroup" value="/"/>
    <info name="OSName" value="Linux"/>
    <info name="OSRelease" value="5.10.165-143.735.amzn2.aarch64"/>
    <info name="OSVersion" value="#1 SMP Wed Jan 25 03:13:36 UTC 2023"/>
    <info name="HostName" value="c7g-st-n64-1"/>
    <info name="Architecture" value="aarch64"/>
    <info name="hwlocVersion" value="1.11.8"/>
    <info name="ProcessName" value="lstopo"/>
    <object type="Package" os_index="60" cpuset="0xffffffff,0xffffffff" complete_cpuset="0xffffffff,0xffffffff" online_cpuset="0xffffffff,0xffffffff" allowed_cpuset="0xffffffff,0xffffffff">
      <object type="Cache" cpuset="0xffffffff,0xffffffff" complete_cpuset="0xffffffff,0xffffffff" online_cpuset="0xffffffff,0xffffffff" allowed_cpuset="0xffffffff,0xffffffff" cache_size="33554432" depth="3" cache_linesize="64" cache_associativity="16" cache_type="0">
        <object type="Cache" cpuset="0x00000001" complete_cpuset="0x00000001" online_cpuset="0x00000001" allowed_cpuset="0x00000001" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000001" complete_cpuset="0x00000001" online_cpuset="0x00000001" allowed_cpuset="0x00000001" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000001" complete_cpuset="0x00000001" online_cpuset="0x00000001" allowed_cpuset="0x00000001" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="0" cpuset="0x00000001" complete_cpuset="0x00000001" online_cpuset="0x00000001" allowed_cpuset="0x00000001">
                <object type="PU" os_index="0" cpuset="0x00000001" complete_cpuset="0x00000001" online_cpuset="0x00000001" allowed_cpuset="0x00000001"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000002" complete_cpuset="0x00000002" online_cpuset="0x00000002" allowed_cpuset="0x00000002" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000002" complete_cpuset="0x00000002" online_cpuset="0x00000002" allowed_cpuset="0x00000002" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000002" complete_cpuset="0x00000002" online_cpuset="0x00000002" allowed_cpuset="0x00000002" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="1" cpuset="0x00000002" complete_cpuset="0x00000002" online_cpuset="0x00000002" allowed_cpuset="0x00000002">
                <object type="PU" os_index="1" cpuset="0x00000002" complete_cpuset="0x00000002" online_cpuset="0x00000002" allowed_cpuset="0x00000002"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000004" complete_cpuset="0x00000004" online_cpuset="0x00000004" allowed_cpuset="0x00000004" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000004" complete_cpuset="0x00000004" online_cpuset="0x00000004" allowed_cpuset="0x00000004" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000004" complete_cpuset="0x00000004" online_cpuset="0x00000004" allowed_cpuset="0x00000004" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="2" cpuset="0x00000004" complete_cpuset="0x00000004" online_cpuset="0x00000004" allowed_cpuset="0x00000004">
                <object type="PU" os_index="2" cpuset="0x00000004" complete_cpuset="0x00000004" online_cpuset="0x00000004" allowed_cpuset="0x00000004"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000008" complete_cpuset="0x00000008" online_cpuset="0x00000008" allowed_cpuset="0x00000008" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000008" complete_cpuset="0x00000008" online_cpuset="0x00000008" allowed_cpuset="0x00000008" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000008" complete_cpuset="0x00000008" online_cpuset="0x00000008" allowed_cpuset="0x00000008" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="3" cpuset="0x00000008" complete_cpuset="0x00000008" online_cpuset="0x00000008" allowed_cpuset="0x00000008">
                <object type="PU" os_index="3" cpuset="0x00000008" complete_cpuset="0x00000008" online_cpuset="0x00000008" allowed_cpuset="0x00000008"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000010" complete_cpuset="0x00000010" online_cpuset="0x00000010" allowed_cpuset="0x00000010" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000010" complete_cpuset="0x00000010" online_cpuset="0x00000010" allowed_cpuset="0x00000010" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000010" complete_cpuset="0x00000010" online_cpuset="0x00000010" allowed_cpuset="0x00000010" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="4" cpuset="0x00000010" complete_cpuset="0x00000010" online_cpuset="0x00000010" allowed_cpuset="0x00000010">
                <object type="PU" os_index="4" cpuset="0x00000010" complete_cpuset="0x00000010" online_cpuset="0x00000010" allowed_cpuset="0x00000010"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000020" complete_cpuset="0x00000020" online_cpuset="0x00000020" allowed_cpuset="0x00000020" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000020" complete_cpuset="0x00000020" online_cpuset="0x00000020" allowed_cpuset="0x00000020" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000020" complete_cpuset="0x00000020" online_cpuset="0x00000020" allowed_cpuset="0x00000020" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="5" cpuset="0x00000020" complete_cpuset="0x00000020" online_cpuset="0x00000020" allowed_cpuset="0x00000020">
                <object type="PU" os_index="5" cpuset="0x00000020" complete_cpuset="0x00000020" online_cpuset="0x00000020" allowed_cpuset="0x00000020"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000040" complete_cpuset="0x00000040" online_cpuset="0x00000040" allowed_cpuset="0x00000040" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000040" complete_cpuset="0x00000040" online_cpuset="0x00000040" allowed_cpuset="0x00000040" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000040" complete_cpuset="0x00000040" online_cpuset="0x00000040" allowed_cpuset="0x00000040" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="6" cpuset="0x00000040" complete_cpuset="0x00000040" online_cpuset="0x00000040" allowed_cpuset="0x00000040">
                <object type="PU" os_index="6" cpuset="0x00000040" complete_cpuset="0x00000040" online_cpuset="0x00000040" allowed_cpuset="0x00000040"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000080" complete_cpuset="0x00000080" online_cpuset="0x00000080" allowed_cpuset="0x00000080" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000080" complete_cpuset="0x00000080" online_cpuset="0x00000080" allowed_cpuset="0x00000080" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000080" complete_cpuset="0x00000080" online_cpuset="0x00000080" allowed_cpuset="0x00000080" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="7" cpuset="0x00000080" complete_cpuset="0x00000080" online_cpuset="0x00000080" allowed_cpuset="0x00000080">
                <object type="PU" os_index="7" cpuset="0x00000080" complete_cpuset="0x00000080" online_cpuset="0x00000080" allowed_cpuset="0x00000080"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000100" complete_cpuset="0x00000100" online_cpuset="0x00000100" allowed_cpuset="0x00000100" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000100" complete_cpuset="0x00000100" online_cpuset="0x00000100" allowed_cpuset="0x00000100" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000100" complete_cpuset="0x00000100" online_cpuset="0x00000100" allowed_cpuset="0x00000100" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="8" cpuset="0x00000100" complete_cpuset="0x00000100" online_cpuset="0x00000100" allowed_cpuset="0x00000100">
                <object type="PU" os_index="8" cpuset="0x00000100" complete_cpuset="0x00000100" online_cpuset="0x00000100" allowed_cpuset="0x00000100"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000200" complete_cpuset="0x00000200" online_cpuset="0x00000200" allowed_cpuset="0x00000200" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000200" complete_cpuset="0x00000200" online_cpuset="0x00000200" allowed_cpuset="0x00000200" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000200" complete_cpuset="0x00000200" online_cpuset="0x00000200" allowed_cpuset="0x00000200" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="9" cpuset="0x00000200" complete_cpuset="0x00000200" online_cpuset="0x00000200" allowed_cpuset="0x00000200">
                <object type="PU" os_index="9" cpuset="0x00000200" complete_cpuset="0x00000200" online_cpuset="0x00000200" allowed_cpuset="0x00000200"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000400" complete_cpuset="0x00000400" online_cpuset="0x00000400" allowed_cpuset="0x00000400" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000400" complete_cpuset="0x00000400" online_cpuset="0x00000400" allowed_cpuset="0x00000400" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000400" complete_cpuset="0x00000400" online_cpuset="0x00000400" allowed_cpuset="0x00000400" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="10" cpuset="0x00000400" complete_cpuset="0x00000400" online_cpuset="0x00000400" allowed_cpuset="0x00000400">
                <object type="PU" os_index="10" cpuset="0x00000400" complete_cpuset="0x00000400" online_cpuset="0x00000400" allowed_cpuset="0x00000400"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000800" complete_cpuset="0x00000800" online_cpuset="0x00000800" allowed_cpuset="0x00000800" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000800" complete_cpuset="0x00000800" online_cpuset="0x00000800" allowed_cpuset="0x00000800" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000800" complete_cpuset="0x00000800" online_cpuset="0x00000800" allowed_cpuset="0x00000800" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="11" cpuset="0x00000800" complete_cpuset="0x00000800" online_cpuset="0x00000800" allowed_cpuset="0x00000800">
                <object type="PU" os_index="11" cpuset="0x00000800" complete_cpuset="0x00000800" online_cpuset="0x00000800" allowed_cpuset="0x00000800"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00001000" complete_cpuset="0x00001000" online_cpuset="0x00001000" allowed_cpuset="0x00001000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00001000" complete_cpuset="0x00001000" online_cpuset="0x00001000" allowed_cpuset="0x00001000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00001000" complete_cpuset="0x00001000" online_cpuset="0x00001000" allowed_cpuset="0x00001000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="12" cpuset="0x00001000" complete_cpuset="0x00001000" online_cpuset="0x00001000" allowed_cpuset="0x00001000">
                <object type="PU" os_index="12" cpuset="0x00001000" complete_cpuset="0x00001000" online_cpuset="0x00001000" allowed_cpuset="0x00001000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00002000" complete_cpuset="0x00002000" online_cpuset="0x00002000" allowed_cpuset="0x00002000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00002000" complete_cpuset="0x00002000" online_cpuset="0x00002000" allowed_cpuset="0x00002000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00002000" complete_cpuset="0x00002000" online_cpuset="0x00002000" allowed_cpuset="0x00002000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="13" cpuset="0x00002000" complete_cpuset="0x00002000" online_cpuset="0x00002000" allowed_cpuset="0x00002000">
                <object type="PU" os_index="13" cpuset="0x00002000" complete_cpuset="0x00002000" online_cpuset="0x00002000" allowed_cpuset="0x00002000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00004000" complete_cpuset="0x00004000" online_cpuset="0x00004000" allowed_cpuset="0x00004000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00004000" complete_cpuset="0x00004000" online_cpuset="0x00004000" allowed_cpuset="0x00004000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00004000" complete_cpuset="0x00004000" online_cpuset="0x00004000" allowed_cpuset="0x00004000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="14" cpuset="0x00004000" complete_cpuset="0x00004000" online_cpuset="0x00004000" allowed_cpuset="0x00004000">
                <object type="PU" os_index="14" cpuset="0x00004000" complete_cpuset="0x00004000" online_cpuset="0x00004000" allowed_cpuset="0x00004000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00008000" complete_cpuset="0x00008000" online_cpuset="0x00008000" allowed_cpuset="0x00008000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00008000" complete_cpuset="0x00008000" online_cpuset="0x00008000" allowed_cpuset="0x00008000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00008000" complete_cpuset="0x00008000" online_cpuset="0x00008000" allowed_cpuset="0x00008000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="15" cpuset="0x00008000" complete_cpuset="0x00008000" online_cpuset="0x00008000" allowed_cpuset="0x00008000">
                <object type="PU" os_index="15" cpuset="0x00008000" complete_cpuset="0x00008000" online_cpuset="0x00008000" allowed_cpuset="0x00008000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00010000" complete_cpuset="0x00010000" online_cpuset="0x00010000" allowed_cpuset="0x00010000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00010000" complete_cpuset="0x00010000" online_cpuset="0x00010000" allowed_cpuset="0x00010000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00010000" complete_cpuset="0x00010000" online_cpuset="0x00010000" allowed_cpuset="0x00010000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="16" cpuset="0x00010000" complete_cpuset="0x00010000" online_cpuset="0x00010000" allowed_cpuset="0x00010000">
                <object type="PU" os_index="16" cpuset="0x00010000" complete_cpuset="0x00010000" online_cpuset="0x00010000" allowed_cpuset="0x00010000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00020000" complete_cpuset="0x00020000" online_cpuset="0x00020000" allowed_cpuset="0x00020000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00020000" complete_cpuset="0x00020000" online_cpuset="0x00020000" allowed_cpuset="0x00020000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00020000" complete_cpuset="0x00020000" online_cpuset="0x00020000" allowed_cpuset="0x00020000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="17" cpuset="0x00020000" complete_cpuset="0x00020000" online_cpuset="0x00020000" allowed_cpuset="0x00020000">
                <object type="PU" os_index="17" cpuset="0x00020000" complete_cpuset="0x00020000" online_cpuset="0x00020000" allowed_cpuset="0x00020000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00040000" complete_cpuset="0x00040000" online_cpuset="0x00040000" allowed_cpuset="0x00040000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00040000" complete_cpuset="0x00040000" online_cpuset="0x00040000" allowed_cpuset="0x00040000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00040000" complete_cpuset="0x00040000" online_cpuset="0x00040000" allowed_cpuset="0x00040000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="18" cpuset="0x00040000" complete_cpuset="0x00040000" online_cpuset="0x00040000" allowed_cpuset="0x00040000">
                <object type="PU" os_index="18" cpuset="0x00040000" complete_cpuset="0x00040000" online_cpuset="0x00040000" allowed_cpuset="0x00040000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00080000" complete_cpuset="0x00080000" online_cpuset="0x00080000" allowed_cpuset="0x00080000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00080000" complete_cpuset="0x00080000" online_cpuset="0x00080000" allowed_cpuset="0x00080000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00080000" complete_cpuset="0x00080000" online_cpuset="0x00080000" allowed_cpuset="0x00080000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="19" cpuset="0x00080000" complete_cpuset="0x00080000" online_cpuset="0x00080000" allowed_cpuset="0x00080000">
                <object type="PU" os_index="19" cpuset="0x00080000" complete_cpuset="0x00080000" online_cpuset="0x00080000" allowed_cpuset="0x00080000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00100000" complete_cpuset="0x00100000" online_cpuset="0x00100000" allowed_cpuset="0x00100000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00100000" complete_cpuset="0x00100000" online_cpuset="0x00100000" allowed_cpuset="0x00100000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00100000" complete_cpuset="0x00100000" online_cpuset="0x00100000" allowed_cpuset="0x00100000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="20" cpuset="0x00100000" complete_cpuset="0x00100000" online_cpuset="0x00100000" allowed_cpuset="0x00100000">
                <object type="PU" os_index="20" cpuset="0x00100000" complete_cpuset="0x00100000" online_cpuset="0x00100000" allowed_cpuset="0x00100000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00200000" complete_cpuset="0x00200000" online_cpuset="0x00200000" allowed_cpuset="0x00200000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00200000" complete_cpuset="0x00200000" online_cpuset="0x00200000" allowed_cpuset="0x00200000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00200000" complete_cpuset="0x00200000" online_cpuset="0x00200000" allowed_cpuset="0x00200000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="21" cpuset="0x00200000" complete_cpuset="0x00200000" online_cpuset="0x00200000" allowed_cpuset="0x00200000">
                <object type="PU" os_index="21" cpuset="0x00200000" complete_cpuset="0x00200000" online_cpuset="0x00200000" allowed_cpuset="0x00200000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00400000" complete_cpuset="0x00400000" online_cpuset="0x00400000" allowed_cpuset="0x00400000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00400000" complete_cpuset="0x00400000" online_cpuset="0x00400000" allowed_cpuset="0x00400000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00400000" complete_cpuset="0x00400000" online_cpuset="0x00400000" allowed_cpuset="0x00400000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="22" cpuset="0x00400000" complete_cpuset="0x00400000" online_cpuset="0x00400000" allowed_cpuset="0x00400000">
                <object type="PU" os_index="22" cpuset="0x00400000" complete_cpuset="0x00400000" online_cpuset="0x00400000" allowed_cpuset="0x00400000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00800000" complete_cpuset="0x00800000" online_cpuset="0x00800000" allowed_cpuset="0x00800000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00800000" complete_cpuset="0x00800000" online_cpuset="0x00800000" allowed_cpuset="0x00800000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00800000" complete_cpuset="0x00800000" online_cpuset="0x00800000" allowed_cpuset="0x00800000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="23" cpuset="0x00800000" complete_cpuset="0x00800000" online_cpuset="0x00800000" allowed_cpuset="0x00800000">
                <object type="PU" os_index="23" cpuset="0x00800000" complete_cpuset="0x00800000" online_cpuset="0x00800000" allowed_cpuset="0x00800000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x01000000" complete_cpuset="0x01000000" online_cpuset="0x01000000" allowed_cpuset="0x01000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x01000000" complete_cpuset="0x01000000" online_cpuset="0x01000000" allowed_cpuset="0x01000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x01000000" complete_cpuset="0x01000000" online_cpuset="0x01000000" allowed_cpuset="0x01000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="24" cpuset="0x01000000" complete_cpuset="0x01000000" online_cpuset="0x01000000" allowed_cpuset="0x01000000">
                <object type="PU" os_index="24" cpuset="0x01000000" complete_cpuset="0x01000000" online_cpuset="0x01000000" allowed_cpuset="0x01000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x02000000" complete_cpuset="0x02000000" online_cpuset="0x02000000" allowed_cpuset="0x02000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x02000000" complete_cpuset="0x02000000" online_cpuset="0x02000000" allowed_cpuset="0x02000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x02000000" complete_cpuset="0x02000000" online_cpuset="0x02000000" allowed_cpuset="0x02000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="25" cpuset="0x02000000" complete_cpuset="0x02000000" online_cpuset="0x02000000" allowed_cpuset="0x02000000">
                <object type="PU" os_index="25" cpuset="0x02000000" complete_cpuset="0x02000000" online_cpuset="0x02000000" allowed_cpuset="0x02000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x04000000" complete_cpuset="0x04000000" online_cpuset="0x04000000" allowed_cpuset="0x04000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x04000000" complete_cpuset="0x04000000" online_cpuset="0x04000000" allowed_cpuset="0x04000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x04000000" complete_cpuset="0x04000000" online_cpuset="0x04000000" allowed_cpuset="0x04000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="26" cpuset="0x04000000" complete_cpuset="0x04000000" online_cpuset="0x04000000" allowed_cpuset="0x04000000">
                <object type="PU" os_index="26" cpuset="0x04000000" complete_cpuset="0x04000000" online_cpuset="0x04000000" allowed_cpuset="0x04000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x08000000" complete_cpuset="0x08000000" online_cpuset="0x08000000" allowed_cpuset="0x08000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x08000000" complete_cpuset="0x08000000" online_cpuset="0x08000000" allowed_cpuset="0x08000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x08000000" complete_cpuset="0x08000000" online_cpuset="0x08000000" allowed_cpuset="0x08000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="27" cpuset="0x08000000" complete_cpuset="0x08000000" online_cpuset="0x08000000" allowed_cpuset="0x08000000">
                <object type="PU" os_index="27" cpuset="0x08000000" complete_cpuset="0x08000000" online_cpuset="0x08000000" allowed_cpuset="0x08000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x10000000" complete_cpuset="0x10000000" online_cpuset="0x10000000" allowed_cpuset="0x10000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x10000000" complete_cpuset="0x10000000" online_cpuset="0x10000000" allowed_cpuset="0x10000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x10000000" complete_cpuset="0x10000000" online_cpuset="0x10000000" allowed_cpuset="0x10000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="28" cpuset="0x10000000" complete_cpuset="0x10000000" online_cpuset="0x10000000" allowed_cpuset="0x10000000">
                <object type="PU" os_index="28" cpuset="0x10000000" complete_cpuset="0x10000000" online_cpuset="0x10000000" allowed_cpuset="0x10000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x20000000" complete_cpuset="0x20000000" online_cpuset="0x20000000" allowed_cpuset="0x20000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x20000000" complete_cpuset="0x20000000" online_cpuset="0x20000000" allowed_cpuset="0x20000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x20000000" complete_cpuset="0x20000000" online_cpuset="0x20000000" allowed_cpuset="0x20000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="29" cpuset="0x20000000" complete_cpuset="0x20000000" online_cpuset="0x20000000" allowed_cpuset="0x20000000">
                <object type="PU" os_index="29" cpuset="0x20000000" complete_cpuset="0x20000000" online_cpuset="0x20000000" allowed_cpuset="0x20000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x40000000" complete_cpuset="0x40000000" online_cpuset="0x40000000" allowed_cpuset="0x40000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x40000000" complete_cpuset="0x40000000" online_cpuset="0x40000000" allowed_cpuset="0x40000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x40000000" complete_cpuset="0x40000000" online_cpuset="0x40000000" allowed_cpuset="0x40000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="30" cpuset="0x40000000" complete_cpuset="0x40000000" online_cpuset="0x40000000" allowed_cpuset="0x40000000">
                <object type="PU" os_index="30" cpuset="0x40000000" complete_cpuset="0x40000000" online_cpuset="0x40000000" allowed_cpuset="0x40000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x80000000" complete_cpuset="0x80000000" online_cpuset="0x80000000" allowed_cpuset="0x80000000" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x80000000" complete_cpuset="0x80000000" online_cpuset="0x80000000" allowed_cpuset="0x80000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x80000000" complete_cpuset="0x80000000" online_cpuset="0x80000000" allowed_cpuset="0x80000000" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="31" cpuset="0x80000000" complete_cpuset="0x80000000" online_cpuset="0x80000000" allowed_cpuset="0x80000000">
                <object type="PU" os_index="31" cpuset="0x80000000" complete_cpuset="0x80000000" online_cpuset="0x80000000" allowed_cpuset="0x80000000"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000001,0x0" complete_cpuset="0x00000001,0x0" online_cpuset="0x00000001,0x0" allowed_cpuset="0x00000001,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000001,0x0" complete_cpuset="0x00000001,0x0" online_cpuset="0x00000001,0x0" allowed_cpuset="0x00000001,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000001,0x0" complete_cpuset="0x00000001,0x0" online_cpuset="0x00000001,0x0" allowed_cpuset="0x00000001,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="32" cpuset="0x00000001,0x0" complete_cpuset="0x00000001,0x0" online_cpuset="0x00000001,0x0" allowed_cpuset="0x00000001,0x0">
                <object type="PU" os_index="32" cpuset="0x00000001,0x0" complete_cpuset="0x00000001,0x0" online_cpuset="0x00000001,0x0" allowed_cpuset="0x00000001,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000002,0x0" complete_cpuset="0x00000002,0x0" online_cpuset="0x00000002,0x0" allowed_cpuset="0x00000002,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000002,0x0" complete_cpuset="0x00000002,0x0" online_cpuset="0x00000002,0x0" allowed_cpuset="0x00000002,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000002,0x0" complete_cpuset="0x00000002,0x0" online_cpuset="0x00000002,0x0" allowed_cpuset="0x00000002,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="33" cpuset="0x00000002,0x0" complete_cpuset="0x00000002,0x0" online_cpuset="0x00000002,0x0" allowed_cpuset="0x00000002,0x0">
                <object type="PU" os_index="33" cpuset="0x00000002,0x0" complete_cpuset="0x00000002,0x0" online_cpuset="0x00000002,0x0" allowed_cpuset="0x00000002,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000004,0x0" complete_cpuset="0x00000004,0x0" online_cpuset="0x00000004,0x0" allowed_cpuset="0x00000004,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000004,0x0" complete_cpuset="0x00000004,0x0" online_cpuset="0x00000004,0x0" allowed_cpuset="0x00000004,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000004,0x0" complete_cpuset="0x00000004,0x0" online_cpuset="0x00000004,0x0" allowed_cpuset="0x00000004,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="34" cpuset="0x00000004,0x0" complete_cpuset="0x00000004,0x0" online_cpuset="0x00000004,0x0" allowed_cpuset="0x00000004,0x0">
                <object type="PU" os_index="34" cpuset="0x00000004,0x0" complete_cpuset="0x00000004,0x0" online_cpuset="0x00000004,0x0" allowed_cpuset="0x00000004,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000008,0x0" complete_cpuset="0x00000008,0x0" online_cpuset="0x00000008,0x0" allowed_cpuset="0x00000008,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000008,0x0" complete_cpuset="0x00000008,0x0" online_cpuset="0x00000008,0x0" allowed_cpuset="0x00000008,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000008,0x0" complete_cpuset="0x00000008,0x0" online_cpuset="0x00000008,0x0" allowed_cpuset="0x00000008,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="35" cpuset="0x00000008,0x0" complete_cpuset="0x00000008,0x0" online_cpuset="0x00000008,0x0" allowed_cpuset="0x00000008,0x0">
                <object type="PU" os_index="35" cpuset="0x00000008,0x0" complete_cpuset="0x00000008,0x0" online_cpuset="0x00000008,0x0" allowed_cpuset="0x00000008,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000010,0x0" complete_cpuset="0x00000010,0x0" online_cpuset="0x00000010,0x0" allowed_cpuset="0x00000010,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000010,0x0" complete_cpuset="0x00000010,0x0" online_cpuset="0x00000010,0x0" allowed_cpuset="0x00000010,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000010,0x0" complete_cpuset="0x00000010,0x0" online_cpuset="0x00000010,0x0" allowed_cpuset="0x00000010,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="36" cpuset="0x00000010,0x0" complete_cpuset="0x00000010,0x0" online_cpuset="0x00000010,0x0" allowed_cpuset="0x00000010,0x0">
                <object type="PU" os_index="36" cpuset="0x00000010,0x0" complete_cpuset="0x00000010,0x0" online_cpuset="0x00000010,0x0" allowed_cpuset="0x00000010,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000020,0x0" complete_cpuset="0x00000020,0x0" online_cpuset="0x00000020,0x0" allowed_cpuset="0x00000020,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000020,0x0" complete_cpuset="0x00000020,0x0" online_cpuset="0x00000020,0x0" allowed_cpuset="0x00000020,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000020,0x0" complete_cpuset="0x00000020,0x0" online_cpuset="0x00000020,0x0" allowed_cpuset="0x00000020,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="37" cpuset="0x00000020,0x0" complete_cpuset="0x00000020,0x0" online_cpuset="0x00000020,0x0" allowed_cpuset="0x00000020,0x0">
                <object type="PU" os_index="37" cpuset="0x00000020,0x0" complete_cpuset="0x00000020,0x0" online_cpuset="0x00000020,0x0" allowed_cpuset="0x00000020,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000040,0x0" complete_cpuset="0x00000040,0x0" online_cpuset="0x00000040,0x0" allowed_cpuset="0x00000040,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000040,0x0" complete_cpuset="0x00000040,0x0" online_cpuset="0x00000040,0x0" allowed_cpuset="0x00000040,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000040,0x0" complete_cpuset="0x00000040,0x0" online_cpuset="0x00000040,0x0" allowed_cpuset="0x00000040,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="38" cpuset="0x00000040,0x0" complete_cpuset="0x00000040,0x0" online_cpuset="0x00000040,0x0" allowed_cpuset="0x00000040,0x0">
                <object type="PU" os_index="38" cpuset="0x00000040,0x0" complete_cpuset="0x00000040,0x0" online_cpuset="0x00000040,0x0" allowed_cpuset="0x00000040,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000080,0x0" complete_cpuset="0x00000080,0x0" online_cpuset="0x00000080,0x0" allowed_cpuset="0x00000080,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000080,0x0" complete_cpuset="0x00000080,0x0" online_cpuset="0x00000080,0x0" allowed_cpuset="0x00000080,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000080,0x0" complete_cpuset="0x00000080,0x0" online_cpuset="0x00000080,0x0" allowed_cpuset="0x00000080,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="39" cpuset="0x00000080,0x0" complete_cpuset="0x00000080,0x0" online_cpuset="0x00000080,0x0" allowed_cpuset="0x00000080,0x0">
                <object type="PU" os_index="39" cpuset="0x00000080,0x0" complete_cpuset="0x00000080,0x0" online_cpuset="0x00000080,0x0" allowed_cpuset="0x00000080,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000100,0x0" complete_cpuset="0x00000100,0x0" online_cpuset="0x00000100,0x0" allowed_cpuset="0x00000100,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000100,0x0" complete_cpuset="0x00000100,0x0" online_cpuset="0x00000100,0x0" allowed_cpuset="0x00000100,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000100,0x0" complete_cpuset="0x00000100,0x0" online_cpuset="0x00000100,0x0" allowed_cpuset="0x00000100,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="40" cpuset="0x00000100,0x0" complete_cpuset="0x00000100,0x0" online_cpuset="0x00000100,0x0" allowed_cpuset="0x00000100,0x0">
                <object type="PU" os_index="40" cpuset="0x00000100,0x0" complete_cpuset="0x00000100,0x0" online_cpuset="0x00000100,0x0" allowed_cpuset="0x00000100,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000200,0x0" complete_cpuset="0x00000200,0x0" online_cpuset="0x00000200,0x0" allowed_cpuset="0x00000200,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000200,0x0" complete_cpuset="0x00000200,0x0" online_cpuset="0x00000200,0x0" allowed_cpuset="0x00000200,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000200,0x0" complete_cpuset="0x00000200,0x0" online_cpuset="0x00000200,0x0" allowed_cpuset="0x00000200,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="41" cpuset="0x00000200,0x0" complete_cpuset="0x00000200,0x0" online_cpuset="0x00000200,0x0" allowed_cpuset="0x00000200,0x0">
                <object type="PU" os_index="41" cpuset="0x00000200,0x0" complete_cpuset="0x00000200,0x0" online_cpuset="0x00000200,0x0" allowed_cpuset="0x00000200,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000400,0x0" complete_cpuset="0x00000400,0x0" online_cpuset="0x00000400,0x0" allowed_cpuset="0x00000400,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000400,0x0" complete_cpuset="0x00000400,0x0" online_cpuset="0x00000400,0x0" allowed_cpuset="0x00000400,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000400,0x0" complete_cpuset="0x00000400,0x0" online_cpuset="0x00000400,0x0" allowed_cpuset="0x00000400,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="42" cpuset="0x00000400,0x0" complete_cpuset="0x00000400,0x0" online_cpuset="0x00000400,0x0" allowed_cpuset="0x00000400,0x0">
                <object type="PU" os_index="42" cpuset="0x00000400,0x0" complete_cpuset="0x00000400,0x0" online_cpuset="0x00000400,0x0" allowed_cpuset="0x00000400,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00000800,0x0" complete_cpuset="0x00000800,0x0" online_cpuset="0x00000800,0x0" allowed_cpuset="0x00000800,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00000800,0x0" complete_cpuset="0x00000800,0x0" online_cpuset="0x00000800,0x0" allowed_cpuset="0x00000800,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00000800,0x0" complete_cpuset="0x00000800,0x0" online_cpuset="0x00000800,0x0" allowed_cpuset="0x00000800,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="43" cpuset="0x00000800,0x0" complete_cpuset="0x00000800,0x0" online_cpuset="0x00000800,0x0" allowed_cpuset="0x00000800,0x0">
                <object type="PU" os_index="43" cpuset="0x00000800,0x0" complete_cpuset="0x00000800,0x0" online_cpuset="0x00000800,0x0" allowed_cpuset="0x00000800,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00001000,0x0" complete_cpuset="0x00001000,0x0" online_cpuset="0x00001000,0x0" allowed_cpuset="0x00001000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00001000,0x0" complete_cpuset="0x00001000,0x0" online_cpuset="0x00001000,0x0" allowed_cpuset="0x00001000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00001000,0x0" complete_cpuset="0x00001000,0x0" online_cpuset="0x00001000,0x0" allowed_cpuset="0x00001000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="44" cpuset="0x00001000,0x0" complete_cpuset="0x00001000,0x0" online_cpuset="0x00001000,0x0" allowed_cpuset="0x00001000,0x0">
                <object type="PU" os_index="44" cpuset="0x00001000,0x0" complete_cpuset="0x00001000,0x0" online_cpuset="0x00001000,0x0" allowed_cpuset="0x00001000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00002000,0x0" complete_cpuset="0x00002000,0x0" online_cpuset="0x00002000,0x0" allowed_cpuset="0x00002000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00002000,0x0" complete_cpuset="0x00002000,0x0" online_cpuset="0x00002000,0x0" allowed_cpuset="0x00002000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00002000,0x0" complete_cpuset="0x00002000,0x0" online_cpuset="0x00002000,0x0" allowed_cpuset="0x00002000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="45" cpuset="0x00002000,0x0" complete_cpuset="0x00002000,0x0" online_cpuset="0x00002000,0x0" allowed_cpuset="0x00002000,0x0">
                <object type="PU" os_index="45" cpuset="0x00002000,0x0" complete_cpuset="0x00002000,0x0" online_cpuset="0x00002000,0x0" allowed_cpuset="0x00002000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00004000,0x0" complete_cpuset="0x00004000,0x0" online_cpuset="0x00004000,0x0" allowed_cpuset="0x00004000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00004000,0x0" complete_cpuset="0x00004000,0x0" online_cpuset="0x00004000,0x0" allowed_cpuset="0x00004000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00004000,0x0" complete_cpuset="0x00004000,0x0" online_cpuset="0x00004000,0x0" allowed_cpuset="0x00004000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="46" cpuset="0x00004000,0x0" complete_cpuset="0x00004000,0x0" online_cpuset="0x00004000,0x0" allowed_cpuset="0x00004000,0x0">
                <object type="PU" os_index="46" cpuset="0x00004000,0x0" complete_cpuset="0x00004000,0x0" online_cpuset="0x00004000,0x0" allowed_cpuset="0x00004000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00008000,0x0" complete_cpuset="0x00008000,0x0" online_cpuset="0x00008000,0x0" allowed_cpuset="0x00008000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00008000,0x0" complete_cpuset="0x00008000,0x0" online_cpuset="0x00008000,0x0" allowed_cpuset="0x00008000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00008000,0x0" complete_cpuset="0x00008000,0x0" online_cpuset="0x00008000,0x0" allowed_cpuset="0x00008000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="47" cpuset="0x00008000,0x0" complete_cpuset="0x00008000,0x0" online_cpuset="0x00008000,0x0" allowed_cpuset="0x00008000,0x0">
                <object type="PU" os_index="47" cpuset="0x00008000,0x0" complete_cpuset="0x00008000,0x0" online_cpuset="0x00008000,0x0" allowed_cpuset="0x00008000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00010000,0x0" complete_cpuset="0x00010000,0x0" online_cpuset="0x00010000,0x0" allowed_cpuset="0x00010000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00010000,0x0" complete_cpuset="0x00010000,0x0" online_cpuset="0x00010000,0x0" allowed_cpuset="0x00010000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00010000,0x0" complete_cpuset="0x00010000,0x0" online_cpuset="0x00010000,0x0" allowed_cpuset="0x00010000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="48" cpuset="0x00010000,0x0" complete_cpuset="0x00010000,0x0" online_cpuset="0x00010000,0x0" allowed_cpuset="0x00010000,0x0">
                <object type="PU" os_index="48" cpuset="0x00010000,0x0" complete_cpuset="0x00010000,0x0" online_cpuset="0x00010000,0x0" allowed_cpuset="0x00010000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00020000,0x0" complete_cpuset="0x00020000,0x0" online_cpuset="0x00020000,0x0" allowed_cpuset="0x00020000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00020000,0x0" complete_cpuset="0x00020000,0x0" online_cpuset="0x00020000,0x0" allowed_cpuset="0x00020000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00020000,0x0" complete_cpuset="0x00020000,0x0" online_cpuset="0x00020000,0x0" allowed_cpuset="0x00020000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="49" cpuset="0x00020000,0x0" complete_cpuset="0x00020000,0x0" online_cpuset="0x00020000,0x0" allowed_cpuset="0x00020000,0x0">
                <object type="PU" os_index="49" cpuset="0x00020000,0x0" complete_cpuset="0x00020000,0x0" online_cpuset="0x00020000,0x0" allowed_cpuset="0x00020000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00040000,0x0" complete_cpuset="0x00040000,0x0" online_cpuset="0x00040000,0x0" allowed_cpuset="0x00040000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00040000,0x0" complete_cpuset="0x00040000,0x0" online_cpuset="0x00040000,0x0" allowed_cpuset="0x00040000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00040000,0x0" complete_cpuset="0x00040000,0x0" online_cpuset="0x00040000,0x0" allowed_cpuset="0x00040000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="50" cpuset="0x00040000,0x0" complete_cpuset="0x00040000,0x0" online_cpuset="0x00040000,0x0" allowed_cpuset="0x00040000,0x0">
                <object type="PU" os_index="50" cpuset="0x00040000,0x0" complete_cpuset="0x00040000,0x0" online_cpuset="0x00040000,0x0" allowed_cpuset="0x00040000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00080000,0x0" complete_cpuset="0x00080000,0x0" online_cpuset="0x00080000,0x0" allowed_cpuset="0x00080000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00080000,0x0" complete_cpuset="0x00080000,0x0" online_cpuset="0x00080000,0x0" allowed_cpuset="0x00080000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00080000,0x0" complete_cpuset="0x00080000,0x0" online_cpuset="0x00080000,0x0" allowed_cpuset="0x00080000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="51" cpuset="0x00080000,0x0" complete_cpuset="0x00080000,0x0" online_cpuset="0x00080000,0x0" allowed_cpuset="0x00080000,0x0">
                <object type="PU" os_index="51" cpuset="0x00080000,0x0" complete_cpuset="0x00080000,0x0" online_cpuset="0x00080000,0x0" allowed_cpuset="0x00080000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00100000,0x0" complete_cpuset="0x00100000,0x0" online_cpuset="0x00100000,0x0" allowed_cpuset="0x00100000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00100000,0x0" complete_cpuset="0x00100000,0x0" online_cpuset="0x00100000,0x0" allowed_cpuset="0x00100000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00100000,0x0" complete_cpuset="0x00100000,0x0" online_cpuset="0x00100000,0x0" allowed_cpuset="0x00100000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="52" cpuset="0x00100000,0x0" complete_cpuset="0x00100000,0x0" online_cpuset="0x00100000,0x0" allowed_cpuset="0x00100000,0x0">
                <object type="PU" os_index="52" cpuset="0x00100000,0x0" complete_cpuset="0x00100000,0x0" online_cpuset="0x00100000,0x0" allowed_cpuset="0x00100000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00200000,0x0" complete_cpuset="0x00200000,0x0" online_cpuset="0x00200000,0x0" allowed_cpuset="0x00200000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00200000,0x0" complete_cpuset="0x00200000,0x0" online_cpuset="0x00200000,0x0" allowed_cpuset="0x00200000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00200000,0x0" complete_cpuset="0x00200000,0x0" online_cpuset="0x00200000,0x0" allowed_cpuset="0x00200000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="53" cpuset="0x00200000,0x0" complete_cpuset="0x00200000,0x0" online_cpuset="0x00200000,0x0" allowed_cpuset="0x00200000,0x0">
                <object type="PU" os_index="53" cpuset="0x00200000,0x0" complete_cpuset="0x00200000,0x0" online_cpuset="0x00200000,0x0" allowed_cpuset="0x00200000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00400000,0x0" complete_cpuset="0x00400000,0x0" online_cpuset="0x00400000,0x0" allowed_cpuset="0x00400000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00400000,0x0" complete_cpuset="0x00400000,0x0" online_cpuset="0x00400000,0x0" allowed_cpuset="0x00400000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00400000,0x0" complete_cpuset="0x00400000,0x0" online_cpuset="0x00400000,0x0" allowed_cpuset="0x00400000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="54" cpuset="0x00400000,0x0" complete_cpuset="0x00400000,0x0" online_cpuset="0x00400000,0x0" allowed_cpuset="0x00400000,0x0">
                <object type="PU" os_index="54" cpuset="0x00400000,0x0" complete_cpuset="0x00400000,0x0" online_cpuset="0x00400000,0x0" allowed_cpuset="0x00400000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x00800000,0x0" complete_cpuset="0x00800000,0x0" online_cpuset="0x00800000,0x0" allowed_cpuset="0x00800000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x00800000,0x0" complete_cpuset="0x00800000,0x0" online_cpuset="0x00800000,0x0" allowed_cpuset="0x00800000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x00800000,0x0" complete_cpuset="0x00800000,0x0" online_cpuset="0x00800000,0x0" allowed_cpuset="0x00800000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="55" cpuset="0x00800000,0x0" complete_cpuset="0x00800000,0x0" online_cpuset="0x00800000,0x0" allowed_cpuset="0x00800000,0x0">
                <object type="PU" os_index="55" cpuset="0x00800000,0x0" complete_cpuset="0x00800000,0x0" online_cpuset="0x00800000,0x0" allowed_cpuset="0x00800000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x01000000,0x0" complete_cpuset="0x01000000,0x0" online_cpuset="0x01000000,0x0" allowed_cpuset="0x01000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x01000000,0x0" complete_cpuset="0x01000000,0x0" online_cpuset="0x01000000,0x0" allowed_cpuset="0x01000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x01000000,0x0" complete_cpuset="0x01000000,0x0" online_cpuset="0x01000000,0x0" allowed_cpuset="0x01000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="56" cpuset="0x01000000,0x0" complete_cpuset="0x01000000,0x0" online_cpuset="0x01000000,0x0" allowed_cpuset="0x01000000,0x0">
                <object type="PU" os_index="56" cpuset="0x01000000,0x0" complete_cpuset="0x01000000,0x0" online_cpuset="0x01000000,0x0" allowed_cpuset="0x01000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x02000000,0x0" complete_cpuset="0x02000000,0x0" online_cpuset="0x02000000,0x0" allowed_cpuset="0x02000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x02000000,0x0" complete_cpuset="0x02000000,0x0" online_cpuset="0x02000000,0x0" allowed_cpuset="0x02000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x02000000,0x0" complete_cpuset="0x02000000,0x0" online_cpuset="0x02000000,0x0" allowed_cpuset="0x02000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="57" cpuset="0x02000000,0x0" complete_cpuset="0x02000000,0x0" online_cpuset="0x02000000,0x0" allowed_cpuset="0x02000000,0x0">
                <object type="PU" os_index="57" cpuset="0x02000000,0x0" complete_cpuset="0x02000000,0x0" online_cpuset="0x02000000,0x0" allowed_cpuset="0x02000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x04000000,0x0" complete_cpuset="0x04000000,0x0" online_cpuset="0x04000000,0x0" allowed_cpuset="0x04000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x04000000,0x0" complete_cpuset="0x04000000,0x0" online_cpuset="0x04000000,0x0" allowed_cpuset="0x04000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x04000000,0x0" complete_cpuset="0x04000000,0x0" online_cpuset="0x04000000,0x0" allowed_cpuset="0x04000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="58" cpuset="0x04000000,0x0" complete_cpuset="0x04000000,0x0" online_cpuset="0x04000000,0x0" allowed_cpuset="0x04000000,0x0">
                <object type="PU" os_index="58" cpuset="0x04000000,0x0" complete_cpuset="0x04000000,0x0" online_cpuset="0x04000000,0x0" allowed_cpuset="0x04000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x08000000,0x0" complete_cpuset="0x08000000,0x0" online_cpuset="0x08000000,0x0" allowed_cpuset="0x08000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x08000000,0x0" complete_cpuset="0x08000000,0x0" online_cpuset="0x08000000,0x0" allowed_cpuset="0x08000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x08000000,0x0" complete_cpuset="0x08000000,0x0" online_cpuset="0x08000000,0x0" allowed_cpuset="0x08000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="59" cpuset="0x08000000,0x0" complete_cpuset="0x08000000,0x0" online_cpuset="0x08000000,0x0" allowed_cpuset="0x08000000,0x0">
                <object type="PU" os_index="59" cpuset="0x08000000,0x0" complete_cpuset="0x08000000,0x0" online_cpuset="0x08000000,0x0" allowed_cpuset="0x08000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x10000000,0x0" complete_cpuset="0x10000000,0x0" online_cpuset="0x10000000,0x0" allowed_cpuset="0x10000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x10000000,0x0" complete_cpuset="0x10000000,0x0" online_cpuset="0x10000000,0x0" allowed_cpuset="0x10000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x10000000,0x0" complete_cpuset="0x10000000,0x0" online_cpuset="0x10000000,0x0" allowed_cpuset="0x10000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="60" cpuset="0x10000000,0x0" complete_cpuset="0x10000000,0x0" online_cpuset="0x10000000,0x0" allowed_cpuset="0x10000000,0x0">
                <object type="PU" os_index="60" cpuset="0x10000000,0x0" complete_cpuset="0x10000000,0x0" online_cpuset="0x10000000,0x0" allowed_cpuset="0x10000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x20000000,0x0" complete_cpuset="0x20000000,0x0" online_cpuset="0x20000000,0x0" allowed_cpuset="0x20000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x20000000,0x0" complete_cpuset="0x20000000,0x0" online_cpuset="0x20000000,0x0" allowed_cpuset="0x20000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x20000000,0x0" complete_cpuset="0x20000000,0x0" online_cpuset="0x20000000,0x0" allowed_cpuset="0x20000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="61" cpuset="0x20000000,0x0" complete_cpuset="0x20000000,0x0" online_cpuset="0x20000000,0x0" allowed_cpuset="0x20000000,0x0">
                <object type="PU" os_index="61" cpuset="0x20000000,0x0" complete_cpuset="0x20000000,0x0" online_cpuset="0x20000000,0x0" allowed_cpuset="0x20000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x40000000,0x0" complete_cpuset="0x40000000,0x0" online_cpuset="0x40000000,0x0" allowed_cpuset="0x40000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x40000000,0x0" complete_cpuset="0x40000000,0x0" online_cpuset="0x40000000,0x0" allowed_cpuset="0x40000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x40000000,0x0" complete_cpuset="0x40000000,0x0" online_cpuset="0x40000000,0x0" allowed_cpuset="0x40000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="62" cpuset="0x40000000,0x0" complete_cpuset="0x40000000,0x0" online_cpuset="0x40000000,0x0" allowed_cpuset="0x40000000,0x0">
                <object type="PU" os_index="62" cpuset="0x40000000,0x0" complete_cpuset="0x40000000,0x0" online_cpuset="0x40000000,0x0" allowed_cpuset="0x40000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
        <object type="Cache" cpuset="0x80000000,0x0" complete_cpuset="0x80000000,0x0" online_cpuset="0x80000000,0x0" allowed_cpuset="0x80000000,0x0" cache_size="1048576" depth="2" cache_linesize="64" cache_associativity="8" cache_type="0">
          <object type="Cache" cpuset="0x80000000,0x0" complete_cpuset="0x80000000,0x0" online_cpuset="0x80000000,0x0" allowed_cpuset="0x80000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="1">
            <object type="Cache" cpuset="0x80000000,0x0" complete_cpuset="0x80000000,0x0" online_cpuset="0x80000000,0x0" allowed_cpuset="0x80000000,0x0" cache_size="65536" depth="1" cache_linesize="64" cache_associativity="4" cache_type="2">
              <object type="Core" os_index="63" cpuset="0x80000000,0x0" complete_cpuset="0x80000000,0x0" online_cpuset="0x80000000,0x0" allowed_cpuset="0x80000000,0x0">
                <object type="PU" os_index="63" cpuset="0x80000000,0x0" complete_cpuset="0x80000000,0x0" online_cpuset="0x80000000,0x0" allowed_cpuset="0x80000000,0x0"/>
              </object>
            </object>
          </object>
        </object>
      </object>
    </object>
    <object type="Bridge" os_index="0" bridge_type="0-1" depth="0" bridge_pci="0000:[00-00]">
      <object type="PCIDev" os_index="64" pci_busid="0000:00:04.0" pci_type="0108 [1d0f:8061] [1d0f:0000] 00" pci_link_speed="0.000000">
        <info name="PCISlot" value="4"/>
      </object>
      <object type="PCIDev" os_index="80" pci_busid="0000:00:05.0" pci_type="0200 [1d0f:ec20] [1d0f:ec20] 00" pci_link_speed="0.000000">
        <info name="PCISlot" value="5"/>
        <object type="OSDev" name="eth0" osdev_type="2">
          <info name="Address" value="0a:01:64:7e:90:b7"/>
        </object>
      </object>
      <object type="PCIDev" os_index="96" pci_busid="0000:00:06.0" pci_type="0200 [1d0f:efa1] [1d0f:efa1] 00" pci_link_speed="0.000000">
        <info name="PCISlot" value="6"/>
        <object type="OSDev" name="rdmap0s6" osdev_type="3">
          <info name="NodeGUID" value="0000:0000:0000:0000"/>
          <info name="SysImageGUID" value="0000:0000:0000:0000"/>
          <info name="Port1State" value="4"/>
          <info name="Port1LID" value="0x0"/>
          <info name="Port1LMC" value="1"/>
          <info name="Port1GID0" value="fe80:0000:0000:0000:0801:64ff:fe7e:90b7"/>
        </object>
      </object>
    </object>
  </object>
</topology>

rhc54 commented 1 year ago

It's the same topology I had before, but I tested against it anyway and it still works for me:

$ mpirun -n 2 --map-by slot --bind-to socket --prtemca hwloc_use_topo_file Amazon2.xml --prtemca ras_simulator_num_nodes 1 --display map-devel hostname

=================================   JOB MAP   =================================
Data for JOB prterun-Ralphs-iMac-2-70007@1 offset 0 Total slots allocated 64
Mapper requested: NULL  Last mapper: round_robin  Mapping policy: BYSLOT:NOOVERSUBSCRIBE  Ranking policy: SLOT
Binding policy: PACKAGE  Cpu set: N/A  PPR: N/A  Cpus-per-rank: N/A  Cpu Type: HWT
Num new daemons: 0  New daemon starting vpid INVALID
Num nodes: 1

Data for node: nodeA0   State: 3    Flags: MAPPED:SLOTS_GIVEN
        Daemon: [prterun-Ralphs-iMac-2-70007@0,1]   Daemon launched: False
            Num slots: 64   Slots in use: 2 Oversubscribed: FALSE
            Num slots allocated: 64 Max slots: 64   Num procs: 2
        Data for proc: [prterun-Ralphs-iMac-2-70007@1,0]
                Pid: 0  Local rank: 0   Node rank: 0    App rank: 0
                State: INITIALIZED  App_context: 0
            Binding: package[0][hwt:0-63]
        Data for proc: [prterun-Ralphs-iMac-2-70007@1,1]
                Pid: 0  Local rank: 1   Node rank: 1    App rank: 1
                State: INITIALIZED  App_context: 0
            Binding: package[0][hwt:0-63]

Warning: This map has been generated with the DONOTLAUNCH option;
    The compute node architecture has not been probed, and the displayed
    map reflects the HEADNODE ARCHITECTURE. On systems with a different
    architecture between headnode and compute nodes, the map can be
    displayed using `prte --display map /bin/true`, which will launch
    enough of the DVM to probe the compute node architecture.

=============================================================

darrylabbate commented 1 year ago

This is the output on my end, with Amazon2.xml being the file I pasted above

[ec2-user@c7g-st-n64-1 ~]$ mpirun -n 2 --map-by slot --bind-to socket --prtemca hwloc_use_topo_file Amazon2.xml --prtemca ras_simulator_num_nodes 1 --display map-devel hostname
--------------------------------------------------------------------------
While computing bindings, we found no available cpus on
the following node:

  Node:  nodeA0

Please check your allocation.
--------------------------------------------------------------------------

I did a fresh recursive clone, submodule pointer update and build to confirm I wasn't using an older mpirun

rhc54 commented 1 year ago

Sorry, but I cannot replicate with HEAD of the PMix/PRRTE master branches.

I did a fresh recursive clone, submodule pointer update and build to confirm I wasn't using an older mpirun

That process will result in an out-of-date build of PRRTE, I'm afraid. The OMPI main branch hasn't committed the updated submodule pointers, so you are still pointing to old code. You have to go to the 3rd-party subdirectories and explicitly checkout the master branch and then "pull" to get the latest code.

darrylabbate commented 1 year ago

Sorry, but I cannot replicate with HEAD of the PMix/PRRTE master branches.

I did a fresh recursive clone, submodule pointer update and build to confirm I wasn't using an older mpirun

That process will result in an out-of-date build of PRRTE, I'm afraid. The OMPI main branch hasn't committed the updated submodule pointers, so you are still pointing to old code. You have to go to the 3rd-party subdirectories and explicitly checkout the master branch and then "pull" to get the latest code.

Sorry if I wasn't clear; this is what I did.

Anyway, I'm satisfied with the original issue being intended behavior, so I'll close this issue. Thanks for your help! Let me know if you'd like any more information from me if that helps with debugging any similar issues at the moment.

open-mpi / ompi

Question re: behavior of `--bind-to socket` when number of processes exceeds available sockets #11541