Other "compass"-like libraries

tdd11235813 commented 7 years ago

There are other libraries around which try to provide a generic hardware discovering. What is done there and what can be re-used in compass? The ideal case would be to gather the authors and merge the work into compass. I will try to list them in this issue...

tdd11235813 commented 7 years ago

DASH

DASH DASH, the C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for HPC and Data-Driven Science http://www.dash-project.org/ Readme

Choosing a DASH runtime (DART)
DASH provides the following variants:
    MPI: the Message Passing Interface
    CUDA: nNvidia’s Compute Unified Device Architecture (contributor distribution only)
    SHMEM: Symmetric Hierarchical Memory access (contributor distribution only)

DASH already implements a hardware information API with hierarchical levels (nested "domains").

illustrating example on domains
domain: can be NUMA host domain, MIC domain, unit of MIC cores, ...
- scopes: GLOBAL > GROUP > NETWORK > NODE > MODULE > NUMA > UNIT > PACKAGE > UNCORE > CORE > CPU
properties: cache_line_sizes, cache_shared, cache_sizes, cpu_id, max_cpu_mhz, max_threads, min_cpu_mhz, min_threads, num_cores, num_numa, num_sockets, numa_id
license: looks like 3-clause BSD

Backend

utilizes likwid.h, papi.h, numa.h, hwloc.h
- see code here

Conclusion

covers performance-relevant properties like caches and affinity => should be part of compass
concept of domain and scope provides hierarchical links => should be part of compass as well
likwid, papi, numa and hwloc required to gather hardware information => probably would be the same in compass
cannot find GPU properties at the moment
it seems DASH/DART provides a good code base for compass

psteinb commented 7 years ago

that is an effort highly appreciated! thanks a mllion.

tdd11235813 commented 7 years ago

talked with a fellow from our institute, he also wants a hardware info tool, but focus is quite I/O low-level. nevertheless, there are common features and we should talk together, maybe at a GCOE meeting. but first, we should gather more information to find must-haves, nice-to-have and must-not-haves :)

tdd11235813 commented 7 years ago

hwloc

hwloc (Open-MPI)

source

... provides a portable abstraction (across OS, versions, architectures, ...) of the hierarchical topology of modern architectures,

information on: NUMA, sockets, caches, processor packages & cores & units, affinities, I/O devices, hierarchy, accelerators
C-API, highly portable
License: 3-clause BSD

Supported OS

 hwloc supports the following operating systems:
    Linux (including old kernels not having sysfs topology information, with knowledge of cgroups, offline CPUs, ScaleMP vSMP, and NumaScale NumaConnect) on all supported hardware, including Intel Xeon Phi.
    Solaris, AIX, HP-UX and OSF/1 (a.k.a., Tru64)
    NetBSD, FreeBSD and kFreeBSD/GNU
    Darwin / OS X
    Microsoft Windows (either using MinGW or a native Visual Studio solution)
    IBM BlueGene/Q Compute Node Kernel (CNK)

Conclusion

most likely we have to rely on it and most parts of compass just might C++-wrap hwloc

psteinb commented 7 years ago

yes and no, is hwloc working on Windows?

tdd11235813 commented 7 years ago

yes it does, see updated hwloc post. We still have to check features which are not covered by hwloc et. al.

tdd11235813 commented 7 years ago

Boost::Predef

Boost::Predef contains system information which can be retrieved by the compiler.

provides information about:

architecture, compiler, language, OS, SIMD type, endianess
probably not important: [C] library, platform (mingw, windows [desktop, phone, runtime, store])

First released in Boost 1.55.0.

Conclusion

could replace our compile-time stuff, which would introduce dependency of Boost >=1.55.0

psteinb commented 6 years ago

and of course we should not forget cpu_features

psteinb / compass