search

oneapi-src / unified-runtime

https://oneapi-src.github.io/unified-runtime/
Other
33 stars 113 forks source link

Generalize ur_usm_pool_limits_desc_t #369

Open igchor opened 1 year ago

igchor commented 1 year ago

Currently, ur_usm_pool_limits_desc_t exposes parameters similar to the ones exposed by SYCL_PI_LEVEL_ZERO_USM_ALLOCATOR.

There are a few problems with existing params:

  1. capacity is tricky to use: it describes a number of allocations and does not take into account allocation size. Also, UR does not expose any statistics that could be used to tweak the capacity. Moreover, it might not be applicable to all heap managers.

    I propose removing this parameter.

  2. maxPoolSize might be ambiguous. Unified Runtime can create multiple 'sub-pools' - it can decide to create pool per each device or to create a pool that spans multiple devices in certain cases. Having the same pool size limit for both of those cases will not be optimal.

    We could replace the maxPoolSize with a decay parameter that, instead of specifying limits in terms of space, defines it in terms of time - for how long the pool is allowed to keep memory pages around, similar to: https://jemalloc.net/jemalloc.3.html#arenas.dirty_decay_ms

Related discussion: https://github.com/oneapi-src/unified-runtime/pull/324

RaviNarayanaswamy commented 1 year ago
igchor commented 1 year ago
  • Capacity allows user to allocate memory suited for his program without going to level0 each time memory is requested. This is used in spec
  • I am not sure if maxPoolSize is useful, I assume you have maxAllocSize which says anything bigger than this should not be pooled.

The problem with capacity is that, I believe, it does not take allocation size into account. Distribution of allocation sizes is usually not flat.

In OpenMP there is both AllocMax (aka maxAllocSize) and poolSize (aka maxPoolSize). We definitely want to keep AllocMax but as you said, poolSize/maxPoolSize might not be useful.

Ref: https://www.intel.com/content/www/us/en/docs/dpcpp-cpp-compiler/developer-guide-reference/2023-0/supported-environment-variables.html