Memory allocation failure at CREATE EXTENSION.

[pashkinelfe]

Hi! I found the following error at trying create extension on one of the customers machine: not enough shared memory for data structure "hnsw LWLock ids" (4 bytes requested) called from the code block

/*
 * Assign a tranche ID for our LWLocks. This only needs to be done by one
 * backend, as the tranche ID is remembered in shared memory.
 *
 * This shared memory area is very small, so we just allocate it from the
 * "slop" that PostgreSQL reserves for small allocations like this. If
 * this grows bigger, we should use a shmem_request_hook and
 * RequestAddinShmemSpace() to pre-reserve space for this.
 */
void
HnswInitLockTranche(void)
{
    int        *tranche_ids;
    bool        found;

    LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
    tranche_ids = ShmemInitStruct("hnsw LWLock ids",
                                  sizeof(int) * 1,
                                  &found);
    if (!found)
        tranche_ids[0] = LWLockNewTrancheId();
    hnsw_lock_tranche_id = tranche_ids[0];
    LWLockRelease(AddinShmemInitLock);

    /* Per-backend registration of the tranche ID */
    LWLockRegisterTranche(hnsw_lock_tranche_id, "HnswBuild");
}

It's a rare error that I met only once. Could we not rely that the 'slop' has enough free space to do allocation? As I think PG user has little power to check or free this memory.

[ankane]

@pashkinelfe It'd be good to understand how this happens, since Postgres allocates an extra 100K bytes for small allocations.

We could request the shared memory up front when the extension is part of shared_preload_libraries (without making it required), but I'm not sure if it's worth the complexity.

[akorotkov]

@pashkinelfe, could we get the full config when the error happens? Probably it's related to the combination of multiple extensions.

[pashkinelfe]

@akorotkov Non-default GUCs:

checkpoint_completion_target = 0.900
default_statistics_target = 100
effective_cache_size = '6GB'
effective_io_concurrency = 200
maintenance_work_mem = '512MB'
max_connections = 160
max_parallel_maintenance_workers = 1
max_parallel_workers = 2
max_parallel_workers_per_gather = 1
max_wal_size = '4GB'
max_worker_processes = 6
min_wal_size = '1GB'
max_wal_senders = 8
max_replication_slots = 8
random_page_cost = 1.100
shared_buffers = '2GB'
wal_buffers = '16MB'
work_mem = '12MB'

"Show all" contents: 1000.txt

[akorotkov]

Can't reproduce this yet. Could you also share exact PostgreSQL and pgvector versions?

[pashkinelfe]

PG 15.1 + pgvector 0.6.0 I also can't reproduce it.