search

syndtr / goleveldb

LevelDB key/value database in Go.
BSD 2-Clause "Simplified" License
6.15k stars 968 forks source link

fatal error: runtime: cannot map pages in arena address space #117

Closed ryanstout closed 6 years ago

ryanstout commented 9 years ago

I'm cross compiling a project to run on an ARM7 system, which has the page size modified to 64k (for various reasons). I'm seeing the following error. Any help would be greatly appreciated. Thanks!

fatal error: runtime: cannot map pages in arena address space

runtime stack:
runtime.SysMap(0x601000, 0x1000, 0x1, 0x4d23e0)
    /goroot/src/runtime/mem_linux.c:161 +0x14c
runtime.MHeap_MapSpans(0x4d5150)
    /goroot/src/runtime/mheap.c:88 +0xbc
runtime.MHeap_SysAlloc(0x4d5150, 0x400000, 0x0)
    /goroot/src/runtime/malloc.c:287 +0x170
runtime.MHeap_Alloc(0x4d5150, 0x200, 0x0, 0x30700101, 0x2b1a88)
    /goroot/src/runtime/mheap.c:240 +0x50

goroutine 1 [running]:
runtime.switchtoM()
    /goroot/src/runtime/asm_arm.s:193 +0x4 fp=0x1076ba58 sp=0x1076ba54
runtime.mallocgc(0x400000, 0x2c26c8, 0x1, 0x400000)
    /goroot/src/runtime/malloc.go:194 +0x9f8 fp=0x1076bab0 sp=0x1076ba58
runtime.newarray(0x2c26c8, 0x400000, 0x30704000)
    /goroot/src/runtime/malloc.go:365 +0xec fp=0x1076bacc sp=0x1076bab0
runtime.makeslice(0x2bc420, 0x0, 0x0, 0x400000, 0x0, 0x0, 0x0, 0x0)
    /goroot/src/runtime/slice.go:32 +0x1fc fp=0x1076baf0 sp=0x1076bacc
github.com/syndtr/goleveldb/leveldb/memdb.New(0x30715ec8, 0x1070a598, 0x400000, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/memdb/memdb.go:466 +0xc0 fp=0x1076bb40 sp=0x1076baf0
github.com/syndtr/goleveldb/leveldb.(*DB).newMem(0x10740540, 0x0, 0x0, 0x0, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_state.go:125 +0x568 fp=0x1076bbb4 sp=0x1076bb40
github.com/syndtr/goleveldb/leveldb.(*DB).recoverJournal(0x10740540, 0x0, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:529 +0x1234 fp=0x1076bddc sp=0x1076bbb4
github.com/syndtr/goleveldb/leveldb.openDB(0x1073e4d0, 0x0, 0x0, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:154 +0xa08 fp=0x1076bebc sp=0x1076bddc
github.com/syndtr/goleveldb/leveldb.Open(0x30715ce0, 0x1072ca80, 0x0, 0x0, 0x0, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:194 +0x1b8 fp=0x1076bef0 sp=0x1076bebc
github.com/syndtr/goleveldb/leveldb.OpenFile(0x375db8, 0x17, 0x0, 0x0, 0x0, 0x0)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:216 +0x90 fp=0x1076bf14 sp=0x1076bef0
_/gopath/infinitydrive/src/file_reference.Setup(0x3da988)
    /gopath/infinitydrive/src/file_reference/file_reference.go:40 +0x138 fp=0x1076bf6c sp=0x1076bf14
main.main()
    /gopath/infinitydrive/src/drive.go:38 +0x170 fp=0x1076bfc0 sp=0x1076bf6c
runtime.main()
    /goroot/src/runtime/proc.go:63 +0x108 fp=0x1076bfe4 sp=0x1076bfc0
runtime.goexit()
    /goroot/src/runtime/asm_arm.s:1322 +0x4 fp=0x1076bfe4 sp=0x1076bfe4

goroutine 5 [select]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0x10782160)
    /golang/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:205 +0x1d4
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
    /golang/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:236 +0x23c

goroutine 7 [runnable]:
github.com/syndtr/goleveldb/leveldb.(*DB).compactionError(0x10740380)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:64
created by github.com/syndtr/goleveldb/leveldb.openDB
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:139 +0x794

goroutine 8 [runnable]:
github.com/syndtr/goleveldb/leveldb.(*DB).mpoolDrain(0x10740380)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_state.go:79
created by github.com/syndtr/goleveldb/leveldb.openDB
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:140 +0x7bc

goroutine 9 [runnable]:
github.com/syndtr/goleveldb/leveldb.(*DB).tCompaction(0x10740380)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:731
created by github.com/syndtr/goleveldb/leveldb.openDB
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:154 +0x9a4

goroutine 10 [runnable]:
github.com/syndtr/goleveldb/leveldb.(*DB).mCompaction(0x10740380)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_compaction.go:699
created by github.com/syndtr/goleveldb/leveldb.openDB
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:154 +0x9cc

goroutine 11 [runnable]:
github.com/syndtr/goleveldb/leveldb.(*DB).jWriter(0x10740380)
    /golang/src/github.com/syndtr/goleveldb/leveldb/db_write.go:34
created by github.com/syndtr/goleveldb/leveldb.openDB
    /golang/src/github.com/syndtr/goleveldb/leveldb/db.go:154 +0x9f4

goroutine 12 [runnable]:
github.com/syndtr/goleveldb/leveldb/util.(*BufferPool).drain(0x10782210)
    /golang/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:202
created by github.com/syndtr/goleveldb/leveldb/util.NewBufferPool
    /golang/src/github.com/syndtr/goleveldb/leveldb/util/buffer_pool.go:236 +0x23c
bcampbell commented 9 years ago

I've bumped into the same "cannot map pages in arena address space" error message running under 32bit windows. I think the overall memory limit for golang on win32 is 2GB, but I didn't think I'd be hitting anything near that... more like 100-200MB. I'm wondering if there's some stuff being kept around in memory longer than it needs to be...

Sorry, not really anything useful here, but just wanted to add another voice :- )

A snippet from my panic:

goroutine 1 [running, locked to thread]:
runtime.systemstack_switch()
        c:/go/src/runtime/asm_386.s:222 fp=0x136cb204 sp=0x136cb200
runtime.mallocgc(0x74181b, 0x884b40, 0x1, 0x121372ec)
        c:/go/src/runtime/malloc.go:635 +0x77f fp=0x136cb26c sp=0x136cb204
runtime.newarray(0x884b40, 0x74181b, 0x28d2d140)
        c:/go/src/runtime/malloc.go:777 +0xbc fp=0x136cb290 sp=0x136cb26c
runtime.makeslice(0x879440, 0x736722, 0x0, 0x74181b, 0x0, 0x0, 0x0, 0x0)
        c:/go/src/runtime/slice.go:32 +0x180 fp=0x136cb2d0 sp=0x136cb290
github.com/syndtr/goleveldb/leveldb.(*Batch).grow(0x1a220360, 0x12)
        c:/Users/ben/mygo/src/github.com/syndtr/goleveldb/leveldb/batch.go:64 +0x150 fp=0x136cb320 sp=0x136cb2d0
github.com/syndtr/goleveldb/leveldb.(*Batch).appendRec(0x1a220360, 0x0, 0x27748000, 0xc, 0x655e, 0x0, 0x0, 0x0)
        c:/Users/ben/mygo/src/github.com/syndtr/goleveldb/leveldb/batch.go:75 +0x48 fp=0x136cb37c sp=0x136cb320
github.com/syndtr/goleveldb/leveldb.(*Batch).Delete(0x1a220360, 0x27748000, 0xc, 0x655e)
        c:/Users/ben/mygo/src/github.com/syndtr/goleveldb/leveldb/batch.go:103 +0x4f fp=0x136cb3a0 sp=0x136cb37c
calmh commented 9 years ago

The heap size is limited to ~500 MB on 32 bit Windows, and this is the limit you're hitting.