search

nlpodyssey / verbaflow

Neural Language Model for Go
BSD 2-Clause "Simplified" License
58 stars 5 forks source link

Memory climbs to >12GB while converting #6

Open mooijtech opened 1 year ago

mooijtech commented 1 year ago

Memory climbs to >12GB using: ./verbaflow -model-dir models/nlpodyssey/RWKV-4-Pile-1B5-Instruct convert before being killed by OS (out of memory). Is this normal?

Thanks for your contributions :)

matteo-grella commented 1 year ago

What OS?

mooijtech commented 1 year ago

(Arch) Linux (x86_64)

matteo-grella commented 1 year ago

Thanks, let me investigate and I'll back to you beginning next week. Spoiler: we will move soon to https://github.com/nlpodyssey/safetensors which is much faster and requires less memory than the current pickle converter

mooijtech commented 1 year ago

Possibly related memory stack-trace:

heap profile: 2: 6060038144 [4: 6060079216] @ heap/1048576
1: 6060032000 [1: 6060032000] @ 0x9f7ec7 0x9f4411 0x9f3fc5 0x9ec667 0x9eab57 0x9f3ab8 0x9f3565 0x9f34e5 0xb1212d 0xb15424 0xb15999 0xb0ff4f 0xb110aa 0xad2bab 0xad2e0d 0xacfa16 0xb0faf2 0xb0facb 0x43b227 0x46e461
#   0x9f7ec6    github.com/nlpodyssey/gopickle/pytorch.(*BFloat16Storage).SetFromFileWithSize+0x46  /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/storage.go:395
#   0x9f4410    github.com/nlpodyssey/gopickle/pytorch.loadTensor+0x190                 /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:127
#   0x9f3fc4    github.com/nlpodyssey/gopickle/pytorch.loadZipFile.func1+0x2a4              /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:99
#   0x9ec666    github.com/nlpodyssey/gopickle/pickle.loadBinPersId+0x46                /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pickle/pickle.go:428
#   0x9eab56    github.com/nlpodyssey/gopickle/pickle.(*Unpickler).Load+0xf6                /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pickle/pickle.go:102
#   0x9f3ab7    github.com/nlpodyssey/gopickle/pytorch.loadZipFile+0x4f7                /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:107
#   0x9f3564    github.com/nlpodyssey/gopickle/pytorch.LoadWithUnpickler+0x44               /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:40
#   0x9f34e4    github.com/nlpodyssey/gopickle/pytorch.Load+0x24                    /home/bot/go/pkg/mod/github.com/nlpodyssey/gopickle@v0.2.0/pytorch/pytorch.go:31
#   0xb1212c    github.com/nlpodyssey/verbaflow/rwkvlm.(*converter[...]).loadTorchModelParams+0x2c  /home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:416
#   0xb15423    github.com/nlpodyssey/verbaflow/rwkvlm.(*converter[...]).run+0x1a3          /home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:117
#   0xb15998    github.com/nlpodyssey/verbaflow/rwkvlm.ConvertPickledModelToRWKVLM[...]+0x518       /home/bot/Documents/2023/verbaflow/rwkvlm/converter.go:78
#   0xb0ff4e    main.convert+0x12e                                  /home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:139
#   0xb110a9    main.main.func3+0x29                                    /home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:65
#   0xad2baa    github.com/urfave/cli/v2.(*Command).Run+0x9ea                       /home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/command.go:273
#   0xad2e0c    github.com/urfave/cli/v2.(*Command).Run+0xc4c                       /home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/command.go:266
#   0xacfa15    github.com/urfave/cli/v2.(*App).RunContext+0x615                    /home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/app.go:332
#   0xb0faf1    github.com/urfave/cli/v2.(*App).Run+0x5d1                       /home/bot/go/pkg/mod/github.com/urfave/cli/v2@v2.24.3/app.go:309
#   0xb0faca    main.main+0x5aa                                     /home/bot/Documents/2023/verbaflow/cmd/verbaflow/main.go:99
#   0x43b226    runtime.main+0x206                                  /usr/lib/go/src/runtime/proc.go:250