search

gorse-io / gorse

Gorse open source recommender system engine
https://gorse.io
Apache License 2.0
8.61k stars 785 forks source link

multipart: NextPart: bufio: buffer full #435

Open ujwachuku opened 2 years ago

ujwachuku commented 2 years ago

Gorse version Gorse v0.4.0

Describe the bug When you try to import a large CSV file containing items via the dashboard, in my case, the file was 39MB, it generates a "multipart: NextPart: bufio: buffer full" error.

To Reproduce Setup Gorse via docker and try to import items via the web UI using a CSV file (my file is 39MB with over 80K rows).

Expected behavior I expected that the items would be successfully imported and a message with the number of affected rows would be returned.

Additional context My setup runs is: OS - Ubuntu 18.04.4 LTS CPUs - 8 Total RAM - 31.28GB Docker version 20.10.14, build a224086 Server: Containers: 5 Running: 5 Paused: 0 Stopped: 0 Images: 8 Server Version: 20.10.14 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false

zhenghaoz commented 2 years ago

Could you provide the log from the master node? There is more detailed information. Thanks

ujwachuku commented 2 years ago

"level":"error","ts":1650478561.2738142,"caller":"server/rest.go:1622","msg":"bad request","error":"multipart: NextPart: bufio: buffer full","stacktrace":"github.com/zhenghaoz/gorse/server.BadRequest\n\t/go/http/server.go:1966"}

zhenghaoz commented 2 years ago

Sorry, I can't reproduce it.

ujwachuku commented 2 years ago

Could it be related to the resources allocated to docker? Perhaps they are inadequate?

zhenghaoz commented 2 years ago

Could it be related to the resources allocated to docker? Perhaps they are inadequate?

maybe. have you restricted the memory size of containers?

ujwachuku commented 2 years ago

I didn't tweak the docker config. It's running with the default config.