search

jackc / pgx

PostgreSQL driver and toolkit for Go
MIT License
9.85k stars 802 forks source link

Memory leak in scanPlanString #2039

Open akshatraika-moment opened 4 weeks ago

akshatraika-moment commented 4 weeks ago

Describe the bug There is a memory leak while using pgx Select Scans. There is a consistent increase of memory usage by the package. Here is a pprof dump -

reflect.New:3256 github.com/jackc/pgx/v5/pgtype.scanPlanString - 4.65 GB
github.com/jackc/pgx/v5/pgtype.scanPlanString - 2.64 GB
reflect.New:3256 github.com/georgysavva/scany - 834.60 MB
*github.com/georgysavva/scany/v2/dbscan.(RowS) - 821.96 MB
reflect.Value.extendSlice:2848 reflect.Append - 816.12 MB
reflect.New:3256 github.com/georgysavva/scany - 794.36 MB
github.com.jackc/pgx/v5/pgtype.TextCodec.Deco - 568.66 MB
github.com.jackc/pgx/v5/pgtype.scanPlanString - 360.29 MB
github.com.jackc/pgx/v5/pgtype.scanPlanString - 320.38 MB

These are the top memory users in my application. The query I make is quite simple and is just made once every 30 minutes. The package does seem to be leaking the memory on every query though.

To Reproduce Steps to reproduce the behavior: You can probably just make a large query again and again and pprof the process to see the memory usage.

Expected behavior Memory should only be used ephemerally by the package.

Actual behavior Gigs of memory leak Version

Additional context Add any other context about the problem here.

jackc commented 4 weeks ago

Are you sure this is showing current memory usage? I think it is actually showing you total allocations. As in, this isn't a leak, it is a lot of allocations which are being garbage collected.