golang / go

The Go programming language
https://go.dev
BSD 3-Clause "New" or "Revised" License
123.98k stars 17.67k forks source link

Code Leaks Memory on FreeBSD, but not on Linux #64806

Closed johnpharmetika closed 10 months ago

johnpharmetika commented 10 months ago

Go version

go version go1.20.8 freebsd/amd64

What operating system and processor architecture are you using (go env)?

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="freebsd"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="freebsd"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go120"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go120/pkg/tool/freebsd_amd64"
GOVCS=""
GOVERSION="go1.20.8"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="cc"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/root/service_utility_go/go.mod"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -Wl,--no-gc-sections -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build3737356626=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Using GoFiber (which utilizes fasthttp) and Excelize to make a service that converts json to xlsx. Code is developed on a Linux machine but ran on FreeBSD ultimately. When running this code on FreeBSD, memory allocated is never GCed, causing a leak. This issue is not present when compiled on Linux (with the same code) Tested code as Linux in console, Linux as service, FreeBSD in console, FreeBSD as service. Compiled/ran in Linux the code did not leak memory, FreeBSD it always did.

Handler function

type excel_json struct {
    Data [][]interface{} `json:"data"`
}

func handle_excel_v1(fiber_context *fiber.Ctx) error {
    var json_values_root excel_json
    err := json.Unmarshal(fiber_context.Body(), &json_values_root)
    if err != nil {
        err = fmt.Errorf("parse data error: %d", err)
        fmt.Println(err)
        return nil
    }

    //Buffer is set for convert_json_to_excel to write bytes to
    var excel_data bytes.Buffer

    err = convert_json_to_excel(json_values_root.Data, &excel_data)
    if err != nil {
        err = fmt.Errorf("conversion error: %d", err)
        fmt.Println(err)
        return nil
    }

    /*if json_values_root.Exists("file_path") {
        file, _ := os.Create(fmt.Sprintf("%s.%s", string(json_values_root.GetStringBytes("file_path")), "xlsx"))
        defer file.Close()

        file.Write(excel_data.Bytes())
        fiber_context.JSON(fiber.Map{
            "success":   true,
            "file_path": string(json_values_root.GetStringBytes("file_path")),
        })
    } else {*/
    fiber_context.Response().Header.Set("Content-Disposition", "attachment; filename=response.xlsx")
    fiber_context.Write(excel_data.Bytes())
    //}
    return nil
}

Excel conversion function

package main

import (
    "bytes"
    "fmt"

    "github.com/xuri/excelize/v2"
)

func convert_json_to_excel(json_data [][]interface{}, buffer_pointer *bytes.Buffer) error {

    var err error

    // Create a working temporary Excel "file"
    excel_file := excelize.NewFile()
    defer excel_file.Close()

    sheet_index, _ := excel_file.NewSheet("Sheet1")

    excel_file.SetActiveSheet(sheet_index)

    //Embolden the first row
    header_style, _ := excel_file.NewStyle(&excelize.Style{
        Font: &excelize.Font{
            Bold: true,
        },
    })
    header_row_opts := excelize.RowOpts{StyleID: header_style}

    stream_writer, _ := excel_file.NewStreamWriter("Sheet1")

    // Widen all used columns based on headers
    header_row := json_data[0]
    stream_writer.SetColWidth(1, len(header_row), 30)

    // Iterate through data and write to file
    for row := 0; row < len(json_data); row++ {

        active_row := json_data[row]
        //fmt.Printf("%s\n", active_row)

        first_cell := fmt.Sprintf("A%d", (row + 1))

        if row == 0 {
            stream_writer.SetRow(first_cell, active_row, header_row_opts)
        } else {
            stream_writer.SetRow(first_cell, active_row)
        }
    }

    stream_writer.Flush()

    _, err = excel_file.WriteTo(buffer_pointer)
    if err != nil {
        err = fmt.Errorf("write fail: %d", err)
        fmt.Println(err)
        return err
    }

    return nil
}

What did you expect to see?

In use space for Linux 10 min after multiple requests processed image

What did you see instead?

In use space for FreeBSD 10 min after multiple requests processed image

seankhliao commented 10 months ago

I think you should try reporting this to the library first.