gogf / gf

GoFrame is a modular, powerful, high-performance and enterprise-class application development framework of Golang.
https://goframe.org
MIT License
11.8k stars 1.61k forks source link

use orm ALL() search 10w data,but server restart #3642

Closed MrGlp closed 2 months ago

MrGlp commented 5 months ago

Go version

1.20

GoFrame version

2.1

Can this bug be reproduced with the latest release?

Option Yes

What did you do?

g.Model(s.TableName(staticTime)).Where("time = ?", staticTime).
        OrderDesc("num").
        Limit(hotNum).All()
if mRes.IsEmpty() {
    return res, err
}
err = mRes.Structs(&res)        

This query returns 10W data, and the program crashes directly

env: alibaba yun k8s

The memory monitoring kanban board doesn't see the change, it's a straight line

What did you see happen?

g.Model(s.TableName(staticTime)).Where("time = ?", staticTime).
        OrderDesc("num").
        Limit(hotNum).All()
if mRes.IsEmpty() {
    return res, err
}
err = mRes.Structs(&res)        

This query returns 10W data, and the program crashes directly

env: alibaba yun k8s

The memory monitoring kanban board doesn't see the change, it's a straight line

What did you expect to see?

i want run success

MrGlp commented 5 months ago

i need you help

gqcn commented 2 months ago

@MrGlp It was OOM killed, right? What can we do for you?

Or you can using Page or Limit when query large amount of data?

MrGlp commented 2 months ago

@MrGlp It was OOM killed, right? What can we do for you?它被 OOM 杀死了,对吗?我们能为您做什么?

Or you can using Page or Limit when query large amount of data?或者当查询大量数据时可以使用PageLimit吗?

Okay, I have changed it to paging query

But is there a way to intercept the OOM caused by querying a large amount of data by default? This behavior causes the container to restart, which is often confusing.

  1. For example, returning an error
  2. For example, Recover
  3. Or directly intercepting data that exceeds the number of queries
gqcn commented 2 months ago

@MrGlp It was OOM killed, right? What can we do for you?它被 OOM 杀死了,对吗?我们能为您做什么? Or you can using Page or Limit when query large amount of data?或者当查询大量数据时可以使用PageLimit吗?

Okay, I have changed it to paging query

But is there a way to intercept the OOM caused by querying a large amount of data by default? This behavior causes the container to restart, which is often confusing.

1. For example, returning an error

2. For example, Recover

3. Or directly intercepting data that exceeds the number of queries

Process is killed by OOM killer from system kernel if its memory reaches the limits, which cannot be intervened by the process logic.