[Tech Request]: Implement Efficient Session-Level Memory Control

[sukki37]

Is there an existing issue for the same feature request?

• [X] I have checked the existing issues.

Is your feature request related to a problem?

Yes. Many oom issues.

Describe the feature you'd like

For optimal performance and stability, it's imperative to implement a session-level memory control mechanism. Here's the main purpose:

1. Memory Usage Estimation: Before executing any operation, estimate the amount of memory it will consume. This will help in ensuring that we allocate the necessary resources right at the outset.
2. Session-Level Memory Limit: Limit the maximum memory each session can use based on available resources and session context.
3. Efficient Resource Utilization: Even with limited memory availability, operations should continue to execute. A degradation in performance can be acceptable, but the system should avoid resource exhaustion.
4. Error Handling with Hard Limits: In scenarios where memory consumption approaches a hard limit(hashtable etc.), the system should gracefully return an error rather than proceeding, which could risk an oom crash.

Describe implementation you've considered

No response

Documentation, Adoption, Use Case, Migration Strategy

No response

Additional information

depend on #11511

[daviszhen]

1,spill 不做的话，内存块到了，主动fail。

[daviszhen]

2，清理batch里面的Agg,hashtable (p-1)

[daviszhen]

3, pk doris

[daviszhen]

operator batch里面64mb 或 8192行，输出。

不同于 spill

[florashi181]

相关的subtask： https://github.com/matrixorigin/matrixone/issues/13524