Change max_executor_or_driver_memory to 16. 64 was a very high limit

[tcassaert]

Is this 16GB a combination of overhead and "regular" memory?

[EmileSonneveld]

yes, executor_memory + executor_memory_overhead: https://github.com/Open-EO/openeo-geopyspark-driver/blob/master/openeogeotrellis/backend.py#L1858 (same for driver memory)

[EmileSonneveld]

@tcassaert It goes through tests. Is the limit fine, or too low? I often specified 20Gb driver-memory in the past, so I got the feeling there is space for more

[jdries]

Pay attention that things depend on the environment. Terrascope has higher limits, cdse limit is 15GB....

---

From: Emile @.> Sent: Monday, April 22, 2024 4:23 PM To: Open-EO/openeo-geopyspark-driver @.> Cc: Subscribed @.***> Subject: Re: [Open-EO/openeo-geopyspark-driver] Change max_executor_or_driver_memory to 16. 64 was a very high limit (PR #759)

@tcassaerthttps://github.com/tcassaert It goes through tests. Is the limit fine, or too low? I often specified 20Gb driver-memory in the past, so I got the feeling there is space for more

— Reply to this email directly, view it on GitHubhttps://github.com/Open-EO/openeo-geopyspark-driver/pull/759#issuecomment-2069639759, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABNJPSCWXHTHE2OCSWVZVKTY6UMOZAVCNFSM6AAAAABGSQZJTKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRZGYZTSNZVHE. You are receiving this because you are subscribed to this thread.Message ID: @.***>

VITO Disclaimer: http://www.vito.be/e-maildisclaimer

[tcassaert]

@EmileSonneveld for CDSE the limit the user sees should then be 15GB instead of 16GB.

[jdries]

Actually we shouldn't change this here, I'll update the config!

[jdries]

Config adjusted in the right place, closing this one.

[EmileSonneveld]

thanks