teragrep / pth_10

Data Processing Language (DPL) translator for Apache Spark
GNU Affero General Public License v3.0
0 stars 6 forks source link

Bind spark listening and server to localhost in tests #296

Closed StrongestNumber9 closed 1 month ago

StrongestNumber9 commented 2 months ago

Description

Bind spark listening and server to localhost in tests.

Use case or motivation behind the feature request

Frewall rules can cause tests to timeout in certain environments if it decides to access the spark server via public ip and firewall drops such traffic

Related issues

Additional context

https://spark.apache.org/docs/latest/configuration.html#networking

StrongestNumber9 commented 2 months ago

Maybe lowering spark.network.timeout to something like 5 seconds is useful as something has gone catastrophically wrong if we stall for even 5 seconds with networking when testing locally

eemhu commented 2 months ago

internal PR submitted