Set SPARK_LOCAL_IP env var.

[ueshin]

Try to fix unstable build by explicitly setting SPARK_LOCAL_IP env var.

https://github.com/databricks/koalas/runs/2297499540

java.net.BindException: Cannot assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.

[codecov-io]

Codecov Report

Merging #2144 (3bd77ed) into master (c8f803d) will decrease coverage by 1.86%. The diff coverage is n/a.

:exclamation: Current head 3bd77ed differs from pull request most recent head 6848178. Consider uploading reports for the commit 6848178 to get more accurate results

@@            Coverage Diff             @@
##           master    #2144      +/-   ##
==========================================
- Coverage   95.36%   93.50%   -1.87%     
==========================================
  Files          60       60              
  Lines       13705    13705              
==========================================
- Hits        13070    12815     -255     
- Misses        635      890     +255     

Impacted Files	Coverage Δ
databricks/koalas/usage_logging/__init__.py	27.27% <0.00%> (-65.29%)	:arrow_down:
databricks/koalas/usage_logging/usage_logger.py	47.82% <0.00%> (-52.18%)	:arrow_down:
databricks/koalas/missing/series.py	60.56% <0.00%> (-39.44%)	:arrow_down:
databricks/koalas/missing/frame.py	75.00% <0.00%> (-25.00%)	:arrow_down:
databricks/koalas/__init__.py	80.26% <0.00%> (-11.85%)	:arrow_down:
databricks/koalas/missing/indexes.py	88.63% <0.00%> (-11.37%)	:arrow_down:
databricks/conftest.py	90.62% <0.00%> (-9.38%)	:arrow_down:
databricks/koalas/spark/accessors.py	88.69% <0.00%> (-5.96%)	:arrow_down:
databricks/koalas/accessors.py	86.03% <0.00%> (-5.86%)	:arrow_down:
databricks/koalas/typedef/typehints.py	91.32% <0.00%> (-4.09%)	:arrow_down:
... and 10 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update c8f803d...6848178. Read the comment docs.

[ueshin]

The error didn't happen during the five builds. I'd merge this now. cc @HyukjinKwon

[HyukjinKwon]

Interesting. Thanks @ueshin for taking a look for this!

[HyukjinKwon]

This is a good idea. is it because localhost fails to get 127.0.01 loopback properly for whatever reason (malformed /etc/hosts?). One way is to manually fix /etc/hosts but I think this is good enough.

[ueshin]

If we set the env var, actually Spark skips some logic in org.apache.spark.util.Utils.findLocalInetAddress to find a local net address, which sometimes fails for some reason, and force to use the value. Unfortunately, I'm not familiar with around the code, though.

[HyukjinKwon]

Nice, yeah, probably we're talking about the same thing I suspect. Thanks for fixing it!!

[harupy]

I applied the same fix in the mlflow repo, but it didn't work 😅

https://github.com/mlflow/mlflow/pull/4233

[HyukjinKwon]

😢

[ueshin]

Using local-cluster might be a different story..

[harupy]

@HyukjinKwon @ueshin

I found a mistake in my PR 😅 My github actions config looked like below.

# github actions config

      - name: Install dependencies
        env:
          SPARK_LOCAL_IP: 127.0.0.1  # this needs to be specified in the `Run tests` step.
        run: |
          ...
      - name: Run tests
        run: |
          ...