Symbolic link fails when "old" Github runner is used.

[johnnylarner]

Describe the bug When Github reuses a runner to execute the action, the spark installation fails due a symbolic link error. See here:

Note that when a "new" runner is used, the installation works fine. See how I distinguish between "old" and "new" runners in the To Reproduce section below.

I forked this repository and solved my issue by by changing the flags in line 67 of dist/.js from -s to -sf. As per the man page, this removes existing links before creating a new one.

ln -sf "${installFolder}/spark-${sparkVersion}-bin-hadoop${hadoopVersion}${scalaBit}" ${installFolder}/spark;

If you're on board, I'd create a PR with this change.

Which version of the action are you using? v1

Environment Self-hosted, linux

Spark Versions

To Reproduce Steps to reproduce the behavior:

1. Create a workflow with a Python installation and include pip3 install pyspark pytest as a command
2. One Python and Spark have installed, cancel the workflow and re-run all jobs to increase the chances of the same runner picking up the job. (This can also happen with normal "on push" triggers, but it's more difficult to reproduce.)
3. If the pip3 command is using cached packages, we know we are using an "old" runner
4. See error

Run/Repo Url My fork: https://github.com/johnnylarner/setup-spark

[vemonet]

Hi @johnnylarner , thanks a lot for the report and the debugging! I can't seems to access your fork, could you send a pull request so that I test and integrate your changes?

[johnnylarner]

Hi @vemonet, take a look at the PR at let me know if you have any issues.

[vemonet]

Sorry for the delay, I merged it thanks!