Warning / Skip Symlinks In a Repo?

[cycomachead]

Would it be possible to improve the experience with symlinks in a repo? It took me a good few hours to realize a bad symlink was the case of many failures.

The 'no symlink' thing is mentioned in the README, but doesn't show in the build output. (I'm also using the default GH Pages build, which isn't super obvious this is the source repo for the build.)

Here's what I ran that tipped me off to the bad file. find -L . -type l -ls. In this case, the repo has been around for a while and in the old old days symlinks weren't an issue when using Pages, and now that I'm re-enabling Pages, I wasn't aware of some of the problem files.

If this sounds useful, I can try to author a PR - though I am no windows expert and now sure how I'd test that. :)

[yoannchaudet]

Typically if you use the Action in this repository, the fact that we don't allow symlinks is transparent because we dereference them in the tar archive we create.

Did you experience a different behavior?

[ilg-ul]

I also faced a similar problem, the web site had some bad symlinks with absolute paths and the job failed with:

2023-02-18T19:55:26.3075573Z ##[group]Run actions/upload-pages-artifact@v1
2023-02-18T19:55:26.3075888Z with:
2023-02-18T19:55:26.3076065Z   path: ./docs
2023-02-18T19:55:26.3076253Z   name: github-pages
2023-02-18T19:55:26.3076563Z   retention-days: 1
2023-02-18T19:55:26.3076760Z env:
2023-02-18T19:55:26.3076935Z   GITHUB_PAGES: true
2023-02-18T19:55:26.3077124Z ##[endgroup]
2023-02-18T19:55:26.3349609Z ##[group]Run chmod -c -R +rX "$INPUT_PATH" | while read line; do
2023-02-18T19:55:26.3350298Z chmod -c -R +rX "$INPUT_PATH" | while read line; do
2023-02-18T19:55:26.3350646Z   echo "::warning title=Invalid file permissions automatically fixed::$line"
2023-02-18T19:55:26.3350943Z done
2023-02-18T19:55:26.3351127Z tar \
2023-02-18T19:55:26.3351358Z   --dereference --hard-dereference \
2023-02-18T19:55:26.3351627Z   --directory "$INPUT_PATH" \
2023-02-18T19:55:26.3351890Z   -cvf "$RUNNER_TEMP/artifact.tar" \
2023-02-18T19:55:26.3352123Z   --exclude=.git \
2023-02-18T19:55:26.3352386Z   --exclude=.github \
2023-02-18T19:55:26.3352581Z   .
2023-02-18T19:55:26.3411442Z shell: /usr/bin/sh -e {0}
2023-02-18T19:55:26.3411714Z env:
2023-02-18T19:55:26.3411909Z   GITHUB_PAGES: true
2023-02-18T19:55:26.3412110Z   INPUT_PATH: ./docs
2023-02-18T19:55:26.3412303Z ##[endgroup]
2023-02-18T19:55:26.3603214Z ./
2023-02-18T19:55:26.3603628Z tar: ./dev: File removed before we read it.  <--- bad symlink
2023-02-18T19:55:26.3603929Z tar: ./stable: File removed before we read it  <--- bad symlink
2023-02-18T19:55:26.3604214Z tar: ./v6.0: File removed before we read it  <--- bad symlink
2023-02-18T19:55:26.3604808Z ./.nojekyll
2023-02-18T19:55:26.3605577Z ./v6.0.0/
2023-02-18T19:55:26.3605969Z ./v6.0.0/frontend/
2023-02-18T19:55:26.3606707Z ./v6.0.0/frontend/yafElement.js
2023-02-18T19:55:26.3607108Z ./v6.0.0/frontend/webComponents/
...
2023-02-18T19:55:26.3727577Z ./v6.0.0/media/fonts/roboto/ThinItalic.woff
2023-02-18T19:55:26.3727791Z ./v6.0.0/index.html
2023-02-18T19:55:26.3727980Z ./versions.js
2023-02-18T19:55:26.3728234Z ./.typedoc-plugin-versions
2023-02-18T19:55:26.3728440Z ./index.html
2023-02-18T19:55:26.3741673Z ##[error]Process completed with exit code 1.

Only after reading this ticket I realised the links are wrong.

I don't know what would be an appropriate solution, but the experience was not great. :-(

[cycomachead]

Thanks for looking into this! I think, but am not sure, that the issue might have been a bad symlink, or at least to one outside of the repo. After finding and removing symlinks, I got the site to build fine.

[ilg-ul]

Ah, I forgot to mention, in my case the issue was definitely caused by the broken absolute links (dev, stable, v6.0). I wrote a shell script to replace them with relative symlinks and this upload action ran smoothly.

Not being able to tar a folder with broken absolute symlinks is somehow expected, but I thing that this action should check if such symlinks exist and throw explicit error messages, since messages like tar: ./dev: File removed before we read it are far from obvious.

Relative symlinks inside the folder are fine, tar can process them successfully (on Unix-like machines).

[RadxaYuntian]

I too want to understand the reasoning behind this no link rule. GitHub Pages works fine with symbolic link when deployed from a branch. We use Pages to host some Debian package archives, where packages are symbolic linked multiple times due to its format. De-reference all of them results in significantly larger artifacts to upload, that not only takes longer time, but also run us close to the size limit.

We most likely will fork this action to make that happen, but before we do that we still want to know why it was forbidden, since there might be side effects that we are unaware of..

[yoannchaudet]

Symbolic links have been a recurring vector of attack of our hosting infrastructure both on GitHub.com and GitHub Enterprise Server. This is the reason we disabled their entire usage sometime last year.

[vorburger]

More just FYI, but somewhat related to this, I've managed to make an upload-pages-artifact action go into an "endless loop" with this, which was a symlink to a higher up directory - that seems to make the tar --dereference --hard-dereference here go crazy. (I've "fixed" for me with work around by this). Not sure if you want to do anything about such cases.