Symlinks not working on mac

[Fake51]

I've changed the directory setup a bit on my mac, and as a result submitting solutions fail. The cause is symlinking. Seems like something that should just work, so posting it here as a bug. Can't find other mentions of it.

I've got an Exercism directory, with language track symlinks in it

Exercism/elixir -> /Users/me/dev/own/Development/elixir/Exercism

Below that, everything is the same. The exercism command happily downloads exercises and they end up exactly like one expects. Submitting a solution is broken, though, with the following error:

Error: not in workspace

My config looks like:

Home: /Users/me Workspace: /Users/me/Exercism Config: /Users/me/.config/exercism

[iHiD]

Triaging to CLI

[iHiD]

(cc @exercism/cli)

[NobbZ]

Symlinks will be resolved, and after following the symlinks, you are not in workspace anymore. Either configure your workspace to be at the desired location or do a hardlink or copy the files over to where the workspace is configured.

[Fake51]

Why are symlinks resolved? What purpose does it solve? It just breaks working file system features - and as noted, it works for downloading exercises, just not submitting solutions.

On Fri, 3 Jan 2020, 17:27 Norbert Melzer, notifications@github.com wrote:

Symlinks will be resolved, and after following the symlinks, you are not in workspace anymore. Either configure your workspace to be at the desired location or do a hardlink or copy the files over to where the workspace is configured.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/exercism/cli/issues/911?email_source=notifications&email_token=AAA6MOVBO6NODN2QK2YWNQLQ35RPJA5CNFSM4KB3TX62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIBQGCY#issuecomment-570622731, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6MOTY364GXFYP3YZ5PXTQ35RPJANCNFSM4KB3TX6Q .

[NobbZ]

As far as I remember, the symlink gets already resolved when you cd into the directory.

It should work though, when you specify an absolute path to the file… exercism submit $(exercism workspace)/elixir/hello-world/lib/hello_world.ex (or what the proper name was…)

Anyway, its probably easier to just stick to the exercism convention to keep everything physically in the exercism workspace.

[Fake51]

I'll test the absolute path like that but what I've seen so far doesn't work. And I'm sure it's better to ignore this issue - I also think this is working the wrong way, so I don't see why it should be ignored.

I'll have a look to see if there's anything I can contribute

On Sun, 5 Jan 2020, 14:18 Norbert Melzer, notifications@github.com wrote:

As far as I remember, the symlink gets already resolved when you cd into the directory.

It should work though, when you specify an absolute path to the file… exercism submit $(exercism workspace)/elixir/hello-world/lib/hello_world.ex (or what the proper name was…)

Anyway, its probably easier to just stick to the exercism convention to keep everything physically in the exercism workspace.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/exercism/cli/issues/911?email_source=notifications&email_token=AAA6MOUN6756DY5ZAANRZULQ4HMZZA5CNFSM4KB3TX62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIDWYRQ#issuecomment-570911814, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA6MOXXVDAVKIC77IOV6JDQ4HMZZANCNFSM4KB3TX6Q .

[Fake51]

I've had a look and tweak of the source code for exercism cli - and it's down to a "validation" of the paths, that evaluates symlinks. It's easy to turn off and the cli will without problem submit the solution to the right place. I can't, from the current code, see why evaluating the symlinks should be a necessity for anything.

This patch removes the problem:

diff --git a/cmd/submit.go b/cmd/submit.go
index 4f2a8df..ef5f3ae 100644
--- a/cmd/submit.go
+++ b/cmd/submit.go
@@ -138,11 +138,7 @@ func (s *submitCmdContext) evaluatedSymlinks(submitPaths []string) ([]string, er
            return nil, err
        }

-       src, err := filepath.EvalSymlinks(path)
-       if err != nil {
-           return nil, err
-       }
-       evalSymlinkSubmitPaths = append(evalSymlinkSubmitPaths, src)
+       evalSymlinkSubmitPaths = append(evalSymlinkSubmitPaths, path)
    }
    return evalSymlinkSubmitPaths, nil
 }

Current code was written on Jan 5th 2019, so it's a bit dated, but would be interesting to know if the author has insight into why symlinks need evaluating. I really don't see how it adds any value at all - unless it's a cross operating system thing.

[NobbZ]

As far as I remember that was introduced to at least partially solve some problems around case sensitivity of the path on case insensitive filesystems.

Though to be honest, it does not seem to work at all, so yes, I'm not sure if it is necessary to still canonicalize pathes.

We should wait for @kytrinyx here.

[Fake51]

Is @kytrinyx currently working on this project? Last activity was three months ago it seems, and nothing on this ticket even though it should be a fairly simple go/no-go

[NobbZ]

She is, though decisions aren't usually made by a single person, but by a team of 4 or 5, meeting irregularly.

Also in the background a lot of energy is going into version 3. Therefore it might be, that there is currently enough put on her and the product team.

[iHiD]

Is @kytrinyx currently working on this project?

Yes.

All told, I get about 150 notifications per day for Exercism. I imagine Katrina gets similar. With all things Exercism, its not too little effort, it's too much going on :) We all also have day jobs :)

So for tickets where only one person has reported an issue over a two year period, that issue will fall lower in the queue of things to reply to than many other things that arise every day.

I really don't see how it adds any value at all - unless it's a cross operating system thing. a fairly simple go/no-go

These things really are never simple. There would be some reason someone went to the effort of writing this code. Probably, as you say, some cross-OS weirdness. And just the process of getting one's mind back into a codebase that one hasn't touched for a while, and remembering why a decision was made, then evaluating the tradeoffs of changing it, for a product used by hundreds of thousands of people, is something that involves care, brain-power and time. All of which are thinly spread in projects run by volunteers :)

We really (genuinely) appreciate you opening the ticket, and the lack of response shouldn't be taken as us not being bothered by it. However, if you look at the number of open issues here, on exercism/exercism, on exercism/v3, etc, you'll notice there that there are quite literally thousands of things that require people's attention, so we'll have to ask for your patience a little longer :)

[Fake51]

ll told, I get about 150 notifications per day for Exercism. I imagine Katrina gets similar. With all things Exercism, its not too little effort, it's too much going on :) We all also have day jobs :)

Yeah, sorry if it came off as a criticism. I was just somewhat surprised, because there was a lot of feedback upon opening the issue, but after it got assigned there was nothing for two weeks. Anyway, I don't know the organisation or the work going on, so it was silly to assume :)

So for tickets where only one person has reported an issue over a two year period, that issue will fall lower in the queue of things to reply to than many other things that arise every day.

Of course :) It's just also a ticket with a solution attached to it, making it low-hanging fruit.

We really (genuinely) appreciate you opening the ticket, and the lack of response shouldn't be taken as us not being bothered by it. However, if you look at the number of open issues here, on exercism/exercism, on exercism/v3, etc, you'll notice there that there are quite literally thousands of things that require people's attention, so we'll have to ask for your patience a little longer :)

Apologies for coming across as annoyed or impatient - that wasn't the intention. I really appreciate the project and the work going into it, it's great!

[rcalixte]

As far as I remember, the symlink gets already resolved when you cd into the directory.

It should work though, when you specify an absolute path to the file… exercism submit $(exercism workspace)/elixir/hello-world/lib/hello_world.ex (or what the proper name was…)

Anyway, its probably easier to just stick to the exercism convention to keep everything physically in the exercism workspace.

~/exercism/python$ exercism submit $(exercism workspace)/python/hello-world/hello_world.py
Error: not in workspace
~/exercism/python$ exercism c -s
Workspace:     (-w, --workspace)  ~/exercism

Adding support for symlinks and not supporting case-insensitive filesystems seems to be the better trade-off here imho. Supporting case-insensitive filesystems seems to be far more troublesome than there is any value gained.

As far as I remember, the symlink gets already resolved when you cd into the directory.

With any POSIX implementation of cd, by default symlinks are not resolved. In order to resolve symlinks, one must cd -P. From the documentation: The -P option causes cd to use the physical directory structure by resolving symbolic links while traversing dir and before processing instances of .. in dir

[joshgoebel]

My 0.02 here since I just ran into this:

• I also do not understand why we're resolving symlinks/real paths. Right now it's making it impossible for me to submit exercises that include library code that I'm symlinking in from a common place.
• I do think we should support case-insensative filesystems (almost everyone with a Mac), but limit it to handling Exercism vs exercism which is the only issue I've ever seen on my Mac related to case.

[joshgoebel]

This can evidently be traced back to dfcb956

[iHiD]

Will accept a PR.

[kotp]

This can evidently be traced back to dfcb956

Specifically the reasoning given:

From: Katrina Owen kytrinyx@github.com Date: Wed, 11 Jul 2018 15:31:11 +0100 Subject: [PATCH] Handle submitting relative paths to files

To test relative paths we change directory into the temporary directory and then pass the relative directory as an argument.

On the Mac, temporary directories are created within /var which is a symlinked directory under /private:

$ ls -l /var
lrwxr-xr-x@ 1 root  wheel  11 Dec 17  2017 /var -> private/var

Temporary directories are returned using the path within the symlinked path (/var/folders/...) rather than the source path (/private/var/folders/...).

To test the relative path change directory into the temporary directory, then pass the relative path as an argument, which then gets resolved into an absolute path.

Before submitting, we make sure that the file is within the configured workspace. However, this now causes a mismatch, because the workspace path is the one with the symlink, whereas the resolved absolute path in this case is the source path.

To fix this, we evaluate the symlinks in the workspace path and resolve these before comparing them to the submitted files.

The Windows build fails with this test, so I've separated the relevant tests out into a conditional build. Once we get it fixed it may make more sense to inline it into the main submit test and do a little bit of conditional setup.

I hope the inclusion of the commit message here helps with context.