`make clean` (only)after `make test` deletes some files (on Windows at least)

[ghost]

originally from here: https://groups.google.com/d/msg/elixir-lang-talk/Z1GmF9FtSgk/N4BaX2bpJlsJ

reproduce with this: the more detailed steps are here: https://groups.google.com/d/msg/elixir-lang-talk/Z1GmF9FtSgk/ni8RagcZ4wkJ

but if you don't care to see when the deletion doesn't happen then do only the following quick steps which shows when deletion does happen:

1. do mingw32-make clean
2. do git checkout . (WARNING: make sure you have no modified files in this repo git status before this, obviously those changes will get lost!)
3. do mingw32-make test
4. do mingw32-make clean
5. do git status and notice all those deleted files

(I hope this doesn't have anything to do with the fact that on Windows some tests fail)

tested above steps with this commit: https://github.com/elixir-lang/elixir/commit/7f18e1426e16e1ac7e55f1da99358de00172bd27

Thu 07/25/2013 15:02:01.10 s:\workspace.elixir.2013\elixir>git status
# On branch master
# Changes not staged for commit:
#   (use "git add/rm <file>..." to update what will be committed)
#   (use "git checkout -- <file>..." to discard changes in working directo
ry)
#
#       deleted:    lib/elixir/test/elixir/fixtures/at_exit.exs
#       deleted:    lib/elixir/test/elixir/fixtures/code_sample.exs
#       deleted:    lib/elixir/test/elixir/fixtures/compile_sample.ex
#       deleted:    lib/elixir/test/elixir/fixtures/compiled_with_docs.ex
#       deleted:    lib/elixir/test/elixir/fixtures/cp_mode
#       deleted:    lib/elixir/test/elixir/fixtures/cp_r/a/1.txt
#       deleted:    lib/elixir/test/elixir/fixtures/cp_r/a/a/2.txt
#       deleted:    lib/elixir/test/elixir/fixtures/cp_r/b/3.txt
#       deleted:    lib/elixir/test/elixir/fixtures/file.txt
#       deleted:    lib/elixir/test/elixir/fixtures/init_sample.exs
#       deleted:    lib/elixir/test/elixir/fixtures/one-liner.txt
#       deleted:    lib/elixir/test/elixir/fixtures/parallel_compiler/bar.ex
#       deleted:    lib/elixir/test/elixir/fixtures/parallel_compiler/foo.ex
#       deleted:    lib/elixir/test/elixir/fixtures/parallel_deadlock/bar.ex
#       deleted:    lib/elixir/test/elixir/fixtures/parallel_deadlock/foo.ex
#       deleted:    lib/elixir/test/elixir/fixtures/utf8.txt
#       deleted:    lib/elixir/test/elixir/fixtures/warnings_sample.ex
#
# Untracked files:
#   (use "git add <file>..." to include in what will be committed)

[josevalim]

Thank you! @tojans could you please check if you can reproduce this issue?

[ghost]

I am curious, if this also happens in unix-like environments? Should I try Linux Mint in a virtualbox? or you already knows it doesn't happen?

EDIT: just tried on Linux Mint and doesn't happen (but all tests pass) Linux atkaaz-VirtualBox 3.8.0-19-generic #29-Ubuntu SMP Wed Apr 17 18:16:28 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux Erlang R16B01 (erts-5.10.2) [source-bdf5300] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10.2 (abort with ^G) on this commit: https://github.com/elixir-lang/elixir/commit/751ebc8c63337c3f629b83d325660615526159f1

EDIT2: also having the bin folder(of the repo) in path doesn't affect the result (on linux mint)

EDIT3: I also tried reproducing these 2 errors(from windows) into linux:

  1) test_path (Kernel.CLI.OptionParsingTest)
     ←[31m** (Protocol.UndefinedError) protocol Enumerable not implemented
 for nil←[0m
     ←[36mat test/elixir/kernel/cli_test.exs:25←[0m

  2) test_rm_rf_with_symlink (FileTest.Rm)
     ←[31m** (MatchError) no match of right hand side value: {:error, {:er
ror, :eperm}, "s:/workspace.elixir.2013/elixir/lib/elixir/tmp/tmp/link"}←[
0m
     ←[36mat test/elixir/file_test.exs:777←[0m

and still make clean doesn't delete files :)

[josevalim]

It definitely doesn't happen here (Max OS X 10.8.4).

[ToJans]

Been with my son in the hospital today (unplanned), so Elixir had to wait a bit. I hope I can start tomorrow... Can you assign this to me please?

[ghost]

Oh I get it, there's a folder symlink S:\workspace.elixir.2013\elixir\lib\elixir\test\tmp\tmp\link to the S:\workspace.elixir.2013\elixir\lib\elixir\test\elixir\fixtures folder and when that is deleted well all files inside it are also deleted, ie. maybe the symlink isn't deleted properly on windows?

there are actually two folder symlinks: S:\workspace.elixir.2013\elixir\lib\elixir\tmp\tmp\link S:\workspace.elixir.2013\elixir\lib\elixir\tmp\tmp\link

they both point to S:\workspace.elixir.2013\elixir\lib\elixir\test\elixir\fixtures

the bad part(EDIT2: which makes me sad btw) is that I've had a gut feeling that this was the case all along, but I just chose to believe there's no effin way elixir tools are making windows folder symlinks (even though I knew they are possible via mklink internal command on windows, and therefore ntfs possible)

EDIT: if I manually delete those symlink folders and then do make clean nothing from the OP is deleted... maybe some delete folder function is getting inside the folder and trying to delete the files? instead of checking if the folder is actually a symlink folder? EDIT3: yet for some reason this isn't a problem in plinux

[ToJans]

Wow, that is a thorough analysis! Good job!

[ghost]

Thanks. (wish I followed my initial gut feeling though, but hey I'm learning from this xp, so I'm more likely to follow it in the future, perhaps)

and it's this command: https://github.com/elixir-lang/elixir/blob/5116e28aacd0c623dae924678e38174744342fa5/Makefile#L96 something odd with rm I will check EDIT: that is rm.exe (for Windows)

EDIT2: C:\Program Files (x86)\Git\bin\rm.EXE

Thu 07/25/2013 21:01:13.20 C:\Users\user>rm --version
rm (fileutils) 4.1
Written by Paul Rubin, David MacKenzie, Richard Stallman, and Jim Meyering
.

Copyright (C) 2001 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE
.

EDIT3: I have a newer version of rm

Thu 07/25/2013 21:12:02.73 s:\gnuwin32\bin>rm --version
rm (GNU coreutils) 5.3.0
Written by Paul Rubin, David MacKenzie, Richard Stallman, and Jim Meyering.

Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

However when I try using this one, make clean fails because this rm acts differently (maybe it's -f arg is weaker than before):

Thu 07/25/2013 21:11:51.44 s:\workspace.elixir.2013\elixir>make clean

Thu 07/25/2013 21:13:42.19 s:\workspace.elixir.2013\elixir>set PATH=s:\gnu
win32\bin;C:\Program Files\erl5.10.2\bin;c:\mingw\bin;C:\Program Files (x8
6)\Git\bin;c:\windows\system32

Thu 07/25/2013 21:13:42.20 s:\workspace.elixir.2013\elixir>mingw32-make cl
ean
cd lib/elixir && "/s/workspace.elixir.2013/elixir/rebar" clean
==> elixir (clean)
rm -rf ebin
rm -rf lib/*/ebin
s:\gnuwin32\bin\rm.exe: cannot lstat `lib/*/ebin': Invalid argument
Makefile:93: recipe for target 'clean' failed
mingw32-make: *** [clean] Error 1

(btw this happens when already cleaned)

EDIT5: that old rm 4.1 is from Git-1.8.3-preview20130601.exe Full installer for official Git for Windows 1.8.3 Featured Beta
Jun 2 Jun 2 14.8 MB 379577 from here: https://code.google.com/p/msysgit/downloads/list which was latest version at this time. So hmm, it doesn't know how to delete folder symlinks like the linux version of rm does.

EDIT6: so i tested this rm 5.3.0 version and it does the same thing

Thu 07/25/2013 21:17:50.31 s:\workspace.elixir.2013\elixir>rm --version
rm (GNU coreutils) 5.3.0
Written by Paul Rubin, David MacKenzie, Richard Stallman, and Jim Meyering
.

Copyright (C) 2005 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE
.

It doesn't fail if you call make clean when not already clean, so make test then make clean doesn't err, but it still deletes those files in OP. EDIT7: double checked that it's using the right rm(v 5.3.0) by renaming the 4.1 rm (and rmdir even) .exe files; same result.

[josevalim]

I have changed Elixir to not symlink to the fixture path (just symlink to stuff in tmp) and clean up the tmp directory after each test.

[ghost]

With this commit: https://github.com/elixir-lang/elixir/commit/91d5ee1d8f1c4a3ba16a8d591bd93fa9d42d70be ;P (I like to keep things connected where possible)

What I was actually doing meanwhile was trying to compile coreutils version 8.0 (into which fileutils have been merged) [and it just failed configure: error: could not determine how to read list of mounted file systems] just to check if rm was somewhat fixed, even so, chances of anyone having this version even if it worked = inexistent, ergo

your solution is way better! Thank you! :+1: (&tested to work as expected)

EDIT:

Finished in 33.7 seconds (17.9s on load, 15.7s on tests)
1196 tests, 99 failures
Makefile:151: recipe for target 'test_kernel' failed

(the tests are different, so not sure if anything works in this regard)