Path too long - Githubissues

mbernard commented 5 years ago

Using the repo command there's no way to specify the checkout path so if you have long path you get this exception 2018-12-03T21:55:26.2138619Z ##[error]Failed on repo Connect NameConflictException : failed to stat 'C:/Users/VssAdministrator/AppData/Local/Temp/NuKeeper/d88a2128699e46e2821ca3b86f86dc80/src/FrontEnd/Montrium.Connect.FrontEnd.WebApi/Spa/src/views/admin-view/list-management-view/contact-organization-view/organization-list-view/organization-list-right.view.tsx': The filename or extension is too long.

Could be nice to control C:/Users/VssAdministrator/AppData/Local/Temp/NuKeeper/d88a2128699e46e2821ca3b86f86dc80 the part so we can put something shorter

shep1987 commented 5 years ago

Hey @mbernard,

Thanks for trying out the tool! There will always come that time when a path is over 260 chars, so i'm just wondering if we can fix that problem instead of specifying a path.

Can i just check which version of windows you are on? Long path support was added in Windows 10 so if it is before that we may need to look into other options.

mbernard commented 5 years ago

@shep1987 Thanks for the fast answer. I'm actually running on an AzureDevOps hosted agent. It should be running on Windows 10.

I'm all with you though. If we can fix the issues at the OS or Git level it will be much better.

Any thoughs on how we could do it?

Here's the full console output log, mayby it can help

2018-12-03T21:55:15.2323187Z ##[section]Starting: NuKeeper
2018-12-03T21:55:15.2326415Z ==============================================================================
2018-12-03T21:55:15.2326472Z Task         : Command Line
2018-12-03T21:55:15.2326522Z Description  : Run a command line script using cmd.exe on Windows and bash on macOS and Linux.
2018-12-03T21:55:15.2326554Z Version      : 2.142.2
2018-12-03T21:55:15.2326583Z Author       : Microsoft Corporation
2018-12-03T21:55:15.2326615Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
2018-12-03T21:55:15.2326665Z ==============================================================================
2018-12-03T21:55:17.1846112Z Generating script.
2018-12-03T21:55:17.1919284Z Script contents:
2018-12-03T21:55:17.1926411Z nukeeper repo https://dev.azure.com/montrium/Connect/_git/Connect/ *** --consolidate
2018-12-03T21:55:17.2515967Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\482ed3c6-1956-4e69-a8b7-d6af46d01ea3.cmd""
2018-12-03T21:55:19.8857459Z Matched uri 'https://dev.azure.com/montrium/' to collaboration platform 'AzureDevOps'
2018-12-03T21:55:19.8884743Z User name missing from profile, falling back to .gitconfig
2018-12-03T21:55:21.4186058Z Using upstream fork as push, for project Connect at https://dev.azure.com/montrium/Connect/_git/Connect/
2018-12-03T21:55:21.4598846Z Git clone https://dev.azure.com/montrium/Connect/_git/Connect/ to C:\Users\VssAdministrator\AppData\Local\Temp\NuKeeper\d88a2128699e46e2821ca3b86f86dc80
2018-12-03T21:55:26.2138619Z ##[error]Failed on repo Connect NameConflictException : failed to stat 'C:/Users/VssAdministrator/AppData/Local/Temp/NuKeeper/d88a2128699e46e2821ca3b86f86dc80/src/FrontEnd/Montrium.Connect.FrontEnd.WebApi/Spa/src/views/admin-view/list-management-view/contact-organization-view/organization-list-view/organization-list-right.view.tsx': The filename or extension is too long.
2018-12-03T21:55:26.2523807Z ##[section]Finishing: NuKeeper

Looks like the git clone worked, but the failed to stat I have no idea what it is. Is it part of nuKeeper?

mbernard commented 5 years ago

@shep1987 someone is suggesting this

https://stackoverflow.com/a/40843613/3352999

Do you think it can help? I have unfortunately no way to test it

shep1987 commented 5 years ago

We have been talking about this on Gitter and think it is that LitGit2 doesn't support long paths https://github.com/libgit2/libgit2/issues/3053

I may look into using the Agent.TempDirectory env var (Set by default on the build server) which should be a shorter path. Will have a chat with people tomorrow about this being a viable work around.

For my ref: %tmp% and %temp% should already get overridden by the agent we use Path.GetTempPath() which doesn't seem to respect this.. https://github.com/Microsoft/azure-pipelines-agent/pull/824/files

AnthonySteele commented 5 years ago

If there's a way to fix this, we should document it.

By the way, if you run NuKeeper with an additional parameter, --verbosity detailed or -v d you will get a lot more details of that error.

So it seems overcomplex to work around with an additional commmandline arg (e.g. --tempFolder c:\some\path but if there's an existing environment var that can be used, we can try that!

mbernard commented 5 years ago

@AnthonySteele totally agree with you

Here's the detailed console log if it can help (we already knew it was LibGit2Sharp, but now it's clear :D)

2018-12-04T15:37:08.3678788Z ##[section]Starting: NuKeeper
2018-12-04T15:37:08.3682516Z ==============================================================================
2018-12-04T15:37:08.3682584Z Task         : Command Line
2018-12-04T15:37:08.3682624Z Description  : Run a command line script using cmd.exe on Windows and bash on macOS and Linux.
2018-12-04T15:37:08.3682682Z Version      : 2.142.2
2018-12-04T15:37:08.3682719Z Author       : Microsoft Corporation
2018-12-04T15:37:08.3682758Z Help         : [More Information](https://go.microsoft.com/fwlink/?LinkID=613735)
2018-12-04T15:37:08.3682985Z ==============================================================================
2018-12-04T15:37:11.7084057Z Generating script.
2018-12-04T15:37:11.7161387Z Script contents:
2018-12-04T15:37:11.7168304Z nukeeper repo https://dev.azure.com/montrium/Connect/_git/Connect/ *** --consolidate -v d
2018-12-04T15:37:11.7731956Z ##[command]"C:\Windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL "D:\a\_temp\0e924ff1-b903-4afd-bfc7-c938434a9933.cmd""
2018-12-04T15:37:13.7881900Z Matched uri 'https://dev.azure.com/montrium/' to collaboration platform 'AzureDevOps'
2018-12-04T15:37:13.7882143Z FindPushFork. Fork Mode is SingleRepositoryOnly
2018-12-04T15:37:13.7882197Z 2018-12-04T15:37:13Z: Started
2018-12-04T15:37:13.7882273Z User name missing from profile, falling back to .gitconfig
2018-12-04T15:37:13.7882326Z GetGitRepositories: Requesting Connect/_apis/git/repositories?api-version=4.1
2018-12-04T15:37:15.4244490Z Using upstream fork as push, for project Connect at https://dev.azure.com/montrium/Connect/_git/Connect/
2018-12-04T15:37:15.4244766Z Git clone https://dev.azure.com/montrium/Connect/_git/Connect/ to C:\Users\VssAdministrator\AppData\Local\Temp\NuKeeper\f320afb11aba423dbeb45b85041f2d95
2018-12-04T15:37:15.8961060Z 0 / 11557
2018-12-04T15:37:15.9523760Z 1155 / 11557
2018-12-04T15:37:16.0387065Z 2310 / 11557
2018-12-04T15:37:16.0451065Z 3465 / 11557
2018-12-04T15:37:16.0536577Z 4620 / 11557
2018-12-04T15:37:16.0776622Z 5775 / 11557
2018-12-04T15:37:16.1729634Z 6930 / 11557
2018-12-04T15:37:16.1980556Z 8085 / 11557
2018-12-04T15:37:16.2203855Z 9240 / 11557
2018-12-04T15:37:16.2635357Z 10395 / 11557
2018-12-04T15:37:16.2833069Z 11550 / 11557
2018-12-04T15:37:19.9600810Z ##[error]Failed on repo Connect NameConflictException : failed to stat 'C:/Users/VssAdministrator/AppData/Local/Temp/NuKeeper/f320afb11aba423dbeb45b85041f2d95/src/FrontEnd/Montrium.Connect.FrontEnd.WebApi/Spa/src/views/admin-view/list-management-view/contact-organization-view/organization-list-view/organization-list-right.view.tsx': The filename or extension is too long.

   at LibGit2Sharp.Core.Ensure.HandleError(Int32 result)
   at LibGit2Sharp.Core.Proxy.git_clone(String url, String workdir, GitCloneOptions& opts)
   at LibGit2Sharp.Repository.Clone(String sourceUrl, String workdirPath, CloneOptions options)
   at NuK
eeper.Git.LibGit2SharpDriver.Clone(Uri pullEndpoint) in D:\a\r1\a\drop\NuKeeper.Git\LibGit2SharpDriver.cs:line 44
   at NuKeeper.Engine.RepositoryUpdater.GitInit(IGitDriver git, RepositoryData repository) in D:\a\r1\a\drop\NuKeeper\Engine\RepositoryUpdater.cs:line 109
   at NuKeeper.Engine.RepositoryUpdater.Run(IGitDriver git, RepositoryData repository, SettingsContainer settings) in D:\a\r1\a\drop\NuKeeper\Engine\RepositoryUpdater.cs:line 50
   at NuKeeper.Engine.GitRepositoryEngine.Run(RepositorySettings repository, UsernamePasswordCredentials gitCreds, Identity userIdentity, SettingsContainer settings) in D:\a\r1\a\drop\NuKeeper\Engine\GitRepositoryEngine.cs:line 79
2018-12-04T15:37:19.9611069Z Done at 2018-12-04T15:37:19Z
2018-12-04T15:37:19.9998528Z ##[section]Finishing: NuKeeper

AnthonySteele commented 5 years ago

Are you happy to use the environment variable temp for the base checkout path? If so, #596 will allow this.

shep1987 commented 5 years ago

Yup is worth a try, you can set that from the build server if the pipeline doesn’t automatically, also may be worth falling back to ‘tmp’ for some Linux distro’s?

AnthonySteele commented 5 years ago

My windows machine has both temp and tmp set to the same value, so if tmp is also used on linux, perhaps that's the better option?

I have googled a bit, and TMPDIR seems to be used on some linux systems.

AnthonySteele commented 5 years ago

I think that we actually don't need to write code to fix this. NuKeeper finds the temp path by calling Path.GetTempPath() and According to the docs, that will check, in order:

The path specified by the TMP environment variable. The path specified by the TEMP environment variable

I don't see any mention of different behviour on Linux or in NetStandard.

Try setting TMP to a short path. e.g.

set TMP=C:\temp

Then this temp dir is used. I have verified that this already works.

mbernard commented 5 years ago

Seems to work fine when we assign the TEMP env var Thanks guys

NuKeeperDotNet / NuKeeper

Path too long #582