Closed jefffhaynes closed 12 months ago
Can you remember any error message, even vaguely?
Going forward, can you use west -v -v init
?
There is no error message until I run update as the init command doesn't fail, it just leaves everything in the wrong place. I'll try the new command.
It could fail without a visible error message. It's extremely rare with Python
and even rarer with west
but it's theoretically possible. Next time this happens please use echo %errorlevel%
immediately
Running west -v -v init
seems to have worked, although I don't know if that was the intention or if it just happened to work that time. If it happens again I'll try to get the error level. Thanks.
-v -v
should just increase the verbosity level without otherwise changing behavior (other than precise timing details of when things happen, of course), but the verbose logs would be very useful for diagnosing this. I haven't seen this happen myself, but I believe you, and it'd be good to get more details from the verbose logs.
Hi everyone, I have same problem when run west init
on windows. All everything in the folder manifest-tmp
moved folder zephyr
, but The folder manifest-tmp
is not removed. That make can not run west update
.
This is error display, Anyone have solution ? Thanks !
west.manifest: loaded C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
Traceback (most recent call last):
File "c:\python38\lib\shutil.py", line 780, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp' -> 'C:\\Users\\congnh\\zephyrproject\\zephyr'
Can you reproduce every time? If yes then please follow the instructions above.
yes, it can reproduce. you mean use west -v -v init
but I try and can not
I think he's asking you to post what it says when you do.
This is all display when you did, it relates permission access folder when run os.rename
$ west -v -v init -m https://bitbucket.com/xx/xx/zephyr.git --mr xxxx.xx.xx.x --mf xxxx.yml
=== Initializing in C:\Users\congnh\zephyrproject
--- Cloning manifest repository from https://bitbucket.com/xx/xx/zephyr.git
running 'git clone --branch https://bitbucket.com/'C:\Users\congnh\zephyrproject\.west\manifest-tmp'' in C:\Users\congnh\zephyrproject
Cloning into 'C:\Users\congnh\zephyrproject\.west\manifest-tmp'...
remote: Enumerating objects: 578417, done.
remote: Counting objects: 100% (578417/578417), done.
remote: Compressing objects: 100% (126962/126962), done.
remote: Total 578417 (delta 440454), reused 577231 (delta 439607)
Receiving objects: 100% (578417/578417), 383.54 MiB | 2.33 MiB/s, done.
Resolving deltas: 100% (440454/440454), done.
Note: switching to 'khsvt32h567586t8k'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
Updating files: 100% (13543/13543), done.
west.manifest: loading C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: group-filter: unset
west.manifest: added project cmsis path modules/hal/cmsis revision from C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: added project hal_nxp path modules/hal/nxp revision from C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
west.manifest: loaded C:\Users\congnh\zephyrproject\.west\manifest-tmp\zephyrbsp.yml
Traceback (most recent call last):
File "c:\python38\lib\shutil.py", line 780, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp' -> 'C:\\Users\\congnh\\zephyrproject\\zephyr'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\python38\lib\runpy.py", line 192, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\python38\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Python38\Scripts\west.exe\__main__.py", line 9, in <module>
File "c:\python38\lib\site-packages\west\app\main.py", line 781, in main
app.run(argv or sys.argv[1:])
File "c:\python38\lib\site-packages\west\app\main.py", line 106, in run
self.run_command(argv)
File "c:\python38\lib\site-packages\west\app\main.py", line 336, in run_command
cmd.run(args, unknown, self.topdir, manifest=self.manifest)
File "c:\python38\lib\site-packages\west\commands.py", line 140, in run
self.do_run(args, unknown)
File "c:\python38\lib\site-packages\west\app\project.py", line 198, in do_run
topdir = self.bootstrap(args)
File "c:\python38\lib\site-packages\west\app\project.py", line 305, in bootstrap
shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
File "c:\python38\lib\shutil.py", line 792, in move
rmtree(src)
File "c:\python38\lib\shutil.py", line 730, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\python38\lib\shutil.py", line 603, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\python38\lib\shutil.py", line 608, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "c:\python38\lib\shutil.py", line 606, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-2535647586gfhf56757.idx'
PermissionError: [WinError 5] Access is denied: 'C:\Users\congnh\zephyrproject\.west\manifest-tmp' -> 'C:\Users\congnh\zephyrproject\zephyr'
What happens if you try to rename this yourself from a command window?
west.manifest: loaded C:\Users\congnh8\Desktop\New folder.west\manifest-tmp\zephyrbsp.yml
Even stranger: is this imported from another manifest?
This is not the same problem as originally reported in the description BTW
west.manifest: loaded C:\Users\congnh8\Desktop\New folder.west\manifest-tmp\zephyrbsp.yml
sorry, I rename it , it must be this
west.manifest: loaded C:\Users\congnh\zephyrproject\manifest-tmp\zephyrbsp.yml
sorry, I rename it , it must be this
I don't understand what you mean, sorry. It looks like you have edited the logs. This makes it hard to understand them since I cannot know for sure how the output comes from the input, or even if the output is accurate.
Maybe you are using a private repository. I understand if you cannot share that.
But do you maybe have some way to make a manifest you can share which reproduces the issue for you? That would be really helpful, thank you.
yep. because some private information, i cannot post, so edited somethings in logs. but basically it is also same
If you cannot reproduce with the open-source Zephyr manifest (did you try?), it means something in your manifest(s) is triggering the problem. To identify what triggers the issue, remove the private elements one by one until the problem disappears. Now you found what triggers the issue. Finally, to reproduce the issue without any confidential information, add something similar (and not confidential) to the open source manifest. Now you have a bug report that you can fully share and something in common we can all work on.
With community source Zephyr, I have same problem. I try with other machines, some can work but some cannt
C:\Users\datnd14>west init -m https://github.com/zephyrproject-rtos/zephyr.git --mr zephyr-v2.7.0 --mf west.yml
=== Initializing in C:\Users\datnd14
--- Cloning manifest repository from https://github.com/zephyrproject-rtos/zephyr.git, rev. zephyr-v2.7.0
Initialized empty Git repository in C:/Users/datnd14/.west/manifest-tmp/.git/
remote: Enumerating objects: 642618, done.
remote: Counting objects: 100% (2/2), done.
remote: Total 642618 (delta 1), reused 1 (delta 1), pack-reused 642616 eceiving objects: 100% (642618/642618), 4
Receiving objects: 100% (642618/642618), 433.69 MiB | 25.00 KiB/s, done.
Resolving deltas: 100% (488975/488975), done.
* [new tag] zephyr-v2.7.0 -> zephyr-v2.7.0
503c7138df2b7b0afa0f649c783b33e8d971f4d0 refs/tags/zephyr-v2.7.0
Updating files: 100% (17693/17693), done.
Note: switching to 'zephyr-v2.7.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:
git switch -c <new-branch-name>
Or undo this operation with:
git switch -
Turn off this advice by setting config variable advice.detachedHead to false
HEAD is now at 3f826560aa release: Zephyr 2.7.0
Traceback (most recent call last):
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 806, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\datnd14\\.west\\manifest-tmp' -> 'C:\\Users\\datnd14\\zephyr'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\datnd14\AppData\Local\Programs\Python\Python39\Scripts\west.exe\__main__.py", line 7, in <module>
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 779, in main
app.run(argv or sys.argv[1:])
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 106, in run
self.run_command(argv)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\main.py", line 336, in run_command
cmd.run(args, unknown, self.topdir, manifest=self.manifest)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\commands.py", line 132, in run
self.do_run(args, unknown)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\project.py", line 198, in do_run
topdir = self.bootstrap(args)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\site-packages\west\app\project.py", line 295, in bootstrap
shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 824, in move
rmtree(src)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 740, in rmtree
return _rmtree_unsafe(path, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 613, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 618, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "c:\users\datnd14\appdata\local\programs\python\python39\lib\shutil.py", line 616, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\datnd14\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-92b9de99850183d98ff55ccee114517db6eeae5c.idx'
What happens when you try this after the failure:
del C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-92b9de99850183d98ff55ccee114517db6eeae5c.idx
Please also share the output of west init -v -v...
and edit your comment above.
I tried delete that, but have access error although I grant permission
del C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-5fe6ccedee5ad97c70bfdbc8b9476c8c056f7a98.idx
C:\Users\datnd14\.west\manifest-tmp\.git\objects\pack\pack-5fe6ccedee5ad97c70bfdbc8b9476c8c056f7a98.idx
Access is denied.
when run west init -v -v
, log is quite long, so you can read in this file log.txt
Thanks! Can you please try this and report any error?
cd C:\Users\datnd14\
git init w558
cd w558
git remote add origin -- https://github.com/zephyrproject-rtos/zephyr.git
git fetch origin --tags -- zephyr-v2.7.0 'refs/heads/*:refs/remotes/origin/*'
del .git\objects\pack\*
here is result when I run that
C:\Users\datnd14\w588>git remote add origin -- https://github.com/zephyrproject-rtos/zephyr.git
C:\Users\datnd14\w588>git fetch origin --tags -- zephyr-v2.7.0 'refs/heads/*:refs/remotes/origin/*'
remote: Enumerating objects: 607149, done.
remote: Counting objects: 100% (2/2), done.
remote: Total 607149 (delta 1), reused 1 (delta 1), pack-reused 607147
Receiving objects: 100% (607149/607149), 417.88 MiB | 596.00 KiB/s, done.
Resolving deltas: 100% (461118/461118), done.
From https://github.com/zephyrproject-rtos/zephyr
* tag zephyr-v2.7.0 -> FETCH_HEAD
* [new tag] v1.0.0 -> v1.0.0
* [new tag] v1.1.0 -> v1.1.0
* [new tag] v1.1.0-rc1 -> v1.1.0-rc1
* [new tag] v1.10.0 -> v1.10.0
* [new tag] v1.10.0-rc1 -> v1.10.0-rc1
* [new tag] v1.10.0-rc2 -> v1.10.0-rc2
* [new tag] v1.10.0-rc3 -> v1.10.0-rc3
[snip]
* [new tag] zephyr-v2.2.0 -> zephyr-v2.2.0
* [new tag] zephyr-v2.2.1 -> zephyr-v2.2.1
* [new tag] zephyr-v2.3.0 -> zephyr-v2.3.0
* [new tag] zephyr-v2.4.0 -> zephyr-v2.4.0
* [new tag] zephyr-v2.5.0 -> zephyr-v2.5.0
* [new tag] zephyr-v2.6.0 -> zephyr-v2.6.0
* [new tag] zephyr-v2.7.0 -> zephyr-v2.7.0
C:\Users\datnd14\w588>del .git\objects\pack\*
C:\Users\datnd14\w588\.git\objects\pack\*, Are you sure (Y/N)? Y
C:\Users\datnd14\w588\.git\objects\pack\pack-9554992bbd0bdd6f346a8acf37ca89822648585f.idx
Access is denied.
C:\Users\datnd14\w588\.git\objects\pack\pack-9554992bbd0bdd6f346a8acf37ca89822648585f.pack
Access is denied.
Great, so now we know you don't have a west problem. I think you have a problem with your git client and/or the way Windows permissions are set up in your home directory. Can you try the same thing with a USB stick or just in a different directory?
It seems normal for git to create read only pack files but somehow this is not a problem for most people. Windows permissions can be very complicated and I'm no expert https://www.ntfs.com/ntfs-permissions-file-effective.htm
yes, I think too and I granted permissions, reinstall git with chocolatey many times. but it can not work
I'm getting the same error invoking west
via cmd.exe
though a VSCode extension.
Full command is:
C:\Windows\System32\cmd.exe /d /c west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git
Here's the output.
> Executing task: west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git <
=== Initializing in C:\Zephyr
--- Cloning manifest repository from https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git
running 'git clone https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git 'C:\Zephyr\.west\manifest-tmp'' in C:\Zephyr
Cloning into 'C:\Zephyr\.west\manifest-tmp'...
remote: Enumerating objects: 899, done.
remote: Counting objects: 100% (899/899), done.
remote: Compressing objects: 100% (521/521), done.
remote: Total 899 (delta 472), reused 750 (delta 327), pack-reused 0
Receiving objects: 100% (899/899), 6.71 MiB | 9.22 MiB/s, done.
Resolving deltas: 100% (472/472), done.
west.manifest: loading C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: group-filter: unset
west.manifest: added project nrf path nrf revision v1.7.1 from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project nrf ('C:\\Zephyr\\nrf') at v1.7.1>: ignored import (True)
west.manifest: added project golioth path modules/lib/golioth revision main from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project golioth ('C:\\Zephyr\\modules\\lib\\golioth') at main>: ignored import ({'name-allowlist': ['qcbor']})
west.manifest: added project pyrinas path pyrinas revision v1.7.x from C:\Zephyr\.west\manifest-tmp\west.yml
west.manifest: project <Project pyrinas ('C:\\Zephyr\\pyrinas') at v1.7.x>: ignored import (True)
west.manifest: loaded C:\Zephyr\.west\manifest-tmp\west.yml
Traceback (most recent call last):
File "C:\Python39\lib\shutil.py", line 803, in move
os.rename(src, real_dst)
PermissionError: [WinError 5] Access is denied: 'C:\\Zephyr\\.west\\manifest-tmp' -> 'C:\\Zephyr\\nfed'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Python39\lib\runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Python39\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "C:\Users\Jared Wolff\.zephyrtools\env\Scripts\west.exe\__main__.py", line 7, in <module>
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 781, in main
app.run(argv or sys.argv[1:])
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 106, in run
self.run_command(argv)
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\main.py", line 336, in run_command
cmd.run(args, unknown, self.topdir, manifest=self.manifest)
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\commands.py", line 140, in run
self.do_run(args, unknown)
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\project.py", line 198, in do_run
topdir = self.bootstrap(args)
File "C:\Users\Jared Wolff\.zephyrtools\env\lib\site-packages\west\app\project.py", line 305, in bootstrap
shutil.move(os.fspath(tempdir), os.fspath(manifest_abspath))
File "C:\Python39\lib\shutil.py", line 815, in move
rmtree(src)
File "C:\Python39\lib\shutil.py", line 737, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python39\lib\shutil.py", line 610, in _rmtree_unsafe
_rmtree_unsafe(fullname, onerror)
File "C:\Python39\lib\shutil.py", line 615, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\Python39\lib\shutil.py", line 613, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 5] Access is denied: 'C:\\Zephyr\\.west\\manifest-tmp\\.git\\objects\\pack\\pack-ae179110ddff9c4a119793934f0f44c69e9e0976.idx'
The terminal process "C:\Windows\System32\cmd.exe /d /c west -v -v init -m https://github.com/circuitdojo/nrf9160-feather-examples-and-drivers.git" terminated with exit code: 1.
Terminal will be reused by tasks, press any key to close it.
Quite bizarre since west
creates the files that it's copying to. Seems to be mostly related to the os.unlink
function. I'm starting from a completely clean directory.
For context, I was using git
to clone before but west init
had an added benefit to initialize the application repo as the correct name as corresponding to self.path
in west.yml
Diff of the environment. Works in a stand alone cmd but not within VSCode
1d0
< !::=::\
3d1
< !ExitCode=00000000
9a8
> CHROME_CRASHPAD_PIPE_NAME=\\.\pipe\crashpad_9232_XCMYRGDVZKPZBDFN
24a24
> ORIGINAL_XDG_CURRENT_DESKTOP=undefined
26c26,27
< PATH=/cygdrive/c/Python39/Scripts:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/usr/bin:/cygdrive/c/Windows/System32/OpenSSH:/cygdrive/c/Program Files/Wireshark:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/ProgramData/chocolatey/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Program Files (x86)/Nordic Semiconductor/nrf-command-line-tools/bin:/cygdrive/c/Program Files (x86)/SEGGER/JLink:/cygdrive/c/Program Files (x86)/ZeroTier/One:/cygdrive/c/Program Files/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn:/cygdrive/c/Program Files (x86)/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/120/DTS/Binn:/cygdrive/c/Program Files (x86)/nodejs:/cygdrive/c/Users/Jared Wolff/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/Jared Wolff/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Users/Jared Wolff/AppData/Roaming/npm
---
> PATH=/cygdrive/c/Python39/Scripts:/cygdrive/c/Windows/system32:/cygdrive/c/Windows:/cygdrive/c/Windows/System32/Wbem:/cygdrive/c/Windows/System32/WindowsPowerShell/v1.0:/usr/bin:/cygdrive/c/Windows/System32/OpenSSH:/cygdrive/c/Program Files/Wireshark:/cygdrive/c/Program Files/PuTTY:/cygdrive/c/ProgramData/chocolatey/bin:/cygdrive/c/Program Files/Git/cmd:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Program Files (x86)/Nordic Semiconductor/nrf-command-line-tools/bin:/cygdrive/c/Program Files (x86)/SEGGER/JLink:/cygdrive/c/Program Files (x86)/ZeroTier/One:/cygdrive/c/Program Files/Microsoft SQL Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/Client SDK/ODBC/110/Tools/Binn:/cygdrive/c/Program Files (x86)/Microsoft SQL
> Server/120/Tools/Binn:/cygdrive/c/Program Files/Microsoft SQL Server/120/DTS/Binn:/cygdrive/c/Program Files (x86)/nodejs:/cygdrive/c/Users/Jared Wolff/AppData/Local/Microsoft/WindowsApps:/cygdrive/c/Users/Jared Wolff/AppData/Local/Programs/Microsoft VS Code/bin:/cygdrive/c/bin:/cygdrive/c/Python39:/cygdrive/c/Users/Jared Wolff/AppData/Roaming/npm
49a51,59
> TERM_PROGRAM=vscode
> TERM_PROGRAM_VERSION=1.63.2
> LANG=en_US.UTF-8
> COLORTERM=truecolor
> VSCODE_GIT_IPC_HANDLE=\\.\pipe\vscode-git-35da2a6a1e-sock
> GIT_ASKPASS=c:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass.sh
> VSCODE_GIT_ASKPASS_NODE=C:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\Code.exe
> VSCODE_GIT_ASKPASS_EXTRA_ARGS=--ms-enable-electron-run-as-node
> VSCODE_GIT_ASKPASS_MAIN=c:\Users\Jared Wolff\AppData\Local\Programs\Microsoft VS Code\resources\app\extensions\git\dist\askpass-main.js
Cygwin looks like the culprit
Looks like it's because I'm running west
in a virtual environment. Without the virtual environment, works ok. With the path to the virtual environment set, it breaks.
The plot thickens.
After lots of testing. I've narrowed it down to whether or not VSCode has the folder open. It appears to be scanning the files inside the .west
folder. When west
is ready to transfer the repo to it's final resting place, it's prevented from doing so because VSCode is indexing the files.
When I close VSCode and run it in a terminal with same environment, it runs perfectly fine.
Hope this helps people that may come across the same problem!
After lots of testing. I've narrowed it down to whether or not VSCode has the folder open. It appears to be scanning the files inside the
.west
folder. Whenwest
is ready to transfer the repo to it's final resting place, it's prevented from doing so because VSCode is indexing the files.
@wbober @trond-snekvik I was thinking about the VS code extension and I wanted to let you know about this to see if you've ever observed random issues like this.
Some of the above discussion looks like random permissions problems that unfortunately I have no idea how to triage right now, but this particular issue reported by @jaredwolff might be something you are running into more often.
it's prevented from doing so because VSCode is indexing the files.
It would help if Windows could delete files in use like Linux filesystems can (it would also save a lot of Windows reboots, I digress)
However similar issues could happen on Linux too: a few years ago, it came as a big surprise to me that some operations like "git diff" appear to be read-only but in fact they have some side effects because they can cause git to update some of its internal data: https://public-inbox.org/git/20160331140515.GA31116@sigill.intra.peff.net/ So in theory we could have a similar race where some Linux IDE triggers some not-so-read-only git commands on a half-moved repo and ends up corrupting it. Very unlikely but possible?
So, @mbolivar-nordic should west switch to some OS-dependent and "standard" /tmp locations to escape this sort of concurrency? Even if the interactions with IDEs ends up being harmless, it's not "nice" to confuse them.
So, @mbolivar-nordic should west switch to some OS-dependent and "standard" /tmp locations to escape this sort of concurrency? Even if the interactions with IDEs ends up being harmless, it's not "nice" to confuse them.
I thought about that, but then you're inviting issues with cross-filesystem moves, which are a can of worms of their own. If you have a strategy for accomplishing this ask, I'm open to ideas, though.
but then you're inviting issues with cross-filesystem moves,
OMG, I referenced something from 2016 but now you're bringing back memories from 2010!! https://bugs.eclipse.org/bugs/show_bug.cgi?id=319233 "Cloning repository fails if destination is on network drive because File.renameTo doesn't succeed"
How about zephyr/.manifest-tmp
instead of zephyr/manifest-tmp
? Or some "hidden" Windows file attribute? Both? @jaredwolff would you be able to test these ~on Windows~ with VS code?
My work around with this, from what I remember, was disabling the Git functionality in VSCode temporarily while west
did its thing. It was trying to cache or open the repository in the manifest-tmp
folder and thus locking up the files. Next time I'm working on my VSCode plugin I'll have to see if changing the manifest-tmp directory name fixes it.
How about
zephyr/.manifest-tmp
instead ofzephyr/manifest-tmp
? Or some "hidden" Windows file attribute? Both?
I don't know what you mean, sorry. The temporary manifest repository already goes under .west
, not zephyr
. I don't know a lot about hidden files on windows or what vs code does. If any body has any resources they can point me at, I'd be grateful.
The temporary manifest repository already goes under .west, not zephyr
Sorry I blindly trusted the description and title at the very top. At the moment they are:
west init completes but leaves everything in zephyr/manifest-tmp
but running west init on windows occasionally leaves everything in the zephyr/manifest-tmp folder upon "successful" completion
In a totally unrelated discussion I just heard that VS2022 is automatically generating some c/Users/me/.gitconfig
file. Sounds nasty, what else does it do?
I just rebuilt a new machine and this is still happening. VSCode was not running at any point. Running west -v -v init
results in:
PS C:\Users\Jeff\Source\Repos\Test> west -v -v init
=== Initializing in C:\Users\Jeff\Source\Repos\Test
--- Cloning manifest repository from https://github.com/zephyrproject-rtos/zephyr
running 'git clone https://github.com/zephyrproject-rtos/zephyr 'C:\Users\Jeff\Source\Repos\Test\.west\manifest-tmp'' in C:\Users\Jeff\Source\Repos\Test
Cloning into 'C:\Users\Jeff\Source\Repos\Test\.west\manifest-tmp'...
remote: Enumerating objects: 765677, done.
remote: Counting objects: 100% (2/2), done.
remote: Total 765677 (delta 1), reused 1 (delta 1), pack-reused 765675
Receiving objects: 100% (765677/765677), 534.13 MiB | 931.00 KiB/s, done.
Resolving deltas: 100% (582305/582305), done.
Updating files: 100% (21810/21810), done.
west.manifest: loading data (no file)
west.manifest: resolving self import submanifests
west.manifest: ignored self import
west.manifest: resolved self import
west.manifest: group-filter: ['-ci']
west.manifest: added project canopennode path modules/lib/canopennode revision 53d3415c14d60f8f4bfca54bfbc5d5a667d7e724
west.manifest: added project chre path modules/lib/chre revision ef76d3456db07e4959df555047d6962279528c8d
west.manifest: added project cmsis path modules/hal/cmsis revision 093de61c2a7d12dc9253daf8692f61f793a9254a groups ['hal']
west.manifest: added project edtt path tools/edtt revision 1ea61a390d2bfcf3b2ecdba8f8b0b98dfdffbd11 groups ['tools']
west.manifest: added project fatfs path modules/fs/fatfs revision a30531af3a95a9a3ea7d771ea8a578ebfed45514 groups ['fs']
west.manifest: added project fff path modules/lib/fff revision 6ce5ba26486e93d5b7696a3e23f0585932c14b16 groups ['ci']
west.manifest: added project hal_altera path modules/hal/altera revision 0d225ddd314379b32355a00fb669eacf911e750d groups ['hal']
west.manifest: added project hal_atmel path modules/hal/atmel revision 1d237f2e2f262751975b6da6e03af569b2b49b2b groups ['hal']
west.manifest: added project hal_espressif path modules/hal/espressif revision 1e29548e4492807eb18cbaa77d3e78801cc7d07e groups ['hal']
west.manifest: added project hal_gigadevice path modules/hal/gigadevice revision dd0e0322474462b58059e6fedaf1d67d2a0864d0 groups ['hal']
west.manifest: added project hal_infineon path modules/hal/infineon revision 4af06965f57ba1e7d170e6a97d24c33785543a8c groups ['hal']
west.manifest: added project hal_microchip path modules/hal/microchip revision 5d079f1683a00b801373bbbbf5d181d4e33b30d5 groups ['hal']
west.manifest: added project hal_nordic path modules/hal/nordic revision 249199ec5a5c31d170659921048764e96d05cc0e groups ['hal']
west.manifest: added project hal_nuvoton path modules/hal/nuvoton revision b4d31f33238713a568e23618845702fadd67386f groups ['hal']
west.manifest: added project hal_nxp path modules/hal/nxp revision 3516aa5f722eb70b969b3bc57bcf9389ad901057 groups ['hal']
west.manifest: added project hal_openisa path modules/hal/openisa revision 40d049f69c50b58ea20473bee14cf93f518bf262 groups ['hal']
west.manifest: added project hal_quicklogic path modules/hal/quicklogic revision b3a66fe6d04d87fd1533a5c8de51d0599fcd08d0 groups ['hal']
west.manifest: added project hal_renesas path modules/hal/renesas revision 468d3f2146d18c7f86a4640fc641cc1d20a4a100 groups ['hal']
west.manifest: added project hal_rpi_pico path modules/hal/rpi_pico revision 191f5ba46fda49523cdaaef27583d1c875ba2c36 groups ['hal']
west.manifest: added project hal_silabs path modules/hal/silabs revision 1ec8dd99aa4ac3e8632d2aa28a7438049bb27102 groups ['hal']
west.manifest: added project hal_st path modules/hal/st revision cccbc24c14decfd3f93959f7b14514536af973c7 groups ['hal']
west.manifest: added project hal_stm32 path modules/hal/stm32 revision 5b1610c3cf46589d50e2634ba240b7b38bc59c53 groups ['hal']
west.manifest: added project hal_telink path modules/hal/telink revision 38573af589173259801ae6c2b34b7d4c9e626746 groups ['hal']
west.manifest: added project hal_ti path modules/hal/ti revision 29af528900d03af02a17b4b7f4fc93f19a81da2a groups ['hal']
west.manifest: added project hal_wurthelektronik path modules/hal/wurthelektronik revision 24ca9873c3d608fad1fea0431836bc8f144c132e groups ['hal']
west.manifest: added project hal_xtensa path modules/hal/xtensa revision 63f655362423aa49507da7977a2d37142e8debeb groups ['hal']
west.manifest: added project libmetal path modules/hal/libmetal revision 2f586b4f1276fb075ee145421bdf6cbe5403aa41 groups ['hal']
west.manifest: added project liblc3 path modules/lib/liblc3 revision 448f3de31f49a838988a162ef1e23a89ddf2d2ed
west.manifest: added project littlefs path modules/fs/littlefs revision 652f2c5646e79b881e6f3099686ad3b7af9e216c groups ['fs']
west.manifest: added project loramac-node path modules/lib/loramac-node revision 0257b50905695192d095667b1c3abb80346db1a1
west.manifest: added project lvgl path modules/lib/gui/lvgl revision af95bdfcf6784edd958ea08139c713e2d3dee7af
west.manifest: added project lz4 path modules/lib/lz4 revision 8e303c264fc21c2116dc612658003a22e933124d
west.manifest: added project mbedtls path modules/crypto/mbedtls revision 7fed49c9b9f983ad6416986661ef637459723bcb groups ['crypto']
west.manifest: added project mcuboot path bootloader/mcuboot revision 21e56a1bd033af263e808940779a1adaf4d5465e
west.manifest: added project mipi-sys-t path modules/debug/mipi-sys-t revision 0d521d8055f3b2b4842f728b0365d3f0ece9c37f groups ['debug']
west.manifest: added project nanopb path modules/lib/nanopb revision dc4deed54fd4c7e1935e3b6387eedf21bb45dc38
west.manifest: added project net-tools path tools/net-tools revision e0828aa9629b533644dc96ff6d1295c939bd713c groups ['tools']
west.manifest: added project nrf_hw_models path modules/bsim_hw_models/nrf_hw_models revision 65bc5305d432c08e24a3f343006d1e7deaff4908
west.manifest: added project open-amp path modules/lib/open-amp revision 8d53544871e1f300c478224faca6be8384ab0d04
west.manifest: added project openthread path modules/lib/openthread revision b21e99b4b3d823f71c902b9174ff62b964c124f0
west.manifest: added project picolibc path modules/lib/picolibc revision 04ada5951cbaf8e7b17f8226ce31cb6837c28ba7
west.manifest: added project segger path modules/debug/segger revision d4e568a920b4bd087886170a5624c167b2d0665e groups ['debug']
west.manifest: added project sof path modules/audio/sof revision fed466c264ad078c66f7bff9218ba1d3fa0eb201
west.manifest: added project tflite-micro path modules/lib/tflite-micro revision 9156d050927012da87079064db59d07f03b8baf6
west.manifest: added project tinycbor path modules/lib/tinycbor revision 9e1f34bc08123aaad7666d3652aaa839e8178b3b
west.manifest: added project tinycrypt path modules/crypto/tinycrypt revision 3e9a49d2672ec01435ffbf0d788db6d95ef28de0 groups ['crypto']
west.manifest: added project TraceRecorderSource path modules/debug/TraceRecorder revision 9893bf1cf649a2c4ee2e27293f887994f3d0da5b groups ['debug']
west.manifest: added project trusted-firmware-m path modules/tee/tf-m/trusted-firmware-m revision 4c984817d861512edaeca27b83308e9b9915a98a groups ['tee']
west.manifest: added project tf-m-tests path modules/tee/tf-m/tf-m-tests revision c99a86b295c4887520da9d8402566d7f225c974e groups ['tee']
west.manifest: added project psa-arch-tests path modules/tee/tf-m/psa-arch-tests revision f4fc2442b8e29e2a03d9899e46e5a3ea3df8c2c9 groups ['tee']
west.manifest: added project uoscore-uedhoc path modules/lib/uoscore-uedhoc revision e8920192b66db4f909eb9cd3f155d5245c1ae825
west.manifest: added project zcbor path modules/lib/zcbor revision a0d6981f14d4001d6f0d608d1a427f9bc6bb6d02
west.manifest: added project zscilib path modules/lib/zscilib revision a54986aa98db4082ac56b582843bb5b5435208a6
west.manifest: loaded data (no file)
FATAL ERROR: Destination path 'C:\Users\Jeff\Source\Repos\Test\zephyr\manifest-tmp' already exists
After running the files once again appear to be in .west/manifest-tmp
and at this point I can't init or update. It would be great if we could get a bug label on this. This is a pristine copy of Windows 11 and I've followed the Zephyr setup instructions to a T. There is obviously an issue of some sort.
Ok I misread the error and it looks like I still had a zephyr/manifest-tmp directory, possibly left over from the "vs code" issue. Still, the whole process seems quite fragile.
If you don't wipe out C:\Users\Jeff\Source\Repos\Test
to actually start over then many things can happen; I wouldn't call that "fragile".
I have a repo at that location so I can't wipe it out. Checking out zephyr to a different project and then running the environment scripts has never worked for me so I have zephyr in my project directly, the same way many projects do.
Understood. Makes sense. You should at least delete .west/
completely:
I had but I missed the zephyr directory, thanks
Had something related to this general issue happen too today. I used west init
and it failed when it tried to link the cloned manifest (using defaults, it freshly cloned). Python threw a permission error when trying to link the manifest directory to the zephyr directory PermissionError: [WinError 5] Access is denied
Unfortunately I lost the log history after executing another command.
Bottom line though is that it worked fine after I deleted the .west
and zephyr
folder and just ran west init
again.
So possible that the issue came from an issue during cloning. The project was definitely "clean" before attempting the first init and had no old files laying around.
I can reliably reproduce this on Linux with the following steps:
mkdir ./zephyr/
west init ./ -m https://github.com/zephyrproject-rtos/zephyr.git
I know, you're not supposed to run west init
in a directory that isn't empty, but it's an easy mistake to make, and no errors are reported... For example, what if you clone Zephyr and then run west init
because you're new to this, or just weren't thinking straight.
I'm reading into it / making some assumptions... but in this case the issue appears to be that the target ./zephyr/
directory isn't empty - so after west init
clones into ./.west/manifest-tmp/
, it performs a "dumb" mv ./.west/manifest-tmp/ ./zephyr/
, which results placing ./.west/manifest-tmp/
into the existing ./zephyr/
directory (final path ./zephyr/manifest-tmp/
), not renaming ./.west/manifest-tmp/
to ./zephyr/
... i.e: no check is performed for ./zephyr/
existing already.
This matches the behavior of /bin/mv
's second mode of invocation, where perhaps the first was assumed:
SYNOPSIS
mv [OPTION]... [-T] SOURCE DEST
mv [OPTION]... SOURCE... DIRECTORY
mv [OPTION]... -t DIRECTORY SOURCE...
Other modules are cloned directly into their final destination, so git
is responsible for checking the state of things.
This behavior is present back to at least v0.12.0.
Understood. Makes sense. You should at least delete .west/ completely:
This isn't enough - as described here, leaving the ./zephyr/
directory will directly cause this issue (though it may not be the only way).
PS: west
uses shutil.move()
to perform this move, with no other checks.
If the destination is an existing directory, then src is moved inside that directory. If the destination already exists but is not a directory, it may be overwritten depending on os.rename() semantics.
I believe that the issue reported by @congnguyenhuu / @jaredwolff / @timonsku is different to the original issue, and is certainly different to mine.
The crux is:
PermissionError: [WinError 5] Access is denied: 'C:\\Users\\congnh\\zephyrproject\\.west\\manifest-tmp' -> 'C:\\Users\\congnh\\zephyrproject\\zephyr'
@jaredwolff may be on to something with this comment:
After lots of testing. I've narrowed it down to whether or not VSCode has the folder open. It appears to be scanning the files inside the .west folder. When west is ready to transfer the repo to it's final resting place, it's prevented from doing so because VSCode is indexing the files.
... if any of you are still experiencing this, please could you open a new ticket to discuss further? (or perhaps this issue should be renamed accordingly, and I should remove the 'Fixes' tag from my PR)
@attie-argentum the issue, in my case, seemed to be caused by the integrated Git instance of VSCode. When I check out and init a repository with my plugin it was wreaking all types of havoc. My fix? I simply turn off the Git pane during init and turn it on afterwords. Problem solved. :)
Very true, though others have reported similar a PermissionError
and claim no VSCode... 🙈
Perhaps this is just "locking on Windows", and it cannot be worked around - VSCode isn't the only thing that may have a peek while west
is busy after all.
I'm going to close this since it seems like many of the issues here have been diagnosed as environment related problems on Windows that are outside of west's control. If someone is still having an issue that they believe can be attributed to west and which matches the symptom in this issue description ('west init completes but leaves everything in zephyr/manifest-tmp'), feel free to reopen. Otherwise, please open new issues for separate problems to keep this tracker clean.
I had this issue today too. Looks like vscode indexed things related to git and that's locked the folders while west init tried to delete it. A workround:
rmdir /s .west\
rmdir /s nrf\manifest-tmp\
west init -m https://github.com/nrfconnect/sdk-nrf --mr v2.5.1
=== Initialized. Now run "west update" inside C:\files\projects\side_projects\setup-nrf
This can't be "fixed" but it looks frequent enough to deserve a warning and recommended workaround in west init -h
. Everyone please review #703 that I just submitted, thanks!
Although it wouldn't fix this in all scenarios, abandoning the manifest-tmp
directory mechanism would resolve this for a lot of users. If the manifest has a self.path
, we'd still need to move the directory after cloning (which could result in the same issue), but I'd argue that most manifest repositories don't have this field. Notably, Zephyr itself doesn't.
I understand that manifest-tmp
offers a cleaner recovery mechanism on repeated calls to west init
, but it also randomly breaks the second step of the getting started procedure on the most popular OS.
It can't be fixed for all scenarios, but it can be fixed for the most common one.
I understand that manifest-tmp offers a cleaner recovery mechanism on repeated calls to west init,
There is worse that "repeated calls": consecutive west
commands believing that west init
was successful when it wasn't. Practically all filesystems guarantee atomicity of renames. This makes rename an extremely useful tool.
BTW the core issue is not just with west
and not just with init
: git can repack and rename and delete files too. Unsurprisingly, git will do whatever it wants whenever it wants with its files.
Most popular or not, users of a filesystem that does not support concurrency should learn to avoid concurrency. This is good practice and important education that's valid way beyond west and VS code. Maybe the newer ReFS is better? I tried to find information about it but the name is unfortunately unsearchable.
If anything, VS Code and other IDEs should be aware of the limitations of the filesystem they're using and refrain from interfering with random files they don't own. Avoiding background use of files in .git/
and .west/
would be good. They could also pause when they don't have focus; it wouldn't be perfect but it would avoid some failures.
I have no idea what causes this or how to reliably reproduce it, but running
west init
on windows occasionally leaves everything in thezephyr/manifest-tmp
folder upon "successful" completion.west update
subsequently fails because everything is in the wrong place.