Error on re-provision: chmod on /srv/www/my_site/.git/config.lock failed

[cagross]

Expected Behavior

I am trying to add a new site to VVV.

Current Behavior

To add my new site, I first added the following entry to vvv-custom.yml:

  feedxl:
    repo: https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    hosts:
      - feedxl.test

I then executed vagrant up --provision. During that process, when it came time to bring up my new site, it failed with the error:

    default: Downloading feedxl, see https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    default: error: chmod on /srv/www/feedxl/.git/config.lock failed: Operation not permitted
    default: fatal: could not set 'core.filemode' to 'false'
    default: Error: The /srv/www/feedxl folder does not exist, and no repo was specified in the config file, there is nothing to provision for this site!

Possible Solution

I tried inspecting my VVV root directory. Inside that directory, the www folder was marked 'read only.' So I removed that designation. But the error still remained.

Steps to Reproduce (for bugs)

1. Add new site code to vvv-custom.yml.
2. Run vagrant up --provision.

Here is a link to more lines of the output. FYI it is not the full output. I can link that if you want.

Your Environment

• VVV version: 2.6.0-4-g678d59b
• VVV Git Branch: develop
• Vagrant version: 2.2.4
• VM Provider name: VirtualBox
• VM Provider version: Version 6.0.6 r130049 (Qt5.6.2)
• Operating System and version: Win 10, x64

[ghost]

have u tried using vagrant reload --provision instead of vagrant up --provision. when u reload, it shutdowns the vm and restarts so that it can properly create the site of your choosing.

[cagross]

@benlumia007 Yes, I have tried that as well--resulting in the same error. Sorry for not including that in my ticket.

[tomjn]

Those errors are from git, failing to write/modify the filesystem, and this is the first step where it encounters that problem:

default: error: chmod on /srv/www/feedxl/.git/config.lock failed: Operation not permitted

That's the root issue, i don't know how your VVV folder became read only, but you might want to double check the permissions of the subfolders and files, setting the VVV folders permissions doesn't mean the www folder is writable etc.

[cagross]

@tomjn OK thanks for the reply. But can you clarify how I can/should go about troubleshooting this?

I checked a few of my existing sites in my vvv/www directory. Each of those is of course in its own folder. Each of those seems to be set to 'read only' (screenshots).

setting the VVV folders permissions doesn't mean the www folder is writable etc.

As a test, in the vvv/www directory, I manually created a new folder. This folder was also created as 'read only' (screenshot, but also seemed to have full read/write permissions (screenshot).

Thoughts?

[tomjn]

There's nothing in VVV that would do this, so I have no clues as to why all those folders are read only, just that they must not be read only. This will be caused by an external factor, something on your machine, or a setting changed by yourself or other software. This isn't something that can be fixed by configuring VVV or changing VVV.

As for how you would recursively go through every folder and unset the read only option, I don't know how to do this on Windows

[tomjn]

This PR tries to improve the messaging by detecting when git clone fails:

https://github.com/Varying-Vagrant-Vagrants/VVV/pull/1765

It won't fix your problem, it just makes it clearer what's happened

[cagross]

There's nothing in VVV that would do this, so I have no clues as to why all those folders are read only,

Right. But are these folders actually read-only? I ask because from the error message, it looks like VVV is able to successfully create a folder (feedxl), then another folder inside (.git). It then fails to chmod on a file within that folder (config.lock).

Furthermore, in the www folder, I can manually create the feedxl folder. Then I can run a git init to successfully create a .git folder (screenshots). If these folders were actually read-only, would I be able to do this?

In the .git folder is a file: conifg This file appears to be fully writeable (screenshots). I realize that's not the same file referenced in the error message.

This PR tries to improve the messaging by detecting when git clone fails: It won't fix your problem, it just makes it clearer what's happened

OK thanks. So how would I implement this new messaging feature on my local machine? Would I simply git pull, then vagrant up --provision?

[tomjn]

It wouldn't tell you anything we don't already know, but you would have to switch branch from develop, I wouldn't recommend it as that branch is interested and doesn't change VVVs behaviour

Are you sure fit actually creates the folder? Just because you the active logged in user can create a folder manually doesn't mean the user virtual box is running under can do it. Fundamentally all I can do is make VVV detect this situation and tell you about it, but it wouldn't tell you anything more than what I've said here.

There's only so much that can be done from the guest VM, the solution to your issue is on the host machine in Windows. Perhaps the ownership of those files has changed? I don't know. What would be perfect for you now to debug this is a Windows IT Administrator

On Thu, 25 Apr 2019 at 11:59, cagross notifications@github.com wrote:

There's nothing in VVV that would do this, so I have no clues as to why all those folders are read only,

Right. But are these folders actually read-only? I ask because from the error message, it looks like VVV is able to successfully create a folder ( feedxl), then another folder inside (.git). It then fails to chmod on a file within that folder (config.lock).

Furthermore, in the www folder, I can manually create the feedxl folder. Then I can run a git init to successfully create a .git folder ( screenshots https://imgur.com/a/K2gPyMU). If these folders were actually read-only, would I be able to do this?

In the .git folder is a file: conifg This file appears to be fully writeable (screenshots https://imgur.com/a/XpRFOAm). I realize that's not the same file referenced in the error message.

This PR tries to improve the messaging by detecting when git clone fails: It won't fix your problem, it just makes it clearer what's happened

OK thanks. So how would I implement this new messaging feature on my local machine? Would I simply git pull, then vagrant up --provision?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-486626087, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZ4FTPGTUIX2B6WCVNTPSGFJPANCNFSM4HIJ5F6Q .

[cagross]

Just because you the active logged in user can create a folder manually doesn't mean the user virtual box is running under can do it.

OK right, I get that. So in light of that, how can I ensure the virtual box can create folders?

the solution to your issue is on the host machine in Windows.

OK I understand that. But my issue is with troubleshooting. I don't know what I'm looking for, nor do I know what to change once I find what I'm looking for :-/

Perhaps the ownership of those files has changed?

OK. I know how to check the ownership of a file in Windows. But which file(s) should I check? And which owner does each file need to have?

[tomjn]

At this point my knowledge of Windows is stretched thin, Id expect you to be the owner and the folders to have similar ownership and group stats as things in your my documents folder.

As for unchecking read only on all subfolders and files, I don't know, I found this on how to do it with the command prompt https://superuser.com/questions/653951/how-to-remove-read-only-attribute-recursively-on-windows-7

There is one longshot, does doing it from within the VM work? A vagrant SSH and a chmod command on a folder? If so that would be great news :) chmod 777 folder name

Any ideas how those folders became set to read only in the first place?

On Thu, 25 Apr 2019 at 12:25, cagross notifications@github.com wrote:

Just because you the active logged in user can create a folder manually doesn't mean the user virtual box is running under can do it.

OK right, I get that. So in light of that, how can I ensure the virtual box can create folders?

the solution to your issue is on the host machine in Windows.

OK I understand that. But my issue is with troubleshooting. I don't know what I'm looking for, nor do I know what to change once I find what I'm looking for :-/

Perhaps the ownership of those files has changed?

OK. I know how to check the ownership of a file in Windows. But which file(s) should I check? And which owner does each file need to have?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-486633247, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZYQSXS33HTJEHWSQLTPSGIJXANCNFSM4HIJ5F6Q .

[cagross]

Also, I this may be of note. I installed Local by Flywheel, and was able to create a new WordPress site without issue. I believe it is also using the same VirtualBox software. The Local by Flywheel site was not created in the same www folder we have been discussing. The folder used by Local by Flywheel is also marked 'Read Only' by Windows, but also, appears to have full read/write permissions (just as the vvv/www folder. Not sure if that means much, but just thought I'd throw it out there.

[cagross]

As for unchecking read only on all subfolders and files, I don't know, I found this on how to do it with the command prompt https://superuser.com/questions/653951/how-to-remove-read-only-attribute-recursively-on-windows-7

I tried unchecking 'read only' for the www folder. It processed that request for a few minutes. When it reached 100%, it seemed to complete without issue. But when I then checked the folder properties again, it was still marked as 'read only.' I tried that twice, with the same result.

There is one longshot, does doing it from within the VM work? A vagrant SSH and a chmod command on a folder? If so that would be great news :) chmod 777 folder name

Stand by while I check.

Any ideas how those folders became set to read only in the first place?

No idea. But even though Windows marks them as 'read only,' are they? What exactly does 'read only' mean? If I can write new files to the folder, is it still 'read only?'

Super User

How to remove read-only attribute recursively on Windows 7

I need to remove read-only attributes of all files under a directory recursively on Windows 7 using command line. Could you please provide an example on this?

[cagross]

There is one longshot, does doing it from within the VM work? A vagrant SSH and a chmod command on a folder? If so that would be great news :) chmod 777 folder name

OK I tried that. As expected, it returned:

chmod: changing permissions of ‘test’: Operation not permitted

That's good--at least we can easily re-create/test the issue now.

[tomjn]

hmmm looking back at your first screenshot, the read only box isn't checked, it's a square in a square so it's indeterminate. I wonder if that means some but not all is read only, or if there's more to it than that?

I've done a bit more digging based on that, and it seems in VirtualBox you can't chmod something in a mounted folder. Which makes me wonder why git feels the need to do it in the first place. This is the first time we've encountered this

As for Local by FlyWheel, I did a little more digging, and yes/no. It uses VirtualBox, but the VM is Boot2Docker, with docker containers internally. It also doesn't use Vagrant, so it's communicating directly with VirtualBox to do its work. I also found a handy snippet that would help with our root certificates in a shell script in Locals app bundle security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain "$*".

I haven't figured out how mounted folders work in Local, but my guess would be it does it via rsync. We have that option as well, but it has tradeoffs, particularly since it means there would be 2 copies of every file, one inside the VM and one on the host

[tomjn]

@cagross I don't know if this will help, but can you run git config --global core.fileMode false inside the VM then try to add a new site and see if this helps?

[cagross]

hmmm looking back at your first screenshot, the read only box isn't checked, it's a square in a square so it's indeterminate.

OK good catch--you're correct on that. It's not a full fledged check mark.

I've done a bit more digging based on that, and it seems in VirtualBox you can't chmod something in a mounted folder. Which makes me wonder why git feels the need to do it in the first place. This is the first time we've encountered this

I can say that I successfully created a new VVV site as recently as ~3 weeks ago. At that time, there were no issues. I can also say that just prior to today's attempt, I updated the VirtualBox software--can't remember the previous version :-/ I also updated VVV to the latest version (also can't remember the previous version I was running). Not sure if that matters.

I don't know if this will help, but can you run git config --global core.fileMode false inside the VM

OK. To be clear, I would vagrant ssh then run this command? Do I need to ensure I run it from any particular directory?

then try to add a new site and see if this helps?

To be clear, I should execute a vagrant reload --provision, just as I've been trying so far?

[tomjn]

OK. To be clear, I would vagrant ssh then run this command? Do I need to ensure I run it from any particular directory?

That's right, vagrant ssh, run the command, and exit, then vagrant reload --provision. It shouldn't matter where you run it inside the VM as it's got --global

[cagross]

OK I did so. The git command completed without any output. The vagrant reload --provision again failed at the same step, but this time with more output:

==> default: Running provisioner: site-feedxl (shell)...
    default: Running: C:/Users/snarl/AppData/Local/Temp/vagrant-shell20190425-9480-ctl93d.sh
    default: Updating feedxl...
    default: fatal: ambiguous argument 'origin/master': unknown revision or path not in the working tree.
    default: Use '--' to separate paths from revisions, like this:
    default: 'git <command> [<revision>...] -- [<file>...]'
    default: fatal: 'origin' does not appear to be a git repository
    default: fatal: Could not read from remote repository.
    default:
    default: Please make sure you have the correct access rights
    default: and the repository exists.
    default: error: pathspec 'master' did not match any file(s) known to git
    default: Warning: A site provisioner was not found at .vvv/vvv-init.conf provision/vvv-init.conf or vvv-init.conf, searching 3 folders down, please be patient...
    default: Warning: No site provisioner was found, VVV could not perform any scripted setup that might install software for this site
    default: Warning: An nginx config was not found at .vvv/vvv-nginx.conf provision/vvv-nginx.conf or vvv-nginx.conf, searching 3 folders down, please be patient...
    default: Warning: No nginx config was found, VVV will not know how to serve this site
    default: Adding domains to the virtual machine's /etc/hosts file...
    default: Adding hosts from the VVV config entry
    default:  * Added feedxl.test from /vagrant/vvv-custom.yml
    default:  * Restarting nginx nginx
    default:    ...done.

[jjsanderson]

I'm trying to get VVV set up from a clean start, on current OS X. With no changes to vvv-config.yml I'm seeing similar errors and no directories are being created in /www in the provision process. I realise much of this thread is now Windows-specific, but the error logs I'm seeing echo those in the original post:

default: Downloading wordpress-one, see https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
default: error: chmod on /srv/www/wordpress-one/.git/config.lock failed: Operation not permitted
default: fatal: could not set 'core.filemode' to 'false'
default: Error: The /srv/www/wordpress-one folder does not exist, and no repo was specified in the config file, there is nothing to provision for this site!

[tomjn]

I would remove the feedxl folder before reprovisioning as it has an incomplete checkout, it doesn't surprise me that it failed to provision

[cagross]

@tomjn Right, of course--my mistake. I'll need a couple hours to re-test.

[cagross]

@tomjn OK I removed the existing feedxl folder and tried the reprovision. But encountered the exact same error :-/

[davidwolfpaw]

Just a +1 that I'm having the same issue on a fresh VVV install on OSX setting up a new laptop.

[ghost]

what if maybe, is a git version issue. if the box has an newer version, and u may ahve an older version of git. im just thinkin out out. or vice versa... .

[jjsanderson]

Ooh, interesting thought…

OK, I've updated git to 2.21 (from 2.20): provisioning still fails with directory creation. Updated the vagrant box to yesterday's release: same issue.

The first 'No such file or directory' error I'm seeing is in the Tideways install sequence, then when provisioning site-wordpress-one and site-wordpress-two.

[tomjn]

I think my next step is trying to reproduce on a clean install, Ive just updated Virtual box locally

Thanks for running through steps with us @cagross it's much appreciated, we've made a lot of messaging and robustness tweaks the last few months thanks to some of your issues that have improved things, glad to see you're still on board :)

On Fri, 26 Apr 2019 at 09:22, Jonathan Sanderson notifications@github.com wrote:

Ooh, interesting thought…

OK, I've updated git to 2.21 (from 2.20): provisioning still fails with directory creation. Updated the vagrant box to yesterday's release: same issue.

The first 'No such file or directory' error I'm seeing is in the Tideways install sequence, then when provisioning site-wordpress-one and site-wordpress-two.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-486972116, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZ76AI5LOV65QJHMXY3PSK3THANCNFSM4HIJ5F6Q .

[jjsanderson]

Thanks, @tomjn. Let me know if I can help diagnose and/or test.

[cagross]

@tomjn Thanks for the help. Keep us posted.

[tomjn]

https://github.com/Varying-Vagrant-Vagrants/VVV/pull/1765 has been tested and merged, when this issue happens I would expect it to abort provisioning and be a little clearer about it rather than attempting to continue provisioning the site

[tomjn]

@jjsanderson can you get me a copy of your provision output? A gist of the entire output from when you provisioned would be super handy. Also are you on Windows? The info in the VVV splash would be handy

[tomjn]

Also, can you both confirm you have more than 8GB of RAM, and sufficient free disk space?

[tomjn]

Also, when a git clone fails, it aborts provisioning, in your opinion, should it also try to remove the folder if it exists? After all it only clones if no folder exists, therefore any folder that does exist must be a broken or partial git checkout /cc @Mte90

[jjsanderson]

@tomjin: hurray for scrollback buffers… https://gist.github.com/jjsanderson/4436933e79c04c024414aebf9c129abc

I'm on OS X (10.14.3), 32Gb RAM iMac, >100Gb available disk space.

Gist

VVV provision output (failure) 2019-04-26

VVV provision output (failure) 2019-04-26. GitHub Gist: instantly share code, notes, and snippets.

[ghost]

so what was the issue? im confused.

[ghost]

i think its the new box, i think try downgrade to the previous box for trusty64.

[ghost]

so after doing some test, if you vagrant ssh to the sites folder to the new creted folder which a provision folder should be there, no mater what kind of repo it is. it will keep saying that could not set core.filemode to false. but if you do it on the host machine and get lcone, tis fine.

[tomjn]

A box update and a reprovision didn't replicate the problem here

@benlumia007 the problem is that when you add a new site, the core provisioner it fails to clone the site template when git tries to chmod a file

[tomjn]

Possible cause: https://stackoverflow.com/questions/50108363/git-init-fatal-could-not-set-core-filemode-to-false

Doesn't seem likely though

Stack Overflow

Git init: fatal: could not set 'core.filemode' to 'false'

Using Team City to check out from a Git Repo. (Gitlabs if it matters) Start with Empty build directory. Get this error: fatal: could not set 'core.filemode' to 'false' (Running on a Windows mac...

[ghost]

this is a strange one because, i ahve vvv and my own custom build, when i use my old box, everything is fine, but if i use my new box, i get the same thing. both projects have their own box so is separate.

[tomjn]

@cagross @jjsanderson some debugging steps:

1. switch to the develop branch
2. SSH into the VM with vagrant ssh
3. Run 2 git clones, I want to know the result of each: a. Clone into a VM folder git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /tmp/test b. Clone into a network share folder git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /srv/www/test

Do both work? Bonus points for the output. You should be able to run those in any folder as the paths are all absolute

[ghost]

@cagross and @jjsanderson

are you guys both on Windows 10.

[tomjn]

JJSAnderson is on MacOS, they haven't confirmed if they are or aren't running antivirus though

[ghost]

my first thought is .vagrant.d and .virtualbox in the home directory. i wonder if u delete this and it should reconfigure everything, no sure if that will help.

[jjsanderson]

Apologies - end of the working week in my timezone, and a hungry 3 year-old to wrangle.

I don't have access to my work system over the weekend, so I've just tried to recreate the problem on my personal laptop. That is: clean clone from github, then vagrant up --provision.

...aaaaaand provisioning succeeds, everything seems to work.

Same versions of VirtualBox, Vagrant, ubuntu/trusty64 20190424.0.0, and so on. The only disparity I can spot: my work iMac is stuck on OS X 10.14.3 (thanks IT admins!), home MacBook is on 10.14.4.

I'm stuck as to how to help until Monday. If I can't even replicate my own issue…

(somebody asked about antivirus? The system with the problem is indeed running the ghastly Microsoft SCEP package. It doesn't seem to have caused any problems with VVV specifically for the last four years. Home system has no anti-virus installed.)

[cagross]

@tomjn

Also, can you both confirm you have more than 8GB of RAM, and sufficient free disk space?

I don't have more than 8GB of RAM--I have 8 GB on the nose. On this particular hard drive, I have ~25 GB free out of ~125 GB.

I tried to clone the two repos. The first succeeded, and the second failed with the same error. Full output of both here.

they haven't confirmed if they are or aren't running antivirus though

I am running Windows Defender for antivirus. I have not tried disabling it yet.

[tomjn]

So the git clone to /tmp succeeded? What happens if you try to move that git clone into a mounted/shared folder? Does that work? If so does a git pull also work?

On Sat, 27 Apr 2019 at 14:07, cagross notifications@github.com wrote:

@tomjn https://github.com/tomjn

Also, can you both confirm you have more than 8GB of RAM, and sufficient free disk space?

I don't have more than 8GB of RAM--I have 8 GB on the nose. On this particular hard drive, I have ~25 GB free out of ~125 GB.

I tried to clone the two repos. The first succeeded, and the second failed with the same error. Full output of both here https://pastebin.com/T91FZVkQ.

they haven't confirmed if they are or aren't running antivirus though

I am running Windows Defender for antivirus. I have not tried disabling it yet.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487284811, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZZJUPAMUS62OLR42RTPSRFZRANCNFSM4HIJ5F6Q .

[cagross]

What happens if you try to move that git clone into a mounted/shared folder? Does that work?

I can try that--but how? How do I access the /tmp folder, then copy/move the git clone?

[tomjn]

Inside the VM:

mv from to

I'm on a mobile so my ability to give instructions is super constrained but a quick Google should tell you how

On Sat, 27 Apr 2019 at 14:13, cagross notifications@github.com wrote:

What happens if you try to move that git clone into a mounted/shared folder? Does that work?

I can try that--but how? How do I access the /tmp folder, then copy/move the git clone?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487285209, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZY3WWJD2FQ4U4QGHDTPSRGPZANCNFSM4HIJ5F6Q .

[tomjn]

Basically take the folder git created in the tmp folder, and move it to /srv/www which is a mounted shared folder

On Sat, 27 Apr 2019 at 14:14, Tom Nowell contact@tomjn.com wrote:

Inside the VM:

mv from to

I'm on a mobile so my ability to give instructions is super constrained but a quick Google should tell you how

On Sat, 27 Apr 2019 at 14:13, cagross notifications@github.com wrote:

What happens if you try to move that git clone into a mounted/shared folder? Does that work?

I can try that--but how? How do I access the /tmp folder, then copy/move the git clone?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487285209, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZY3WWJD2FQ4U4QGHDTPSRGPZANCNFSM4HIJ5F6Q .

[cagross]

OK I believe I did it. The full output from my mv test /srv/www is here. Despite those errors, the directory--and the files within--appear to have moved to the expected location (screenshot). Did I do that correctly?