Error on re-provision: chmod on /srv/www/my_site/.git/config.lock failed

cagross commented 5 years ago

Expected Behavior

I am trying to add a new site to VVV.

Current Behavior

To add my new site, I first added the following entry to vvv-custom.yml:

  feedxl:
    repo: https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    hosts:
      - feedxl.test

I then executed vagrant up --provision. During that process, when it came time to bring up my new site, it failed with the error:

    default: Downloading feedxl, see https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    default: error: chmod on /srv/www/feedxl/.git/config.lock failed: Operation not permitted
    default: fatal: could not set 'core.filemode' to 'false'
    default: Error: The /srv/www/feedxl folder does not exist, and no repo was specified in the config file, there is nothing to provision for this site!

Possible Solution

I tried inspecting my VVV root directory. Inside that directory, the www folder was marked 'read only.' So I removed that designation. But the error still remained.

Steps to Reproduce (for bugs)

Add new site code to vvv-custom.yml.
Run vagrant up --provision.

Here is a link to more lines of the output. FYI it is not the full output. I can link that if you want.

Your Environment

VVV version: 2.6.0-4-g678d59b
VVV Git Branch: develop
Vagrant version: 2.2.4
VM Provider name: VirtualBox
VM Provider version: Version 6.0.6 r130049 (Qt5.6.2)
Operating System and version: Win 10, x64

tomjn commented 5 years ago

Can you embed those screenshots directly rather than via Imgur and use gist? Paste bin is under heavy load and won't show the output, and Imgur tries to optimise for mobile making the text unreadable

On Sat, 27 Apr 2019 at 14:26, cagross notifications@github.com wrote:

OK I believe I did it. The full output from my mv test /srv/www is here https://pastebin.com/HJihMvWC. Despite those errors, the directory--and the files within--appear to have moved to the expected location ( screenshot https://i.imgur.com/zZLyGp3.jpg). Did I do that correctly?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487286108, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZ3UWIQY67NOYTOHPXLPSRH7XANCNFSM4HIJ5F6Q .

cagross commented 5 years ago

OK here is the gist, and I'll attach the screenshot now.

cagross commented 5 years ago

oh-vvv-move

tomjn commented 5 years ago

Hmm so you can't change any permissions at all

If you move it to the place it tried to do the feed site that failed, does it provision?

I'm trying to figure out if we can git clone to a non mounted folder then move it into place

Also does disabling Windows defender fix the original issue? Best to rule out the antivirus theory

On Sat, 27 Apr 2019 at 14:30, cagross notifications@github.com wrote:

[image: oh-vvv-move] https://user-images.githubusercontent.com/10165488/56850320-4134cb80-692b-11e9-9cb6-f73bd205146c.jpg

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487286384, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZ33SIHQOPTM7X6EFW3PSRIOVANCNFSM4HIJ5F6Q .

tomjn commented 5 years ago

@cagross can you switch try this PR:

https://github.com/Varying-Vagrant-Vagrants/VVV/pull/1766

git fetch
git checkout tomjn/modify/forcev3networkshares
vagrant halt && vagrant up --provision

It adds the default fmode and dmode for /srv/www

cagross commented 5 years ago

If you move it to the place it tried to do the feed site that failed, does it provision?

Well now, I can clone the same repo to the VM, just as before. But when I try to move it, I cannot. It fails with:

vagrant@vvv:/tmp$ mv test /srv/www
mv: inter-device move failed: ‘test’ to ‘/srv/www/test’; unable to remove target: Is a directory

I'm a bit baffled by that right now. The same thing just worked a couple hours ago.

Also does disabling Windows defender fix the original issue? Best to rule out the antivirus theory

Just tried that, but no change--same provision error.

can you switch try this PR:

1766

OK I executed the three commands you specified--the first two completed without issue. The provision step though still indicated the same error, but more verbose:

    default: Downloading feedxl, git cloning from https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    default: error: chmod on /srv/www/feedxl/.git/config.lock failed: Operation not permitted
    default: fatal: could not set 'core.filemode' to 'false'
    default: Git failed to clone the site template for feedxl. It tried to clone the master of https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git into /srv/www/feedxl
    default: VVV won't be able to provision feedxl without the template. Check that you have permission to access the repo, and that the filesystem is writable
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

The full output is here.

tomjn commented 5 years ago

as a stopgap, you can git clone on the host using a git client, then reprovision. I'm inclined to merge #1766 for consistency purposes

tomjn commented 5 years ago

mv: inter-device move failed: ‘test’ to ‘/srv/www/test’; unable to remove target: Is a directory

Was there already a test folder in /srv/www? If so that would explain the issue.

Try again but:

vagrant up
add a site called cagrosstest with the custom site template to vvv-custom.yml
run vagrant ssh
git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /tmp/cagrosstest
mv /tmp/cagrosstest /srv/www
exit
vagrant provision

Do those comments work, and does it provision the site as expected?

Finally, you can remove the cagrosstest folder to cleanup for any other tests we do via rm -r /tmp/cagrosstest and rm -r /srv/www/cagrosstest

cagross commented 5 years ago

Was there already a test folder in /srv/www? If so that would explain the issue.

OK right, good catch. I removed it manually, then re-added the test folder to /srv/www, then tried to provision, but same error.

Do those comments work, and does it provision the site as expected?

Your latest suggestion seems to have worked--the cagrosstest site provisioned without error full output here, and I can open the front-end in my browser.

Finally, you can remove the cagrosstest folder to cleanup for any other tests we do via rm -r /tmp/cagrosstest and rm -r /srv/www/cagrosstest

Right O, thanks.

tomjn commented 5 years ago

That's awesome news, can you do a little more stuff and check the site continues to work as expected? My other thought was to test rsync based shared folders but I think cloning and moving into place is the trick.

My other concern is using git commands. Can you do this again but with a personal git repo? I'm not so interested in testing provisioner a so you might not need them at all, but rather the ability to git pull and commit

On Sun, 28 Apr 2019 at 17:55, cagross notifications@github.com wrote:

Was there already a test folder in /srv/www? If so that would explain the issue.

OK right, good catch. I removed it manually, then re-added the test folder to /srv/www, then tried to provision, but same error.

Do those comments work, and does it provision the site as expected?

Your latest suggestion seems to have worked--the cagrosstest site provisioned without error full output here https://gist.github.com/cagross/4c7027ed26fb73c5d28ed77d4dcb859f, and I can open the front-end in my browser.

Finally, you can remove the cagrosstest folder to cleanup for any other tests we do via rm -r /tmp/cagrosstest and rm -r /srv/www/cagrosstest

Right O, thanks.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487396628, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZY265XFKFWUNPXG3ZDPSXJGJANCNFSM4HIJ5F6Q .

jjsanderson commented 5 years ago

Back at my desk.

switch to the develop branch

git checkout develop
git pull
vagrant up --provision

Fails with:

[...]
default: Downloading wordpress-one, git cloning from https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git
    default: error: chmod on /srv/www/wordpress-one/.git/config.lock failed: Operation not permitted
    default: fatal: could not set 'core.filemode' to 'false'
    default: Git failed to clone the site template for wordpress-one. It tried to clone the master of https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git into /srv/www/wordpress-one
    default: VVV won't be able to provision wordpress-one without the template. Check that you have permission to access the repo, and that the filesystem is writable
The SSH command responded with a non-zero exit status. Vagrant
assumes that this means the command failed. The output for this command
should be in the log above. Please read the output to determine what
went wrong.

SSH into the VM with vagrant ssh

Run 2 git clones, I want to know the result of each: a. Clone into a VM folder git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /tmp/test

$ git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /tmp/test
Cloning into '/tmp/test'...
remote: Enumerating objects: 9, done.
remote: Counting objects: 100% (9/9), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 108 (delta 2), reused 9 (delta 2), pack-reused 99
Receiving objects: 100% (108/108), 25.51 KiB | 967.00 KiB/s, done.
Resolving deltas: 100% (44/44), done.

b. Clone into a network share folder git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /srv/www/test

$ git clone https://github.com/Varying-Vagrant-Vagrants/custom-site-template.git /srv/www/test
Cloning into '/srv/www/test'...
error: chmod on /srv/www/test/.git/config.lock failed: Operation not permitted
fatal: could not set 'core.filemode' to 'false'

What happens if you try to move [the first] git clone into a mounted/shared folder? Does that work? If so does a git pull also work?

vagrant@vvv:~$ cp -r /tmp/test /srv/www/test
vagrant@vvv:~$ cd /srv/www/test
vagrant@vvv:/srv/www/test$ ll
total 12
drwxrwxr-x 1 www-data vagrant  224 Apr 29 08:08 ./
drwxrwxr-x 1 www-data vagrant  128 Apr 29 08:08 ../
drwxrwxr-x 1 www-data vagrant  416 Apr 29 08:08 .git/
-rwxrwxr-- 1 www-data vagrant   46 Apr 29 08:08 .gitignore*
drwxrwxr-x 1 www-data vagrant  128 Apr 29 08:08 provision/
-rwxrwxr-- 1 www-data vagrant 2878 Apr 29 08:08 README.md*
-rwxrwxr-- 1 www-data vagrant   17 Apr 29 08:08 wp-cli.yml*
vagrant@vvv:/srv/www/test$ git pull
Already up to date.

So... that works. Huh. Continuing to work through thread...

jjsanderson commented 5 years ago

OK, so manually pulling in the custom-site-template and moving it across to /srv/www, then (on host) vagrant provision, does seem to work for me. I had to do this for wordpress-one and wordpress-two, or the provisioner errored out on the former.

cagross commented 5 years ago

@tomjn I'll need a few days to continue testing. Some things came up on my end.

ghost commented 5 years ago

question is the box that you are using is from 20190424.0.0

tomjn commented 5 years ago

I've tested with that box locally and not been able to reproduce the problem

On Mon, 29 Apr 2019 at 16:11, Benjamin Lu notifications@github.com wrote:

question is the box that you are using is from 20190424.0.0

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764#issuecomment-487619270, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZZ7TCADASUO3WGLB6TPS4FYHANCNFSM4HIJ5F6Q .

ghost commented 5 years ago

So i have tested the following OS: Windows 10, MacOS, and Linux VirtualBox: 6.0.6r130049 Vagrant: 2.2.4

I couldn't replicate the problem anymore, i had one computer had this problem but somehow when i did some vagrant commands and it just worked again. This is a weird case.

jjsanderson commented 5 years ago

question is the box that you are using is from 20190424.0.0

Yes, that's the box I'm using.

So i have tested the following OS: Windows 10, MacOS, and Linux VirtualBox: 6.0.6r130049 Vagrant: 2.2.4

I couldn't replicate the problem anymore, i had one computer had this problem but somehow when i did some vagrant commands and it just worked again. This is a weird case.

Quite. I only have OS X systems available for testing, but my laptop with apparently-identical software doesn't exhibit the same response. I'm glad I'm not the only one who sees the issue.

ghost commented 5 years ago

@jjsanderson did you had any issues before when this started.

jjsanderson commented 5 years ago

@benlumia007 no problems prior to this, in several years’ use. However, I’d been away from the affected iMac for an extended period, and it had a bunch of updates pending to the system, apps, brewed packages, and so on. I can’t for certain say which updates, if any, caused the issue. No apparent disk problem, either.

jjsanderson commented 5 years ago

In a previous comment, I wrote:

vagrant@vvv:~$ cp -r /tmp/test /srv/www/test

However, if I mv the directory I get a slew of 'Operation not permitted' errors on 'preserving times' and 'preserving permissions', for each file. So the permissions issue really isn't limited to git operations.

tomjn commented 5 years ago

Do you have the VB Guest extensions updated inside the VM? Try the vagrant plugin install vagrant-vbguest plugin, and open virtualbox and check the extension pack matches your current version

My understanding is that on a Virtualbox shared folder, we can set the permissions on mount, but can't change them, yet this hasn't been a problem for 99.999% of VVV users, it's odd that 2 cases of it would show up out of the blue, and on systems that were previously working

tomjn commented 5 years ago

As an additional note, on Windows I'd advise using VirtualBox 5.x branch. 6 might work but if Hyper-V or any portion of the Windows hypervisor is enabled ( it can be enabled even when Hyper-V is turned off if the WLS is active, or virtualmachine services, etc ), then VBox attempts to use the Windows Hyper Visor, giving a big performance hit, and enabling not so well tested VBox code

jjsanderson commented 5 years ago

Apologies for my absence.

I don't think I have an even vaguely-working install on which to test, at this point; my main install is now in a borked state somewhere between VVV 2 and 3; downgrading to 2 fails with unretrievable PHP packages (which I think is as expected, currently?).

A clean install of the develop branch, as of this morning, threw the same errors I was seeing with v2.6. When I have chance I'll try with the vbguest plugin.

However, we should probably park this issue and revisit post-VVV 3 release?

tomjn commented 5 years ago

I don't think I have an even vaguely-working install on which to test, at this point; my main install is now in a borked state somewhere between VVV 2 and 3; downgrading to 2 fails with unretrievable PHP packages (which I think is as expected, currently?).

VVV 2 is broken, with no plans to fix it, the steps needed to rebuild all the necessary packages would take infrastructure and time that simply aren't available, with the end result being people split across 2 major forks of VVV

A clean install of the develop branch, as of this morning, threw the same errors I was seeing with v2.6. When I have chance I'll try with the vbguest plugin.

I actually encountered the issue when testing VVV 3, and made a change to the vagrantfile that fixed it for myself

jjsanderson commented 5 years ago

VVV 2 is broken, with no plans to fix it, the steps needed to rebuild all the necessary packages would take infrastructure and time that simply aren't available, with the end result being people split across 2 major forks of VVV

Understood, and appreciated. Is it worth a banner on the repo. README to warn any newcomers of the current status?

I actually encountered the issue when testing VVV 3, and made a change to the vagrantfile that fixed it for myself

Hah! Is that #1686 ?

Thanks for your continued efforts; fingers crossed for a smooth VVV 3 process.

tomjn commented 5 years ago

Hah! Is that #1686 ?

Nope, it was owner/group/fmode/dmode changes

Having said that, I've a theory I'd like you to test:

switch back to VVV 3
eliminate the folder if it's got a partial git clone
Adjust provision/provision-site.sh by removing the noroot on the git commands, specifically those around line 105 where it clones the repo
reprovision and check if it makes a difference

jjsanderson commented 5 years ago

OK, will try to do this tomorrow morning (UK time) and will report back.

tomjn commented 5 years ago

OK, will try to do this tomorrow morning (UK time) and will report back.

Thanks! I know it's close to end of day here ( Manchester here )

@cagross same goes for you, removing the noroot fixed the same issue but for cloning utilities during provisioning. I'm still not sure why it fixes it, or what causes it, but it seems the next logical step

tomjn commented 5 years ago

@cagross @jjsanderson that change got merged into develop, so a pull down and reprovision should be all that's necessary to test

cagross commented 5 years ago

@tomjn OK I'm back, sorry for the delay.

so a pull down and reprovision should be all that's necessary to test

OK I can give this a shot. But to be clear, this is what I will do:

Ensure my VVV directory is free of any test sites previously created during this exercise.
From my VVV directory, run a git pull
Ensure vvv-custom.yml contains an entry for a new VVV site.
From my VVV directory, run a vagrant up --provision

Is that correct?

tomjn commented 5 years ago

Thats right, if you haven't updated to VVV 3 you'll also need to destroy your existing VM to recreate it on Ubuntu 18

On Wed, 15 May 2019 at 06:24, cagross notifications@github.com wrote:

@tomjn https://github.com/tomjn OK I'm back, sorry for the delay.

so a pull down and reprovision should be all that's necessary to test

OK I can give this a shot. But to be clear, this is what I will do:

1.

Ensure my VVV directory is free of any test sites previously created during this exercise. 2.

From my VVV directory, run a git pull 3.

Ensure vvv-custom.yml contains an entry for a new VVV site. 4.

From my VVV directory, run a vagrant up --provision

Is that correct?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/Varying-Vagrant-Vagrants/VVV/issues/1764?email_source=notifications&email_token=AAAOLZ3KE5YMYCEYM2MKDBLPVONCPA5CNFSM4HIJ5F62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVNRJNQ#issuecomment-492508342, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAOLZ3X6JQLNYZNN7FLD5LPVONCPANCNFSM4HIJ5F6Q .

jjsanderson commented 5 years ago

:-(

Fresh clone of develop branch, vagrant up --provision fails (initially quite early on, with adding hosts), then a second attempt fails in the same way as previously.

I've tried with fresh pulls and boxes, both before and after installing the vagrant-vbguest plugin.

provision-site.sh does not have noroot directives in the site provision clone (line 105).

Drat.

cagross commented 5 years ago

@tomjn OK. I need to first update to VVV 3. Let me carefully do that, then revisit this test.

jjsanderson commented 5 years ago

I just updated to the new VirtualBox 6.0.8 release: there are several lines in the release notes about shared folders and permissions (albeit for Windows hosts, not Mac OS). Working in my VVV3 develop pull from this morning:

vagrant destroy
vagrant up --provision

...completes without error. Wordpress-one and -two are provisioned, vvv.test is working, one.wordpress.test is working, everything smells of roses.

tomjn commented 5 years ago

yikes, how did I miss a vbox maintenance release, @cagross can you update to VirtualBox 6.0.8?

jjsanderson commented 5 years ago

The vbox update was on Monday (though it didn't show up for me until today).

I've just added sites to vvv-custom.yml, and they've provisioned correctly too. This text box is too small to contain my jubilation.

cagross commented 5 years ago

@cagross can you update to VirtualBox 6.0.8?

I can. But I'm still struggling to update to VVV 3 (over in the the VVV 3 issue).

Mte90 commented 5 years ago

Can you check with the latest version on develop branch?

ghost commented 5 years ago

this is due to permissions, I was having issues with permissions like this, I had to use :owner and :group a with mount_options to fixed the permissions for the shared folder on my of my laptops

Mte90 commented 5 years ago

I know but in the meantime we changed a lot of things and fixed permissions in few part of the provisioner so a new check with the latest release is helpful to understand if it is still an issue or not.

cagross commented 5 years ago

@Mte90 Sorry I won't be able to carry out any more testing at this point. Due to the issues during VVV update, I had to switch to a different local development platform. I'll probably stick with it for at least the near future.

tomjn commented 5 years ago

I'm closing this out, we've changed our box, provisioners, and moved from Ubuntu 14 to 18. If anybody gets this again please feel free to reopen

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

Varying-Vagrant-Vagrants / VVV