Closed syelle closed 13 years ago
dougal
commented
13 years ago Hi there.
Can you show me the directory listing as above for the Windows system?
Windows does not natively support file locks from Ruby, so these are not used unlike on Unix.
I suspect that when the index file is finished being written to the file is not marked as closed by Windows, so the next time AAI tries to overwrite it (using the mv operator), Windows throws a wobbly.
syelle
commented
13 years ago @dougal: Directory listing for the windows box is below.
C:\Users\shaun\Projects\enmasse-online>ls -la index\development\RefinerySetting
total 11
drwxr-xr-x 3 shaun Administrators 4096 Jul 11 17:44 .
drwxr-xr-x 3 shaun Administrators 0 Jul 5 16:22 ..
-rw-r--r-- 1 shaun Administrators 105 Jul 8 18:38 100_114_97.ind
-rw-r--r-- 1 shaun Administrators 111 Jul 11 17:44 100_114_97.ind.tmp
-rw-r--r-- 1 shaun Administrators 78 Jul 5 16:22 117_115_101.ind
-rw-r--r-- 1 shaun Administrators 80 Jul 11 17:42 117_115_101.ind.tmp
-rw-r--r-- 1 shaun Administrators 1 Jul 5 16:22 size
-rw-r--r-- 1 shaun Administrators 1 Jul 6 12:11 size.tmp
-rw-r--r-- 1 shaun Administrators 5 Jul 5 16:22 version
For reference, here is what the ubuntu system displays when listing the mount:
vagrant@ubuntu-10:/srv/enmasse-online/current$ ls -la index/development/RefinerySetting/
total 7
drwxrwxrwx 1 vagrant vagrant 4096 2011-07-12 02:44 .
drwxrwxrwx 1 vagrant vagrant 0 2011-07-06 01:22 ..
-rwxrwxrwx 1 vagrant vagrant 105 2011-07-09 03:38 100_114_97.ind
-rwxrwxrwx 1 vagrant vagrant 111 2011-07-12 02:44 100_114_97.ind.tmp
-rwxrwxrwx 1 vagrant vagrant 78 2011-07-06 01:22 117_115_101.ind
-rwxrwxrwx 1 vagrant vagrant 80 2011-07-12 02:42 117_115_101.ind.tmp
-rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 01:22 size
-rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 21:11 size.tmp
-rwxrwxrwx 1 vagrant vagrant 5 2011-07-06 01:22 version
I can move manually after db:migrate fails, so if the file isn't being marked as closed during db:migrate, it gets marked as closed after the error.
gonzedge
commented
13 years ago I had this issue creating the refinerycms application under Ubuntu on a virtual machine on Windows, but on a NTFS partition. It was being caused by a folder name not permitted, and the only workaround that I found was to create the application under the virtual ext3/ext4 partition.
I hope this helps.
ugisozols
commented
13 years ago @egonzalez0787 windows folder issue was fixed by this commit.
ericgj
commented
13 years ago I'm getting this error too, exact same scenario. Any hope of a fix?
dougal
commented
13 years ago Using the latest release, 0.7.5?
I've never been able to replicate this error (Windows Vista).
ericgj
commented
13 years ago Yes - using 0.7.5. I am on a Windows 7 host and NTFS partition. I think the problem is that lock_file is locking the file, since running on linux, but the underlying file is managed by Windows, so there is some conflict related to the lock. Will look around, this has got to be a common issue, not only in ruby.
dougal
commented
13 years ago Windows misses out on all the cool stuff! Let me know if this works for you guys, it should do as there is no longer any file moves under Windows.
This has been released in 0.7.6.
ericgj
commented
13 years ago Sorry, thanks for the quick response, but this doesn't work for me. I am running on Arch, so windows? returns false, and FileUtils.mv is still executed (and fails). It has something to do with the underlying file system being shared with the Windows host OS. It works on my ext3/ext4 partition.
I occasionally have other problems with mv on files on the shared mount, so I think it may be a deeper issue. See for example this and that open tickets on virtualbox.
dougal
commented
13 years ago The solution here is to move the index so it is not on the NTFS partition.
It seems I didn't fully understand this issue before, I will likely revert my previous fixes for this as a result.
gabynaiman
commented
13 years ago I think with these changes in the Storage class would be solved.
def write_file(file_path)
new_file = file_path.to_s
tmp_file = new_file + TEMP_FILE_EXTENSION
# Windows doesn't seem to play nice with writing then moving the file.
# https://github.com/dougal/acts_as_indexed/issues/15
writeable_file = windows? ? new_file : tmp_file
#File.open(tmp_file, 'w+') do |f|
File.open(writeable_file, 'w+') do |f|
yield(f)
end
FileUtils.mv(tmp_file, new_file) unless windows?
end
def prepare
version_path = @path.join('version')
if @path.exist?
#version_path_to_check = Pathname.new(version_path.to_s + (TEMP_FILE_EXTENSION if windows?).to_s)
version_path_to_check = Pathname.new(version_path.to_s)
unless version_path_to_check.exist? && version_path_to_check.read == ActsAsIndexed::INDEX_VERSION
raise OldIndexVersion, "Index was created prior to version #{ActsAsIndexed::INDEX_VERSION}. Please delete it, it will be rebuilt automatically."
end
else
@path.mkpath
# Do we need to lock for this? I don't think so as it is only ever
# creating, not modifying.
write_file(version_path) do |f|
f.write(ActsAsIndexed::INDEX_VERSION)
end
end
endI hope it's useful. I tried it and it works. I'm sorry for my bad english.
zigomir
commented
12 years ago I get same error with RefineryCMS 2.0.3 (Rails 3.2.3) when calling rake db:seed, using Vagrant (VirtualBox) for development from Windows. Gem acts_as_indexed is a version 0.7.8 and already has writeable_file = windows? ? new_file : tmp_file line in it.
How can I do about it? Thanks!
parndt
commented
12 years ago Of course the virtual box doesn't think it's windows, right?
zigomir
commented
12 years ago Yeah, sorry. It's probably problem because of this https://www.virtualbox.org/ticket/4890 mentioned also by ericgj.
axsuul
commented
12 years ago I'm also on VirtualBox and having this issue. Anyone have a temporary workaround?
ghost
commented
10 years ago Monkey patch is_windows_filesystem to fix it if you're using Vagrant on Windows. You need to install vagrant-host-path first: "vagrant plugin install vagrant-host-path"
Code: if ENV["VAGRANT_HOST_PATH"].present? && ENV["VAGRANT_HOST_PATH"].starts_with?('C:/') module ActsAsIndexed
class Configuration
def is_windows_filesystem?
return true
end
endend end
We're creating a development environment deployment that utilizes Refinery CMS (which, in turn, uses acts_as_indexed). Deployment is happening using vagrant & VirtualBox. Everything seems to go well when we deploy on Mac OS X, but we're seeing an issue during deployment on Windows. During the execution of db:migrate on the Ubuntu VM, deployment fails because a file cannot be renamed (moved) on the mount to the Windows share that holds the git checkout for the project.
Full stacktrace is below, but the relevant bit:
Operation not permitted - (/srv/enmasse-online/current/index/development/ RefinerySetting/100_114_97.ind.tmp, /srv/enmasse-online/current/index/ development/RefinerySetting/100_114_97.ind)When I run the mv outside of rake on the Ubuntu VM (
mv 100_114_97.ind.tmp 100_114_97.ind), the file is moved properly. I made the directory on the Windows box world writable and the issue was not resolved.What is act_as_indexed trying to do during the db:migrate that would work on a unix-based system but encounter issues on a unix-based system trying to move a file on a Windows mount?
Reference: http://groups.google.com/group/refinery-cms/browse_thread/thread/3f19f1c7ae5bbeee
Stack trace & manual move of the file below:
vagrant@lucid64:/srv/enmasse-online/current$ bundle exec rake db:migrate --trace * Invoke db:migrate (first_time) * Invoke environment (first_time) ** Execute environment rake aborted! Operation not permitted - (/srv/enmasse-online/current/index/development/RefinerySetting/100_114_97.ind.tmp, /srv/enmasse-online/current/index/development/RefinerySetting/100_114_97.ind) /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:515:in'
Tasks: TOP => db:migrate => environment rename' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:515:inblock in mv' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:1423:inblock in fu_each_src_dest' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:1439:infu_each_src_dest0' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:1421:infu_each_src_dest' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/fileutils.rb:504:inmv' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:164:inwrite_file' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:95:inblock (2 levels) in operate' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:175:inblock in lock_file' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/pathname.rb:829:inopen' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/pathname.rb:829:inopen' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:172:inlock_file' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:83:inblock in operate' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:80:ineach' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:80:inoperate' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/storage.rb:23:inadd' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/search_index.rb:28:inadd_record' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/class_methods.rb:65:inindex_add' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/acts_as_indexed-0.7.2/lib/acts_as_indexed/instance_methods.rb:16:inadd_to_index' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:416:in_run_create_callbacks' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/callbacks.rb:277:increate' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/persistence.rb:250:increate_or_update' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/callbacks.rb:273:inblock in create_or_update' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/callbacks.rb:419:in_run_save_callbacks' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/callbacks.rb:273:increate_or_update' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/persistence.rb:40:insave' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/validations.rb:43:insave' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/attribute_methods/dirty.rb:21:insave' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:240:inblock (2 levels) in save' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:292:inblock in with_transaction_returning_status' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/connection_adapters/abstract/database_statements.rb:139:intransaction' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:207:intransaction' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:290:inwith_transaction_returning_status' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:240:inblock in save' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:251:inrollback_active_record_state!' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activerecord-3.0.9/lib/active_record/transactions.rb:239:insave' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/refinerycms-settings-0.9.9.22/app/models/refinery_setting.rb:133:inset' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/refinerycms-settings-0.9.9.22/app/models/refinery_setting.rb:98:infind_or_set' /srv/enmasse-online/current/vendor/engines/refinerycms-images/lib/refinerycms-images.rb:35:inblock (2 levels) in <class:Engine>' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/dragonfly-0.9.4/lib/dragonfly/configurable.rb:35:inconfigure' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/dragonfly-0.9.4/lib/dragonfly/configurable.rb:42:inconfigure_with' /srv/enmasse-online/current/vendor/engines/refinerycms-images/lib/refinerycms-images.rb:26:inblock in class:Engine' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/initializable.rb:25:ininstance_exec' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/initializable.rb:25:inrun' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/initializable.rb:50:inblock in run_initializers' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/initializable.rb:49:ineach' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/initializable.rb:49:inrun_initializers' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/application.rb:134:ininitialize!' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/application.rb:77:inmethod_missing' /srv/enmasse-online/current/config/environment.rb:13:in<top (required)>' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:239:inrequire' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:239:inblock in require' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:225:inblock in load_dependency' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:596:innew_constants_in' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:225:inload_dependency' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/activesupport-3.0.9/lib/active_support/dependencies.rb:239:inrequire' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/application.rb:103:inrequire_environment!' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/railties-3.0.9/lib/rails/application.rb:218:inblock in initialize_tasks' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:205:incall' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:205:inblock in execute' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:200:ineach' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:200:inexecute' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:158:inblock in invoke_with_call_chain' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:151:ininvoke_with_call_chain' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:176:inblock in invoke_prerequisites' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:174:ineach' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:174:ininvoke_prerequisites' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:157:inblock in invoke_with_call_chain' /usr/local/rvm/rubies/ruby-1.9.2-p180/lib/ruby/1.9.1/monitor.rb:201:inmon_synchronize' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:151:ininvoke_with_call_chain' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/task.rb:144:ininvoke' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:112:ininvoke_task' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:90:inblock (2 levels) in top_level' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:90:ineach' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:90:inblock in top_level' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:129:instandard_exception_handling' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:84:intop_level' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:62:inblock in run' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:129:instandard_exception_handling' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/lib/rake/application.rb:59:inrun' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/gems/rake-0.9.2/bin/rake:32:in<top (required)>' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/bin/rake:19:inload' /usr/local/rvm/gems/ruby-1.9.2-p180@enmasse_online/bin/rake:19:invagrant@lucid64:/srv/enmasse-online/current$ ls -la index/development/RefinerySetting/ total 8 drwxrwxrwx 1 vagrant vagrant 4096 2011-07-09 03:37 . drwxrwxrwx 1 vagrant vagrant 0 2011-07-06 01:22 .. -rwxrwxrwx 1 vagrant vagrant 99 2011-07-09 03:33 100_114_97.ind -rwxrwxrwx 1 vagrant vagrant 105 2011-07-09 03:38 100_114_97.ind.tmp -rwxrwxrwx 1 vagrant vagrant 78 2011-07-06 01:22 117_115_101.ind -rwxrwxrwx 1 vagrant vagrant 80 2011-07-06 01:22 117_115_101.ind.tmp -rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 01:22 size -rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 21:11 size.tmp -rwxrwxrwx 1 vagrant vagrant 5 2011-07-06 01:22 version vagrant@lucid64:/srv/enmasse-online/current$ mv index/development/RefinerySetting/100_114_97.ind.tmp index/development/RefinerySetting/100_114_97.ind vagrant@lucid64:/srv/enmasse-online/current$ ls -la index/development/RefinerySetting/ total 7 drwxrwxrwx 1 vagrant vagrant 4096 2011-07-09 03:39 . drwxrwxrwx 1 vagrant vagrant 0 2011-07-06 01:22 .. -rwxrwxrwx 1 vagrant vagrant 105 2011-07-09 03:38 100_114_97.ind -rwxrwxrwx 1 vagrant vagrant 78 2011-07-06 01:22 117_115_101.ind -rwxrwxrwx 1 vagrant vagrant 80 2011-07-06 01:22 117_115_101.ind.tmp -rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 01:22 size -rwxrwxrwx 1 vagrant vagrant 1 2011-07-06 21:11 size.tmp -rwxrwxrwx 1 vagrant vagrant 5 2011-07-06 01:22 version vagrant@lucid64:/srv/enmasse-online/current$