Uncaught exception: file too big for single read

[Bindong18]

I encountered the following issue when reading the content of .OVA file “test.ova” (its size is 2.1GB). Uncaught exception: file too big for single read e:/RailsInstaller/Ruby2.3.0/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in read' e:/RailsInstaller/Ruby2.3.0/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:indeployOVF'

My sample code is as follows: vim = RbVmomi::VIM.connect :host => hyper, :user => user, :password => password, :insecure => true dc = vim.serviceInstance.find_datacenter(dcName) or abort "datacenter not found" network = dc.network.first #Network computeResourceArray = dc.hostFolder.children #Array for ComputeResource computeResourceForDeploy=nil #ComputeResource computeResourceForDeploy = computeResourceArray.find { |x| x.host.first.summary.config.name == 'crName' } hostForDeploy=nil #HostSystem computeResourceArray.each do |x| host = x.host.first if host.summary.config.name == 'hostName' hostForDeploy = host
break; end end dataStoreArray = dc.datastore #Array for DataStore dataStoreDeploy=nil #DataStore dataStoreDeploy = dataStoreArray.find { |x| x.name ==’dsName’ } VIM = RbVmomi::VIM root_vm_folder = dc.vmFolder vm_folder = root_vm_folder vm_name = 'test327' ovf_url='//fileServer/test.ova' vm = vim.serviceContent.ovfManager.deployOVF( uri: ovf_url, vmName: vm_name, vmFolder: vm_folder, host: hostForDeploy, resourcePool: computeResourceForDeploy.resourcePool, datastore: dataStoreDeploy, ) Did anyone encounter the same issue ? Does anyone know how to resolve it?

[Bindong18]

Hi ,

Could you pls. tell me if there is any update?

Thank you very much

[jrgarcia]

@Bindong18 Unfortunately, I haven't been able to track this down. What version of Windows are you using? From the looks of it, you installed Ruby via the Rails Installer. Is that the case?

[Bindong18]

@jrgarcia windows: win2008r2 ,Yes Rails Installer3.3.0

[jrgarcia]

So it looks like this is an issue with reading large files with Ruby. You can see here where we call .read. Ruby can't handle reading a file of that size in that way. I'll look at changing this for reading large files, but I probably won't be able to modify that this week (I'm traveling right now).

[bergholdt]

If you are using 32bit then you have a limit of 2GB (https://en.wikipedia.org/wiki/2_GB_limit)

Could you check if your Ruby is 32 or 64 bit.

[Bindong18]

@bergholdt C:\Users\administrator>ruby -v ruby 2.3.3p222 (2016-11-21 revision 56859) [i386-mingw32]

C:\Users\administrator>irb irb(main):001:0> ['foo'].pack('p').size => 4

[jrgarcia]

@Bindong18 Yeah, it looks like you have a 32-bit version of Ruby installed (i386-mingw32). This means there is no way to read a file that large. When I get a chance, I'll test to see if there is a way to stream this rather than just reading it that will work, but this will never work with 32-bit.

[Bindong18]

@jrgarcia @bergholdt I changed ruby to 64bit, but still got the same error:

Uncaught exception: file too big for single read C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:in read' C:/Ruby23-x64/lib/ruby/gems/2.3.0/gems/rbvmomi-1.10.0/lib/rbvmomi/vim/OvfManager.rb:43:indeployOVF'

C:\Users\administrator.CRE3>ruby -v ruby 2.3.3p222 (2016-11-21 revision 56859) [x64-mingw32]

[jrgarcia]

@Bindong18 can you output ['foo'].pack('p').size again? If it's 8, then there isn't a problem with the OS. If it's 4, Ruby is still somehow installed as 32-bit.

Aside from that, I'm guessing the problem is just that Ruby itself can not read a file that large. I can change the implementation from the stdlib File#read, but I'll need to test it quite a bit to ensure I don't break anything that worked previously.

[Bindong18]

@jrgarcia

C:\Users\administrator.CRE3>irb irb(main):001:0> ['foo'].pack('p').size => 8 irb(main):002:0>

[jrgarcia]

Ok, so the problem then is that Ruby's File#read can't read files that large. I'll look at changing to something that can handle large files.

[Bindong18]

@jrgarcia Thank you very much