Closed cu12 closed 7 years ago
I'm running into the same issue on Windows 10, using Hyper-V.
Environment:
hyperv
minikube-v0.18.0.iso
v0.19.1
2 CPUs, 2048MB RAM, 20GB VHD, Guest Services Disabled
Steps to reproduce: To reproduce, see this repository.
In the repo, there are two branches (master
, and failing
). The only difference is the failing
branch has one more copy of the lorem ipsum text than the master
branch.
There appears to be a relationship between the number of files, length of the filenames, and the size on disk. For example, if I shortened the filename lengths I would need more files in order to produce the same error, despite the fact that they took more total disk space. This makes me think that there's some sort of relationship between the size of the folder contents and the size of them represented on disk.
i.e: sizeOnDisk = fileSize + fileMetadataSize
Passing Scenario:
master
> minikube ssh -- ls /test
-v=10
):
18:07:53 connected
18:07:53 >>> 192.168.1.12:51580 Tversion tag 65535 msize 8192 version '9P2000.L'
18:07:53 <<< 192.168.1.12:51580 Rversion tag 65535 msize 8192 version '9P2000'
18:07:53 >>> 192.168.1.12:51580 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
18:07:53 <<< 192.168.1.12:51580 Rattach tag 1 aqid (a000 9932cdd9 'd')
18:07:53 >>> 192.168.1.12:51580 Tstat tag 1 fid 0
18:07:53 <<< 192.168.1.12:51580 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9932cdd9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:07:53 >>> 192.168.1.12:51580 Tstat tag 1 fid 0
18:07:53 <<< 192.168.1.12:51580 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9932cdd9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:07:53 >>> 192.168.1.12:51580 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
18:07:53 <<< 192.168.1.12:51580 Rwstat tag 1
SSH cmd err, output: <nil>:
18:08:02 >>> 192.168.1.12:51580 Tstat tag 1 fid 0
18:08:02 <<< 192.168.1.12:51580 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9932cdd9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:08:02 >>> 192.168.1.12:51580 Twalk tag 1 fid 0 newfid 1
18:08:02 <<< 192.168.1.12:51580 Rwalk tag 1
18:08:02 >>> 192.168.1.12:51580 Topen tag 1 fid 1 mode 0
18:08:02 <<< 192.168.1.12:51580 Ropen tag 1 qid (a000 9932cdd9 'd') iounit 0
18:08:02 >>> 192.168.1.12:51580 Tstat tag 1 fid 0
18:08:02 <<< 192.168.1.12:51580 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9932cdd9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:08:02 >>> 192.168.1.12:51580 Tread tag 1 fid 1 offset 0 count 8168
18:08:02 <<< 192.168.1.12:51580 Rread tag 1 count 8165
18:08:02 >>> 192.168.1.12:51580 Tread tag 1 fid 1 offset 8165 count 3
18:08:02 <<< 192.168.1.12:51580 Rread tag 1 count 0
18:08:02 >>> 192.168.1.12:51580 Tread tag 1 fid 1 offset 8165 count 8168
18:08:02 <<< 192.168.1.12:51580 Rread tag 1 count 0
18:08:02 >>> 192.168.1.12:51580 Tread tag 1 fid 1 offset 8165 count 8168
18:08:02 <<< 192.168.1.12:51580 Rread tag 1 count 0
18:08:02 >>> 192.168.1.12:51580 Tclunk tag 1 fid 1
18:08:02 <<< 192.168.1.12:51580 Rclunk tag 1
Broken Scenario:
failing
> minikube ssh -- ls /test
ls: reading directory '/test': Unknown error 526
18:05:39 connected
18:05:39 >>> 192.168.1.12:38418 Tversion tag 65535 msize 8192 version '9P2000.L'
18:05:39 <<< 192.168.1.12:38418 Rversion tag 65535 msize 8192 version '9P2000'
18:05:39 >>> 192.168.1.12:38418 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
18:05:39 <<< 192.168.1.12:38418 Rattach tag 1 aqid (a000 9930c4f9 'd')
18:05:39 >>> 192.168.1.12:38418 Tstat tag 1 fid 0
18:05:39 <<< 192.168.1.12:38418 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9930c4f9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:05:39 >>> 192.168.1.12:38418 Tstat tag 1 fid 0
18:05:39 <<< 192.168.1.12:38418 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9930c4f9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:05:39 >>> 192.168.1.12:38418 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
18:05:39 <<< 192.168.1.12:38418 Rwstat tag 1
SSH cmd err, output: <nil>:
18:05:58 >>> 192.168.1.12:38418 Tstat tag 1 fid 0
18:05:58 <<< 192.168.1.12:38418 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9930c4f9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:05:58 >>> 192.168.1.12:38418 Twalk tag 1 fid 0 newfid 1
18:05:58 <<< 192.168.1.12:38418 Rwalk tag 1
18:05:58 >>> 192.168.1.12:38418 Topen tag 1 fid 1 mode 0
18:05:58 <<< 192.168.1.12:38418 Ropen tag 1 qid (a000 9930c4f9 'd') iounit 0
18:05:58 >>> 192.168.1.12:38418 Tstat tag 1 fid 0
18:05:58 <<< 192.168.1.12:38418 Rstat tag 1 st ('.\test' 'none' 'none' 'none' q (a000 9930c4f9 'd') m d777 at 0 mt 0 l 40960 t 0 d 0 ext )
18:05:58 >>> 192.168.1.12:38418 Tread tag 1 fid 1 offset 0 count 8168
18:05:58 <<< 192.168.1.12:38418 Rread tag 1 count 8000
18:05:58 >>> 192.168.1.12:38418 Tread tag 1 fid 1 offset 8000 count 168
18:05:58 <<< 192.168.1.12:38418 Rerror tag 1 ename 'too small read size for dir entry' ecode 0
18:05:58 >>> 192.168.1.12:38418 Tclunk tag 1 fid 1
18:05:58 <<< 192.168.1.12:38418 Rclunk tag 1
For reference, it appears that the error is being emitted from go9p/ufs.go#L384.
The key section being:
if count == 0 && int(tc.Offset) < len(fid.dirents) && len(fid.dirents) > 0 {
req.RespondError(&Error{"too small read size for dir entry", EINVAL})
return
}
Thank you very much for the detailed bug report. The examples in the github repo were really useful in testing this. This 9p ufs implementation is adapted from https://github.com/rminnich/go9p/ with some small tweaks for cross-platform support. So bear with me as I will need to read over this. The tests below involve running ls
from within the mounted /test
directory from the repo.
The failure condition for this the failing branch appears to be going through:
The second switch case, len(fid.dirents[tc.Offset:]) > int(tc.Count)
:
https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L366-L367
This 3rd nested if statement, nextend > 0
:
https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L368-L369
and then hitting the error you mentioned above: https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L383-L386
A gist of some debug output for the failing case is here: https://gist.github.com/aaron-prindle/1a6e09d1649bd50f646b19c16c07dcfe
It seems that an initial read is performed for the directory
In the successful case, the flow is: The default switch case: https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L368-L369
The first nested if statement (but none of the nested if statements trigger): https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L374
A gist of some debug output for the successful case is here: https://gist.github.com/aaron-prindle/918a6ff79d697674ae8f20583c3e7235
From the implementation it seems that this error condition is designed to catch when the tc.Offset smaller than fid.dirents. fid.dirents is byte[] where each entry in the list is a single character and the whole e list contains all of the filenames concatinated with some metadata: fid.dirents example gist: https://gist.github.com/aaron-prindle/bb7b415901a71f10b3bbf7613efd9554
This value grows as you put more files/longer-file-names in a directory. tc.Offset (and tc.Count) seem to be a parameters that are passed into the call and as such cannot be modified to be "larger". From investigating the code it seems that it is possible that when this statement is triggered: https://github.com/kubernetes/minikube/blob/master/third_party/go9p/ufs.go#L377 the count ends up equaling 0 even though the 0 value arrived by the subtracting and it is not a sentinel as the 0 value is usually used throughout.
There's actually a flag option for 9p with mount, msize, that can change the message size allowable for mount. Configuring this flag should allow you to use arbitrary sized/named directories. The PR for this is here, you can use it with minikube mount --msize=<the number of bytes to use for 9p packet payload>
The new PR is here:
I am running into this too.
Minikube version: v0.20.0 Environment: OS: Windows 10 Pro (Anniversary Edition) VM Driver: hyperv ISO version: minikube-v0.20.0.iso
I have a directory with 121 files in it, and it shows as empty with error 526.
The directory is actually this project: https://github.com/Shopify/sarama , and none of the file names are long or have weird characters.
If I delete all the files and create empty files one by one, I can see the error is caused at exactly 102 files (ie: I can ls
and see 101 blank files just fine, but if I add one more blank file I get error 526).
So strange! I'm glad you've already fixed it and I'm looking forward to the next release!
Would this be able to make it into a v0.20.1 release?
@veqryn I think we're planning a release today
This should be fixed with https://github.com/kubernetes/minikube/pull/1705. Closing.
Hello everybody, I'm running into this on ls -la /node_modules
for the mounted folder inside the minikube cluster, could someone please specify how big should be --msize
to get rid of this "error 526"??
Regards!
Is this a BUG REPORT or FEATURE REQUEST? (choose one):
Minikube version (use
minikube version
):Environment:
cat ~/.minikube/machines/minikube/config.json | grep DriverName
): virtualboxcat ~/.minikube/machines/minikube/config.json | grep -i ISO
orminikube ssh cat /etc/VERSION
): minikube-v0.18.0.isoWhat happened: Mounted a directory with gazillions of folders/files (69893/315668)
minikube mount ../:/src
What you expected to happen: Files to be listed, but got
How to reproduce it (as minimally and precisely as possible): Not sure yet, but I guess it has to do something with the amount of files and directories as mounting a simple directory inside the structure works fine.
Anything else do we need to know: mount part of
--v=10
the error: