genimage-10: Test failure in jffs2

pengutronix / genimage

tool to generate multiple filesystem and flash images from a tree

GNU General Public License v2.0

308 stars 110 forks source link

genimage-10: Test failure in jffs2 #37

Closed daym closed 5 years ago

daym commented 6 years ago

expecting success: 
        run_genimage jffs2.config test.jffs2 &&
        md5sum -c '/tmp/guix-build-genimage-10.drv-0/genimage-10/test/jffs2.md5'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00075539 s, 1.4 GB/s
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "rm -rf "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp"/*" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/images"" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "cp -a "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/root" "/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp/root"" (stderr+stdout):
INFO: jffs2(test.jffs2): cmd: "mkfs.jffs2 --eraseblock=131072 -d '/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp/root' -o '/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/images/test.jffs2' " (stderr+stdout):
md5sum: WARNING: 1 computed checksum did NOT match

images/test.jffs2: FAILED
not ok 10 - jffs2
FAIL: test/basic-images.test 10 - jffs2
#       
#               run_genimage jffs2.config test.jffs2 &&
#               md5sum -c '/tmp/guix-build-genimage-10.drv-0/genimage-10/test/jffs2.md5'
#

michaelolbrich commented 5 years ago

These checks are pretty fragile. What's the version of your mtd-utils?

michaelolbrich commented 5 years ago

Hmm, travis fails if I install mtd-utils there. That's version 1.5.0 according to the apt install log. I've built that version myself and I don't see the problem here. Is mkfs.jffs2 patched in any way? And what's the file size for test.jffs2? I get 4500 bytes here.

daym commented 5 years ago

mtd-utils 1.5.2. No patches, it's the vanilla package that GNU Guix provides.

michaelolbrich commented 5 years ago

Very strange. Can you send me the test.jffs2? Just hack the test script to copy the file elsewhere before it is deleted.

daym commented 5 years ago

test.jffs2, base64 encoded (using the unix program "base64"):

hRkDIAwAAACxsB7khRkB4CsAAADmbiZ9AQAAAAAAAAACAAAAgGW8TgMEAABVcZ6cuFW+iWJhcv+F
GQLgRAAAAB3795gCAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAAONvz34UZAeArAAAA5m4mfQEAAAABAAAAAwAAAIBlvE4DBAAAq6xTqYrdZYdiYXr/hRkC
4EQAAAAd+/eYAwAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAOQ3QuKFGQHgKwAAAOZuJn0BAAAAAgAAAAQAAACAZbxOAwQAANbxA/QzvDJzZm9v/4UZAuBE
AAAAHfv3mAQAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AADwsVdShRkB4DMAAACWEYvoAQAAAAMAAAAFAAAAgGW8TgsEAADHBHoEWr3+93dpdGggc3BhY2Vz
/4UZAuBEAAAAHfv3mAUAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAA
AAAAAAAAAAAsXeZvhRkB4CkAAABtpi/XAgAAAAQAAAAGAAAAgGW8TgEEAADsgwlOOgDeUTH///+F
GQLgRAAAAB3795gGAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAASGg0KYUZAeApAAAAbaYv1wIAAAAFAAAABwAAAIBlvE4BBAAAEl7Ee4BR18gy////hRkC
4EQAAAAd+/eYBwAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAJSEhRSFGQHgKQAAAG2mL9cCAAAABgAAAAgAAACAZbxOAQQAAJF1mCAWYdC/M////4UZAuBE
AAAAHfv3mAgAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AAAhYm6ShRkB4CsAAADmbiZ9BgAAAAcAAAAJAAAAgGW8TgMIAADZcqv4418thW9uZf+FGQLgRAAA
AB3795gJAAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA
7gRc5IUZAeArAAAA5m4mfQYAAAAIAAAACgAAAIBlvE4DCAAAE8ffAXRTi+50d2//hRkC4EQAAAAd
+/eYCgAAAAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAIox
jqKFGQHgKwAAAOZuJn0HAAAACQAAAAsAAACAZbxOAwgAAKshdVHjXy2Fb25l/4UZAuBEAAAAHfv3
mAsAAAABAAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAABW3T+f
hRkB4CsAAADmbiZ9BwAAAAoAAAAMAAAAgGW8TgMIAADWfCUMdFOL7nR3b/+FGQLgRAAAAB3795gM
AAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAQlsqL4UZ
AeArAAAA5m4mfQgAAAALAAAADQAAAIBlvE4DCAAACdF3ReNfLYVvbmX/hRkC4EQAAAAd+/eYDQAA
AAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAJ63mxKFGQHg
KwAAAOZuJn0IAAAADAAAAA4AAACAZbxOAwgAAIz7EM90U4vudHdv/4UZAuBEAAAAHfv3mA4AAAAB
AAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAD6gklUhRkB4CkA
AABtpi/XAwAAAA0AAAAPAAAAgGW8TgEEAADljLxrOgDeUTH///+FGQLgRAAAAB3795gPAAAAAQAA
AMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAANeR7IoUZAeApAAAA
baYv1wMAAAAOAAAAEAAAAIBlvE4BBAAAmkr4PIBR18gy////hRkC4EQAAAAd+/eYEAAAAAEAAADA
QQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAMLDbMmFGQHgKQAAAG2m
L9cDAAAADwAAABEAAACAZbxOAQQAAGSXNQkWYdC/M////4UZAuBEAAAAHfv3mBEAAAABAAAAwEEA
AAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAAeL930hRkB4CsAAADmbiZ9
DwAAABAAAAASAAAAgGW8TgMIAAA0EENm418thW9uZf+FGQLgRAAAAB3795gSAAAAAQAAAICBAABZ
a+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAaZCM+YUZAeArAAAA5m4mfQ8A
AAARAAAAEwAAAIBlvE4DCAAAys2OU3RTi+50d2//hRkC4EQAAAAd+/eYEwAAAAEAAACAgQAAWWvm
AwAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAALV8PcSFGQHgKwAAAOZuJn0QAAAA
EgAAABQAAACAZbxOAwgAALNLh5LjXy2Fb25l/4UZAuBEAAAAHfv3mBQAAAABAAAAgIEAAFlr5gMA
AAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAACh+ih0hRkB4CsAAADmbiZ9EAAAABMA
AAAVAAAAgGW8TgMIAABNlkqndFOL7nR3b/+FGQLgRAAAAB3795gVAAAAAQAAAICBAABZa+YDAAAA
AIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAfRaZSYUZAeArAAAA5m4mfREAAAAUAAAA
FgAAAIBlvE4DCAAAjodKSONfLYVvbmX/hRkC4EQAAAAd+/eYFgAAAAEAAACAgQAAWWvmAwAAAACA
ZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAABkjSw+FGQHgKwAAAOZuJn0RAAAAFQAAABcA
AACAZbxOAwgAAHBah310U4vudHdv/4UZAuBEAAAAHfv3mBcAAAABAAAAgIEAAFlr5gMAAAAAgGW8
ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADFz/oyhRkB4CkAAABtpi/XBAAAABYAAAAYAAAA
gGW8TgEEAAAmO4ROOgDeUTH///+FGQLgRAAAAB3795gYAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6A
ZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAY6OS/4UZAeApAAAAbaYv1wQAAAAXAAAAGQAAAIBl
vE4BBAAA2OZJe4BR18gy////hRkC4EQAAAAd+/eYGQAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8
ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAL9PI8KFGQHgKQAAAG2mL9cEAAAAGAAAABoAAACAZbxO
AQQAABJTPYIWYdC/M////4UZAuBEAAAAHfv3mBoAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6A
ZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADbevGEhRkB4CsAAADmbiZ9GAAAABkAAAAbAAAAgGW8TgMI
AADNKRMn418thW9uZf+FGQLgRAAAAB3795gbAAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8
TgAAAAAAAAAAAAAAAAAAAAAAAAAAFBzD8oUZAeArAAAA5m4mfRgAAAAaAAAAHAAAAIBlvE4DCAAA
sHRDenRTi+50d2//hRkC4EQAAAAd+/eYHAAAAAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4A
AAAAAAAAAAAAAAAAAAAAAAAAAACa1kKFGQHgKwAAAOZuJn0ZAAAAGwAAAB0AAACAZbxOAwgAAAiS
6SrjXy2Fb25l/4UZAuBEAAAAHfv3mB0AAAABAAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAA
AAAAAAAAAAAAAAAAAAAAAADcdmd/hRkB4CsAAADmbiZ9GQAAABwAAAAeAAAAgGW8TgMIAACNuI6g
dFOL7nR3b/+FGQLgRAAAAB3795geAAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAA
AAAAAAAAAAAAAAAAAAAAuEO1OYUZAeArAAAA5m4mfRoAAAAdAAAAHwAAAIBlvE4DCAAAuSjqOuNf
LYVvbmX/hRkC4EQAAAAd+/eYHwAAAAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4AAAAAAAAA
AAAAAAAAAAAAAAAAAGSvBASFGQHgKwAAAOZuJn0aAAAAHgAAACAAAACAZbxOAwgAAD41n3V0U4vu
dHdv/4UZAuBEAAAAHfv3mCAAAAABAAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAA
AAAAAAAAAAAAAAAXCuo0hRkB4CkAAABtpi/XBQAAAB8AAAAhAAAAgGW8TgEEAAArAhh/OgDeUTH/
//+FGQLgRAAAAB3795ghAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAA
AAAAAAAAAAAA2GzYQoUZAeApAAAAbaYv1wUAAAAgAAAAIgAAAIBlvE4BBAAAAvJ0d4BR18gy////
hRkC4EQAAAAd+/eYIgAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAA
AAAAAAAAALxZCgSFGQHgKQAAAG2mL9cFAAAAIQAAACMAAACAZbxOAQQAAPwvuUIWYdC/M////4UZ
AuBEAAAAHfv3mCMAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAA
AAAAAABgtbs5hRkB4CsAAADmbiZ9IQAAACIAAAAkAAAAgGW8TgMIAAA8+Ovo418thW9uZf+FGQLg
RAAAAB3795gkAAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAA
AAAAZ7ktwoUZAeArAAAA5m4mfSEAAAAjAAAAJQAAAIBlvE4DCAAAwiUm3XRTi+50d2//hRkC4EQA
AAAd+/eYJQAAAAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAA
ALtVnP+FGQHgKwAAAOZuJn0iAAAAJAAAACYAAACAZbxOAwgAAI1C6PjjXy2Fb25l/4UZAuBEAAAA
Hfv3mCYAAAABAAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADf
YE65hRkB4CsAAADmbiZ9IgAAACUAAAAnAAAAgGW8TgMIAABznyXNdFOL7nR3b/+FGQLgRAAAAB37
95gnAAAAAQAAAICBAABZa+YDAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAA4z/
hIUZAeArAAAA5m4mfSMAAAAmAAAAKAAAAIBlvE4DCAAAto8e8+NfLYVvbmX/hRkC4EQAAAAd+/eY
KAAAAAEAAACAgQAAWWvmAwAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAALZqFAKF
GQHgKwAAAOZuJn0jAAAAJwAAACkAAACAZbxOAwgAAEhS08Z0U4vudHdv/4UZAuBEAAAAHfv3mCkA
AAABAAAAgIEAAFlr5gMAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAABqhqU/

daym commented 5 years ago

test.jffs2 has 4500 bytes for me as well.

daym commented 5 years ago

Log of this new run:

expecting success: 
        run_genimage jffs2.config test.jffs2 &&
        md5sum -c '/tmp/guix-build-genimage-10.drv-37/genimage-10/test/jffs2.md5'
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.000954874 s, 1.1 GB/s
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "rm -rf "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/tmp"/*" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/images"" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "cp -a "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/root" "/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/tmp/root"" (stderr+stdout):
INFO: jffs2(test.jffs2): cmd: "mkfs.jffs2 --eraseblock=131072 -d '/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/tmp/root' -o '/tmp/guix-build-genimage-10.drv-37/genimage-10/trash directory.basic-images.test/images/test.jffs2' " (stderr+stdout):
md5sum: WARNING: 1 computed checksum did NOT match

images/test.jffs2: FAILED
not ok 10 - jffs2
FAIL: test/basic-images.test 10 - jffs2

michaelolbrich commented 5 years ago

I've noticed two differences in the images:

the permissions are different I've pushed #44 to set a fix umask. That should fix this problem
the files and directories in your image do not belong to root I'm guessing, that fakeroot is not available in your case. We don't check for that currently and I think the automake test magic may just skip it

michaelolbrich commented 5 years ago

I've enabled testing jffs2 in travis with #45. That works correct now that I've merged #44. That should fix part of your problem. Btw, I've used jffs2reader to compare the jffs2 images. If you use it on your image, then it's easy to see that the files belong to some non-root user/group.

daym commented 5 years ago

(mkfs.jffs2 seems to have a "-U" option which will make root own all the files)

daym commented 5 years ago

We're trying to package it for the Guix distribution, and it's indeed not the root user. This is a distribution build server and allowing packages to build as root would provide a much too large attack surface for bad actors (essentially any package could backdoor the distribution build servers). We have a container, though, so I guess we could allow the root user there, but currently that's not done.

daym commented 5 years ago

What's the correct jffs2reader output?

michaelolbrich commented 5 years ago

drwx------ 1    0        0                0 /baz/
drwx------ 1    0        0                0 /baz/1/
-rw------- 1    0        0                0 /baz/1/one 
-rw------- 1    0        0                0 /baz/1/two 
drwx------ 1    0        0                0 /baz/2/
-rw------- 1    0        0                0 /baz/2/one 
-rw------- 1    0        0                0 /baz/2/two 
drwx------ 1    0        0                0 /baz/3/
-rw------- 1    0        0                0 /baz/3/one 
-rw------- 1    0        0                0 /baz/3/two 
drwx------ 1    0        0                0 /foo/
drwx------ 1    0        0                0 /foo/1/
-rw------- 1    0        0                0 /foo/1/one 
-rw------- 1    0        0                0 /foo/1/two 
drwx------ 1    0        0                0 /foo/2/
-rw------- 1    0        0                0 /foo/2/one 
-rw------- 1    0        0                0 /foo/2/two 
drwx------ 1    0        0                0 /foo/3/
-rw------- 1    0        0                0 /foo/3/one 
-rw------- 1    0        0                0 /foo/3/two 
drwx------ 1    0        0                0 /with spaces/
drwx------ 1    0        0                0 /with spaces/1/
-rw------- 1    0        0                0 /with spaces/1/one 
-rw------- 1    0        0                0 /with spaces/1/two 
drwx------ 1    0        0                0 /with spaces/2/
-rw------- 1    0        0                0 /with spaces/2/one 
-rw------- 1    0        0                0 /with spaces/2/two 
drwx------ 1    0        0                0 /with spaces/3/
-rw------- 1    0        0                0 /with spaces/3/one 
-rw------- 1    0        0                0 /with spaces/3/two

In your file the permisions where different (probably fixed now with the explicit umasl) and the user/group.

It is not necessary to run the test as root. fakeroot should handle this correctly. The test script should be executed with fakeroot (see TEST_LOG_COMPILER in Makefile.am).

daym commented 5 years ago

I get:

drwx------ 1    0        0                0 /baz/
drwx------ 1    0        0                0 /baz/1/
-rw------- 1    0        0                0 /baz/1/one 
-rw------- 1    0        0                0 /baz/1/two 
drwx------ 1    0        0                0 /baz/2/
-rw------- 1    0        0                0 /baz/2/one 
-rw------- 1    0        0                0 /baz/2/two 
drwx------ 1    0        0                0 /baz/3/
-rw------- 1    0        0                0 /baz/3/one 
-rw------- 1    0        0                0 /baz/3/two 
drwx------ 1    0        0                0 /foo/
drwx------ 1    0        0                0 /foo/1/
-rw------- 1    0        0                0 /foo/1/one 
-rw------- 1    0        0                0 /foo/1/two 
drwx------ 1    0        0                0 /foo/2/
-rw------- 1    0        0                0 /foo/2/one 
-rw------- 1    0        0                0 /foo/2/two 
drwx------ 1    0        0                0 /foo/3/
-rw------- 1    0        0                0 /foo/3/one 
-rw------- 1    0        0                0 /foo/3/two 
drwx------ 1    0        0                0 /with spaces/
drwx------ 1    0        0                0 /with spaces/1/
-rw------- 1    0        0                0 /with spaces/1/one 
-rw------- 1    0        0                0 /with spaces/1/two 
drwx------ 1    0        0                0 /with spaces/2/
-rw------- 1    0        0                0 /with spaces/2/one 
-rw------- 1    0        0                0 /with spaces/2/two 
drwx------ 1    0        0                0 /with spaces/3/
-rw------- 1    0        0                0 /with spaces/3/one 
-rw------- 1    0        0                0 /with spaces/3/two

And the test fails...

INFO: jffs2(test.jffs2): cmd: "mkfs.jffs2 -U --eraseblock=131072 -d '/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/tmp/root' -o '/tmp/guix-build-genimage-10.drv-0/genimage-10/trash directory.basic-images.test/images/test.jffs2' " (stderr+stdout):
md5sum: WARNING: 1 computed checksum did NOT match

images/test.jffs2: FAILED
not ok 10 - jffs2

daym commented 5 years ago

The new test.jffs2, base64 encoded (using the unix program "base64"):

hRkDIAwAAACxsB7khRkB4CsAAADmbiZ9AQAAAAAAAAACAAAAgGW8TgMEAABVcZ6cuFW+iWJhcv+F
GQLgRAAAAB3795gCAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAAONvz34UZAeArAAAA5m4mfQEAAAABAAAAAwAAAIBlvE4DBAAAq6xTqYrdZYdiYXr/hRkC
4EQAAAAd+/eYAwAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAOQ3QuKFGQHgKwAAAOZuJn0BAAAAAgAAAAQAAACAZbxOAwQAANbxA/QzvDJzZm9v/4UZAuBE
AAAAHfv3mAQAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AADwsVdShRkB4DMAAACWEYvoAQAAAAMAAAAFAAAAgGW8TgsEAADHBHoEWr3+93dpdGggc3BhY2Vz
/4UZAuBEAAAAHfv3mAUAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAA
AAAAAAAAAAAsXeZvhRkB4CkAAABtpi/XAgAAAAQAAAAGAAAAgGW8TgEEAADsgwlOOgDeUTH///+F
GQLgRAAAAB3795gGAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAASGg0KYUZAeApAAAAbaYv1wIAAAAFAAAABwAAAIBlvE4BBAAAEl7Ee4BR18gy////hRkC
4EQAAAAd+/eYBwAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAJSEhRSFGQHgKQAAAG2mL9cCAAAABgAAAAgAAACAZbxOAQQAAJF1mCAWYdC/M////4UZAuBE
AAAAHfv3mAgAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AAAhYm6ShRkB4CsAAADmbiZ9BgAAAAcAAAAJAAAAgGW8TgMIAADZcqv4418thW9uZf+FGQLgRAAA
AB3795gJAAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA
Lk+/QoUZAeArAAAA5m4mfQYAAAAIAAAACgAAAIBlvE4DCAAAE8ffAXRTi+50d2//hRkC4EQAAAAd
+/eYCgAAAAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAEp6
bQSFGQHgKwAAAOZuJn0HAAAACQAAAAsAAACAZbxOAwgAAKshdVHjXy2Fb25l/4UZAuBEAAAAHfv3
mAsAAAABAAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAACWltw5
hRkB4CsAAADmbiZ9BwAAAAoAAAAMAAAAgGW8TgMIAADWfCUMdFOL7nR3b/+FGQLgRAAAAB3795gM
AAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAghDJiYUZ
AeArAAAA5m4mfQgAAAALAAAADQAAAIBlvE4DCAAACdF3ReNfLYVvbmX/hRkC4EQAAAAd+/eYDQAA
AAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAF78eLSFGQHg
KwAAAOZuJn0IAAAADAAAAA4AAACAZbxOAwgAAIz7EM90U4vudHdv/4UZAuBEAAAAHfv3mA4AAAAB
AAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAA6yaryhRkB4CkA
AABtpi/XAwAAAA0AAAAPAAAAgGW8TgEEAADljLxrOgDeUTH///+FGQLgRAAAAB3795gPAAAAAQAA
AMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAANeR7IoUZAeApAAAA
baYv1wMAAAAOAAAAEAAAAIBlvE4BBAAAmkr4PIBR18gy////hRkC4EQAAAAd+/eYEAAAAAEAAADA
QQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAMLDbMmFGQHgKQAAAG2m
L9cDAAAADwAAABEAAACAZbxOAQQAAGSXNQkWYdC/M////4UZAuBEAAAAHfv3mBEAAAABAAAAwEEA
AAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAAeL930hRkB4CsAAADmbiZ9
DwAAABAAAAASAAAAgGW8TgMIAAA0EENm418thW9uZf+FGQLgRAAAAB3795gSAAAAAQAAAICBAAAA
AAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAqdtvX4UZAeArAAAA5m4mfQ8A
AAARAAAAEwAAAIBlvE4DCAAAys2OU3RTi+50d2//hRkC4EQAAAAd+/eYEwAAAAEAAACAgQAAAAAA
AAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAHU33mKFGQHgKwAAAOZuJn0QAAAA
EgAAABQAAACAZbxOAwgAALNLh5LjXy2Fb25l/4UZAuBEAAAAHfv3mBQAAAABAAAAgIEAAAAAAAAA
AAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAABhscvShRkB4CsAAADmbiZ9EAAAABMA
AAAVAAAAgGW8TgMIAABNlkqndFOL7nR3b/+FGQLgRAAAAB3795gVAAAAAQAAAICBAAAAAAAAAAAA
AIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAvV1674UZAeArAAAA5m4mfREAAAAUAAAA
FgAAAIBlvE4DCAAAjodKSONfLYVvbmX/hRkC4EQAAAAd+/eYFgAAAAEAAACAgQAAAAAAAAAAAACA
ZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAANloqKmFGQHgKwAAAOZuJn0RAAAAFQAAABcA
AACAZbxOAwgAAHBah310U4vudHdv/4UZAuBEAAAAHfv3mBcAAAABAAAAgIEAAAAAAAAAAAAAgGW8
ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAAFhBmUhRkB4CkAAABtpi/XBAAAABYAAAAYAAAA
gGW8TgEEAAAmO4ROOgDeUTH///+FGQLgRAAAAB3795gYAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6A
ZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAY6OS/4UZAeApAAAAbaYv1wQAAAAXAAAAGQAAAIBl
vE4BBAAA2OZJe4BR18gy////hRkC4EQAAAAd+/eYGQAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8
ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAL9PI8KFGQHgKQAAAG2mL9cEAAAAGAAAABoAAACAZbxO
AQQAABJTPYIWYdC/M////4UZAuBEAAAAHfv3mBoAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6A
ZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADbevGEhRkB4CsAAADmbiZ9GAAAABkAAAAbAAAAgGW8TgMI
AADNKRMn418thW9uZf+FGQLgRAAAAB3795gbAAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8
TgAAAAAAAAAAAAAAAAAAAAAAAAAA1FcgVIUZAeArAAAA5m4mfRgAAAAaAAAAHAAAAIBlvE4DCAAA
sHRDenRTi+50d2//hRkC4EQAAAAd+/eYHAAAAAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4A
AAAAAAAAAAAAAAAAAAAAAAAAAMDRNeSFGQHgKwAAAOZuJn0ZAAAAGwAAAB0AAACAZbxOAwgAAAiS
6SrjXy2Fb25l/4UZAuBEAAAAHfv3mB0AAAABAAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAA
AAAAAAAAAAAAAAAAAAAAAAAcPYTZhRkB4CsAAADmbiZ9GQAAABwAAAAeAAAAgGW8TgMIAACNuI6g
dFOL7nR3b/+FGQLgRAAAAB3795geAAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAA
AAAAAAAAAAAAAAAAAAAAeAhWn4UZAeArAAAA5m4mfRoAAAAdAAAAHwAAAIBlvE4DCAAAuSjqOuNf
LYVvbmX/hRkC4EQAAAAd+/eYHwAAAAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAA
AAAAAAAAAAAAAAAAAKTk56KFGQHgKwAAAOZuJn0aAAAAHgAAACAAAACAZbxOAwgAAD41n3V0U4vu
dHdv/4UZAuBEAAAAHfv3mCAAAAABAAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAA
AAAAAAAAAAAAAADXQQmShRkB4CkAAABtpi/XBQAAAB8AAAAhAAAAgGW8TgEEAAArAhh/OgDeUTH/
//+FGQLgRAAAAB3795ghAAAAAQAAAMBBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAA
AAAAAAAAAAAA2GzYQoUZAeApAAAAbaYv1wUAAAAgAAAAIgAAAIBlvE4BBAAAAvJ0d4BR18gy////
hRkC4EQAAAAd+/eYIgAAAAEAAADAQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAA
AAAAAAAAALxZCgSFGQHgKQAAAG2mL9cFAAAAIQAAACMAAACAZbxOAQQAAPwvuUIWYdC/M////4UZ
AuBEAAAAHfv3mCMAAAABAAAAwEEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAA
AAAAAABgtbs5hRkB4CsAAADmbiZ9IQAAACIAAAAkAAAAgGW8TgMIAAA8+Ovo418thW9uZf+FGQLg
RAAAAB3795gkAAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAA
AAAAp/LOZIUZAeArAAAA5m4mfSEAAAAjAAAAJQAAAIBlvE4DCAAAwiUm3XRTi+50d2//hRkC4EQA
AAAd+/eYJQAAAAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAA
AHsef1mFGQHgKwAAAOZuJn0iAAAAJAAAACYAAACAZbxOAwgAAI1C6PjjXy2Fb25l/4UZAuBEAAAA
Hfv3mCYAAAABAAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAAf
K60fhRkB4CsAAADmbiZ9IgAAACUAAAAnAAAAgGW8TgMIAABznyXNdFOL7nR3b/+FGQLgRAAAAB37
95gnAAAAAQAAAICBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAw8cc
IoUZAeArAAAA5m4mfSMAAAAmAAAAKAAAAIBlvE4DCAAAto8e8+NfLYVvbmX/hRkC4EQAAAAd+/eY
KAAAAAEAAACAgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAHYh96SF
GQHgKwAAAOZuJn0jAAAAJwAAACkAAACAZbxOAwgAAEhS08Z0U4vudHdv/4UZAuBEAAAAHfv3mCkA
AAABAAAAgIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAACqzUaZ

michaelolbrich commented 5 years ago

did you use master for this? The permissions are still different and the umask commit in master should fix that. If I set the umask explicitly to 0077 then the image is identical to the base64 encoded one.

daym commented 5 years ago

No, but the latest release with that umask patch applied.

I'll try master next just to be sure.

daym commented 5 years ago

Tried with master - it's unchanged.

About your comment on Dec 13, 2018, are you sure that is the jffs2reader output of the correct image? (i.e. not the one that I produced)

michaelolbrich commented 5 years ago

Right, of course. The output should look like this:

drwxr-xr-x 1    0        0                0 /baz/
drwxr-xr-x 1    0        0                0 /baz/1/
-rw-r--r-- 1    0        0                0 /baz/1/one 
-rw-r--r-- 1    0        0                0 /baz/1/two 
drwxr-xr-x 1    0        0                0 /baz/2/
-rw-r--r-- 1    0        0                0 /baz/2/one 
-rw-r--r-- 1    0        0                0 /baz/2/two 
drwxr-xr-x 1    0        0                0 /baz/3/
-rw-r--r-- 1    0        0                0 /baz/3/one 
-rw-r--r-- 1    0        0                0 /baz/3/two 
drwxr-xr-x 1    0        0                0 /foo/
drwxr-xr-x 1    0        0                0 /foo/1/
-rw-r--r-- 1    0        0                0 /foo/1/one 
-rw-r--r-- 1    0        0                0 /foo/1/two 
drwxr-xr-x 1    0        0                0 /foo/2/
-rw-r--r-- 1    0        0                0 /foo/2/one 
-rw-r--r-- 1    0        0                0 /foo/2/two 
drwxr-xr-x 1    0        0                0 /foo/3/
-rw-r--r-- 1    0        0                0 /foo/3/one 
-rw-r--r-- 1    0        0                0 /foo/3/two 
drwxr-xr-x 1    0        0                0 /with spaces/
drwxr-xr-x 1    0        0                0 /with spaces/1/
-rw-r--r-- 1    0        0                0 /with spaces/1/one 
-rw-r--r-- 1    0        0                0 /with spaces/1/two 
drwxr-xr-x 1    0        0                0 /with spaces/2/
-rw-r--r-- 1    0        0                0 /with spaces/2/one 
-rw-r--r-- 1    0        0                0 /with spaces/2/two 
drwxr-xr-x 1    0        0                0 /with spaces/3/
-rw-r--r-- 1    0        0                0 /with spaces/3/one 
-rw-r--r-- 1    0        0                0 /with spaces/3/two

michaelolbrich commented 5 years ago

A few more things to check: What's the output of the following commands:

umask
mkdir foo
ls -l foo
umask 0022
umask
mkdir bar
ls -l bar

Please run this on the same filesystem as the make check. And what kind of filesystem is that anyways?

daym commented 5 years ago

0022
drwxr-xr-x 2 nixbld nixbld 4096 Dec 15 20:29 foo
0022
drwxr-xr-x 2 nixbld nixbld 4096 Dec 15 20:29 bar

(I used "ls -ld" because that's probably what you meant?)

And what kind of filesystem is that anyways?

/dev/sda1 on /tmp/guix-build-genimage-10.drv-0 type ext4 (rw,relatime)

daym commented 5 years ago

For reference, this is the exact package definition I used:

; FAIL: mke2fs [free space], jffs2 [checksum], ubi [size].
(define-public genimage
  (package
    (name "genimage")
    (version "10")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "https://github.com/pengutronix/genimage.git")
                    ;(commit "f3bdfaa34c1ca66131ee079c85a68cdac55dbff0")
                    (commit (string-append "v" version))))
              (file-name (string-append name "-" version "-checkout"))
              (sha256
               (base32
                "1zdy81aaf1cmzmhn5q13gn6r294963v673ciffk21p3hk4zqc2q0"))))
    (build-system gnu-build-system)
    (arguments
     `(#:tests? #t
       #:phases
       (modify-phases %standard-phases
         (add-after 'unpack 'guixify
           (lambda* (#:key inputs #:allow-other-keys)
             ;; Kinda dirty because it also renames the command-line option.
             (substitute* '("test/basic-images.test"
                            "config.c"
                            "image-vfat.c")
              ;; We don't provide "mkdosfs" in dosfstools.
              (("mkdosfs") "mkfs.fat")
              ;; Work around bug in sharness.sh.
              (("dd,mkfs\\.fat,mcopy")
               "dd,mkfs_fat,mcopy"))
             (substitute* "test/basic-images.test"
              ;; Should be in the next upstream release.
              (("qemu_img") "qemu-img")
              ;; FIXME: Figure out whether that's needed.
              (("setup_data[(][)][ ][{]") "setup_data() {\n\tumask 0022"))
             (substitute* "util.c"
              (("\"/bin/sh\"")
               (string-append "\"" (assoc-ref inputs "bash") "/bin/sh\"")))
             ;; We don't have /etc/passwd so uid 0 is not known as "root".
             ;; Thus patch it out.
             (substitute* '("test/ext2test.dump"
                            "test/ext3test.dump"
                            "test/ext4test.dump"
                            "test/mke2fs.dump")
              (("root") "unknown"))
             ;(mkdir-p "trash directory.basic-images.test/tmp")
             #t))
         (add-before 'check 'setenv-check
           (lambda _
             ;(umask #o0022)
             ;; Our container doesn't provide access to /etc/mtab
             (setenv "EXT2FS_NO_MTAB_OK" "1")
             ;; Make test reproducible
             (setenv "GENIMAGE_MKFJFFS2" "mkfs.jffs2 -U")
             (setenv "GENIMAGE_MKE2FS" "mke2fs -E no_copy_xattrs")
             (write "XXX")
             (newline)
             (call-with-output-file "x.sh"
               (lambda (port)
                 (format port "#
umask
mkdir foo
ls -ld foo
umask 0022
umask
mkdir bar
ls -ld bar
mount
")))
             (system* "bash" "x.sh")
             #t)))))
    (native-inputs
     ;; TODO: Add cramfs.
     `(("autoconf" ,autoconf)
       ("automake" ,automake)
       ("cdrkit-libre" ,cdrkit-libre) ; FIXME: Remove for now.
       ("coreutils" ,coreutils) ; chmod ; note: called by final executable.
       ("cpio" ,cpio) ; for the tests
       ("dosfstools" ,dosfstools) ; for the tests
       ("dtc" ,dtc) ; for the tests
       ("e2fsprogs" ,e2fsprogs) ; for the tests
       ("fakeroot" ,fakeroot) ; for the tests
       ("fdisk" ,fdisk) ; for the tests
       ("genext2fs" ,genext2fs) ; for the tests
       ("mtd-utils" ,mtd-utils) ; for the tests
       ("mtools" ,mtools) ; for the tests
       ("pkg-config" ,pkg-config)
       ("qemu" ,qemu-minimal) ; for the tests
       ("squashfs-tools" ,squashfs-tools) ; for the tests
       ("u-boot-tools" ,u-boot-tools) ; for the tests
       ("util-linux" ,util-linux))) ; for the tests
    (inputs
     `(("bash" ,bash)
       ("libconfuse" ,libconfuse)))
    (synopsis "Create Flash images according to specification")
    (description "@command{genimage} creates Flash images according to a
specification file.")
    (home-page "https://github.com/pengutronix/genimage")
    (license license:gpl2)))

michaelolbrich commented 5 years ago

Right, ls -ld ... is what I meant. So you get the correct permissions here. So why do the test fail?

Can you add this at the end of setup_data() and check the result in test-suite.log:

find "${root_orig}" -print0 | xargs -0 ls -ld

daym commented 5 years ago

drwxr-xr-x 6 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root
drwxr-xr-x 5 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/1
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/1/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/1/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/2
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/2/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/2/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/3
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/3/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/bar/3/two
drwxr-xr-x 5 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/1
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/1/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/1/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/2
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/2/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/2/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/3
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/3/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/baz/3/two
drwxr-xr-x 5 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/1
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/1/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/1/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/2
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/2/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/2/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/3
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/3/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/foo/3/two
drwxr-xr-x 5 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/1
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/1/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/1/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/2
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/2/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/2/two
drwxr-xr-x 2 0 0 4096 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/3
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/3/one
-rw-r--r-- 1 0 0    0 Nov 11  2011 /tmp/guix-build-genimage-10.drv-0/genimage-10-checkout/trash directory.basic-images.test/root/with spaces/3/two

michaelolbrich commented 5 years ago

So, this looks exactly the same, except for root root vs. 0 0 but that's the missing passwd. So why is the content of the images different? I'm out of ideas.

daym commented 5 years ago

With the newest version, I've used jffs2reader test.jffs2 > D and then diffed D with your output above - I get no differences - however, the test still fails.

Could you send me an image file that passes the tests for comparison?

daym commented 5 years ago

Any news on this? If I got the working image it would be easy to debug this test failure. Otherwise it's impossible.

michaelolbrich commented 5 years ago

Here is s base64 dump of my test.jffs2 file.

hRkDIAwAAACxsB7khRkB4CsAAADmbiZ9AQAAAAAAAAACAAAAgGW8TgMEAABVcZ6cuFW+iWJhcv+F
GQLgRAAAAB3795gCAAAAAQAAAO1BAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAA4or66IUZAeArAAAA5m4mfQEAAAABAAAAAwAAAIBlvE4DBAAAq6xTqYrdZYdiYXr/hRkC
4EQAAAAd+/eYAwAAAAEAAADtQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAD5mS9WFGQHgKwAAAOZuJn0BAAAAAgAAAAQAAACAZbxOAwQAANbxA/QzvDJzZm9v/4UZAuBE
AAAAHfv3mAQAAAABAAAA7UEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AAAq4F5lhRkB4DMAAACWEYvoAQAAAAMAAAAFAAAAgGW8TgsEAADHBHoEWr3+93dpdGggc3BhY2Vz
/4UZAuBEAAAAHfv3mAUAAAABAAAA7UEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAA
AAAAAAAAAAD2DO9YhRkB4CkAAABtpi/XAgAAAAQAAAAGAAAAgGW8TgEEAADsgwlOOgDeUTH///+F
GQLgRAAAAB3795gGAAAAAQAAAO1BAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAA
AAAAAAAAkjk9HoUZAeApAAAAbaYv1wIAAAAFAAAABwAAAIBlvE4BBAAAEl7Ee4BR18gy////hRkC
4EQAAAAd+/eYBwAAAAEAAADtQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAA
AAAAAE7VjCOFGQHgKQAAAG2mL9cCAAAABgAAAAgAAACAZbxOAQQAAJF1mCAWYdC/M////4UZAuBE
AAAAHfv3mAgAAAABAAAA7UEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAA
AAD7M2elhRkB4CsAAADmbiZ9BgAAAAcAAAAJAAAAgGW8TgMIAADZcqv4418thW9uZf+FGQLgRAAA
AB3795gJAAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA
TA2BoIUZAeArAAAA5m4mfQYAAAAIAAAACgAAAIBlvE4DCAAAE8ffAXRTi+50d2//hRkC4EQAAAAd
+/eYCgAAAAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAACg4
U+aFGQHgKwAAAOZuJn0HAAAACQAAAAsAAACAZbxOAwgAAKshdVHjXy2Fb25l/4UZAuBEAAAAHfv3
mAsAAAABAAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAD01OLb
hRkB4CsAAADmbiZ9BwAAAAoAAAAMAAAAgGW8TgMIAADWfCUMdFOL7nR3b/+FGQLgRAAAAB3795gM
AAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA4FL3a4UZ
AeArAAAA5m4mfQgAAAALAAAADQAAAIBlvE4DCAAACdF3ReNfLYVvbmX/hRkC4EQAAAAd+/eYDQAA
AAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAADy+RlaFGQHg
KwAAAOZuJn0IAAAADAAAAA4AAACAZbxOAwgAAIz7EM90U4vudHdv/4UZAuBEAAAAHfv3mA4AAAAB
AAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAABYi5QQhRkB4CkA
AABtpi/XAwAAAA0AAAAPAAAAgGW8TgEEAADljLxrOgDeUTH///+FGQLgRAAAAB3795gPAAAAAQAA
AO1BAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA77VyFYUZAeApAAAA
baYv1wMAAAAOAAAAEAAAAIBlvE4BBAAAmkr4PIBR18gy////hRkC4EQAAAAd+/eYEAAAAAEAAADt
QQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAABiSZf6FGQHgKQAAAG2m
L9cDAAAADwAAABEAAACAZbxOAQQAAGSXNQkWYdC/M////4UZAuBEAAAAHfv3mBEAAAABAAAA7UEA
AAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADEftTDhRkB4CsAAADmbiZ9
DwAAABAAAAASAAAAgGW8TgMIAAA0EENm418thW9uZf+FGQLgRAAAAB3795gSAAAAAQAAAKSBAAAA
AAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAy5lRvYUZAeArAAAA5m4mfQ8A
AAARAAAAEwAAAIBlvE4DCAAAys2OU3RTi+50d2//hRkC4EQAAAAd+/eYEwAAAAEAAACkgQAAAAAA
AAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAABd14ICFGQHgKwAAAOZuJn0QAAAA
EgAAABQAAACAZbxOAwgAALNLh5LjXy2Fb25l/4UZAuBEAAAAHfv3mBQAAAABAAAApIEAAAAAAAAA
AAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAAD8/UwhRkB4CsAAADmbiZ9EAAAABMA
AAAVAAAAgGW8TgMIAABNlkqndFOL7nR3b/+FGQLgRAAAAB3795gVAAAAAQAAAKSBAAAAAAAAAAAA
AIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAA3x9EDYUZAeArAAAA5m4mfREAAAAUAAAA
FgAAAIBlvE4DCAAAjodKSONfLYVvbmX/hRkC4EQAAAAd+/eYFgAAAAEAAACkgQAAAAAAAAAAAACA
ZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAALsqlkuFGQHgKwAAAOZuJn0RAAAAFQAAABcA
AACAZbxOAwgAAHBah310U4vudHdv/4UZAuBEAAAAHfv3mBcAAAABAAAApIEAAAAAAAAAAAAAgGW8
ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAABnxid2hRkB4CkAAABtpi/XBAAAABYAAAAYAAAA
gGW8TgEEAAAmO4ROOgDeUTH///+FGQLgRAAAAB3795gYAAAAAQAAAO1BAAAAAAAAAAAAAIBlvE6A
ZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAufKbyIUZAeApAAAAbaYv1wQAAAAXAAAAGQAAAIBl
vE4BBAAA2OZJe4BR18gy////hRkC4EQAAAAd+/eYGQAAAAEAAADtQQAAAAAAAAAAAACAZbxOgGW8
ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAAGUeKvWFGQHgKQAAAG2mL9cEAAAAGAAAABoAAACAZbxO
AQQAABJTPYIWYdC/M////4UZAuBEAAAAHfv3mBoAAAABAAAA7UEAAAAAAAAAAAAAgGW8ToBlvE6A
ZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAABK/izhRkB4CsAAADmbiZ9GAAAABkAAAAbAAAAgGW8TgMI
AADNKRMn418thW9uZf+FGQLgRAAAAB3795gbAAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8
TgAAAAAAAAAAAAAAAAAAAAAAAAAAthUetoUZAeArAAAA5m4mfRgAAAAaAAAAHAAAAIBlvE4DCAAA
sHRDenRTi+50d2//hRkC4EQAAAAd+/eYHAAAAAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4A
AAAAAAAAAAAAAAAAAAAAAAAAAKKTCwaFGQHgKwAAAOZuJn0ZAAAAGwAAAB0AAACAZbxOAwgAAAiS
6SrjXy2Fb25l/4UZAuBEAAAAHfv3mB0AAAABAAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAA
AAAAAAAAAAAAAAAAAAAAAAB+f7o7hRkB4CsAAADmbiZ9GQAAABwAAAAeAAAAgGW8TgMIAACNuI6g
dFOL7nR3b/+FGQLgRAAAAB3795geAAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAA
AAAAAAAAAAAAAAAAAAAAGkpofYUZAeArAAAA5m4mfRoAAAAdAAAAHwAAAIBlvE4DCAAAuSjqOuNf
LYVvbmX/hRkC4EQAAAAd+/eYHwAAAAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAA
AAAAAAAAAAAAAAAAAMam2UCFGQHgKwAAAOZuJn0aAAAAHgAAACAAAACAZbxOAwgAAD41n3V0U4vu
dHdv/4UZAuBEAAAAHfv3mCAAAAABAAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAA
AAAAAAAAAAAAAAC1AzdwhRkB4CkAAABtpi/XBQAAAB8AAAAhAAAAgGW8TgEEAAArAhh/OgDeUTH/
//+FGQLgRAAAAB3795ghAAAAAQAAAO1BAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAA
AAAAAAAAAAAAAj3RdYUZAeApAAAAbaYv1wUAAAAgAAAAIgAAAIBlvE4BBAAAAvJ0d4BR18gy////
hRkC4EQAAAAd+/eYIgAAAAEAAADtQQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAA
AAAAAAAAAGYIAzOFGQHgKQAAAG2mL9cFAAAAIQAAACMAAACAZbxOAQQAAPwvuUIWYdC/M////4UZ
AuBEAAAAHfv3mCMAAAABAAAA7UEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAA
AAAAAAC65LIOhRkB4CsAAADmbiZ9IQAAACIAAAAkAAAAgGW8TgMIAAA8+Ovo418thW9uZf+FGQLg
RAAAAB3795gkAAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAA
AAAAxbDwhoUZAeArAAAA5m4mfSEAAAAjAAAAJQAAAIBlvE4DCAAAwiUm3XRTi+50d2//hRkC4EQA
AAAd+/eYJQAAAAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAA
ABlcQbuFGQHgKwAAAOZuJn0iAAAAJAAAACYAAACAZbxOAwgAAI1C6PjjXy2Fb25l/4UZAuBEAAAA
Hfv3mCYAAAABAAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAAB9
aZP9hRkB4CsAAADmbiZ9IgAAACUAAAAnAAAAgGW8TgMIAABznyXNdFOL7nR3b/+FGQLgRAAAAB37
95gnAAAAAQAAAKSBAAAAAAAAAAAAAIBlvE6AZbxOgGW8TgAAAAAAAAAAAAAAAAAAAAAAAAAAoYUi
wIUZAeArAAAA5m4mfSMAAAAmAAAAKAAAAIBlvE4DCAAAto8e8+NfLYVvbmX/hRkC4EQAAAAd+/eY
KAAAAAEAAACkgQAAAAAAAAAAAACAZbxOgGW8ToBlvE4AAAAAAAAAAAAAAAAAAAAAAAAAABRjyUaF
GQHgKwAAAOZuJn0jAAAAJwAAACkAAACAZbxOAwgAAEhS08Z0U4vudHdv/4UZAuBEAAAAHfv3mCkA
AAABAAAApIEAAAAAAAAAAAAAgGW8ToBlvE6AZbxOAAAAAAAAAAAAAAAAAAAAAAAAAADIj3h7

daym commented 5 years ago

Thanks!

The reason is the "cp -a" which copies the "root" to "tmp".

dannym@dayas /tmp/qq$ LC_ALL=C ls -lrt root
total 16
drwxr-xr-x 5 guixbuilder01 guixbuild 4096 Nov 11  2011 'with spaces'
drwxr-xr-x 5 guixbuilder01 guixbuild 4096 Nov 11  2011  foo
drwxr-xr-x 5 guixbuilder01 guixbuild 4096 Nov 11  2011  baz
drwxr-xr-x 5 guixbuilder01 guixbuild 4096 Nov 11  2011  bar
dannym@dayas /tmp/qq$ LC_ALL=C ls -lrt tmp
total 4
drwx------ 6 guixbuilder01 guixbuild 4096 Nov 11  2011 root

umask is 022 in setup_data (and should be inherited). The "cp -a" is executed using the guixbuilder01 user.

If I replace it by "cp -r --preserve=ownership,timestamps,links", it works (also as guixbuilder01 user). As soon as I add "mode", it breaks the tests (mode of everything in tmp is then without g and o modes).

If I execute it as root it works regardless.

michaelolbrich commented 5 years ago

Can you try the following as user guixbuilder01 with fakeroot:

set umaks to 022
create a empty file (make sure the mode is 644)
copy it to some other name with -a and -r --preserve=ownership,timestamps,links
check the mode of the resulting files

Do the third step with strace and take a look at the output. Most of it should be very similar. The last ~30 lines are probably the interesting once. Maybe we can see what happens.

daym commented 5 years ago

I completely forgot that there's a fakeroot in there! If I remove fakeroot and don't patch the "cp" (i.e. "make TEST_LOG_COMPILER= check"), all tests succeed in the container!

I'll be trying your suggested steps next.

daym commented 5 years ago

In the container:

             (call-with-output-file "x.sh"
              (lambda (port)
                (display
"#!/bin/sh
umask 022
touch empty
chmod 0644 empty
cp -a empty empty-a
cp -r --preserve=ownership,timestamps,links empty empty-r
ls -l -n empty*
getfacl -n empty*
exit 1
"
port)))
             (invoke "fakeroot" "bash" "x.sh")

-rw-r--r--  1     0     0 0 Feb 15 14:28 empty
-rw-------+ 1 30002 30000 0 Feb 15 14:28 empty-a
-rw-r--r--  1     0     0 0 Feb 15 14:28 empty-r
# file: empty
# owner: 0
# group: 0
user::rw-
group::r--
other::r--

# file: empty-a
# owner: 30002
# group: 30000
user::rw-
group::r--
other::r--

# file: empty-r
# owner: 0
# group: 0
user::rw-
group::r--
other::r--

(fakeroot is 1.23 with no patches)

daym commented 5 years ago

I've submitted a merge request for a unit test in fakeroot catching this problem: https://salsa.debian.org/clint/fakeroot/merge_requests/1

michaelolbrich commented 5 years ago

I think your fakeroot is built without acl support and that causes this problem.

daym commented 5 years ago

Aaaargh, you are entirely right. I've tried the test https://salsa.debian.org/clint/fakeroot/merge_requests/1, it fails without acl support and succeeds with acl support.

Thanks!

I hope they take the patch--that would save others a lot of time.