ckruse / CFPropertyList

Read, write and manipulate both binary and XML property lists as defined by apple
MIT License
212 stars 47 forks source link

BigEndian platform support #46

Closed voxik closed 7 years ago

voxik commented 7 years ago

Trying to run the test suite of CFProperty list on PPC64, which is BigEndian platfor, fails with following errors:

+ ruby -Ilib:test -e 'Dir.glob "./test/**/test_*.rb", &method(:require)'
Run options: --seed 51868
# Running:
FFF..FF.FFF...................................................................................
Finished in 13.451500s, 6.9881 runs/s, 14917.2949 assertions/s.
  1) Failure:
TestReal#test_read_float [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_real.rb:14]:
Expected: 1.5
  Actual: 6.896490392174587e-41
  2) Failure:
TestReal#test_write_double [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_real.rb:26]:
--- expected
+++ actual
@@ -1 +1 @@
-"bplist00#?\xF8\x00\x00\x00\x00\x00\x00\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
+"bplist00#\x00\x00\x00\x00\x00\x00\xF8?\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
  3) Failure:
TestReal#test_read_double [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_real.rb:19]:
Expected: 1.5
  Actual: 3.13984e-319
  4) Failure:
TestDate#test_write_date_1900 [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_date.rb:32]:
--- expected
+++ actual
@@ -1 +1 @@
-"bplist003\xC1\xE7\xBF3\xC8\x00\x00\x00\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
+"bplist003\x00\x00\x00\xC83\xBF\xE7\xC1\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
  5) Failure:
TestDate#test_write_date_from_date_object [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_date.rb:39]:
--- expected
+++ actual
@@ -1 +1 @@
-"bplist003A\xB3{Z\x80\x00\x00\x00\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
+"bplist003\x00\x00\x00\x80Z{\xB3A\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
  6) Failure:
TestDate#test_write_date_epoch [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_date.rb:25]:
--- expected
+++ actual
@@ -1 +1 @@
-"bplist003\xC1\xCD'\xE4@\x00\x00\x00\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
+"bplist003\x00\x00\x00@\xE4'\xCD\xC1\b\x00\x00\x00\x00\x00\x00\x01\x01\x00\x00\x00\x00\x00\x00\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x11"
  7) Failure:
TestDate#test_read_date_1900 [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_date.rb:18]:
Expected: 1900-01-01 12:00:00 UTC
  Actual: 2001-01-01 00:00:00 +0000
  8) Failure:
TestDate#test_read_date_epoch [/builddir/build/BUILD/CFPropertyList-2.3.3/usr/share/gems/gems/CFPropertyList-2.3.3/test/test_date.rb:13]:
Expected: 1970-01-01 00:00:00 +0000
  Actual: 2001-01-01 00:00:00 +0000
94 runs, 200660 assertions, 8 failures, 0 errors, 0 skips
ckruse commented 7 years ago

Hu, this is surprising for me. I have to set up a testing environment to debug this.

voxik commented 7 years ago

So I was going to submit this patch:

diff --git a/lib/cfpropertylist/rbBinaryCFPropertyList.rb b/lib/cfpropertylist/rbBinaryCFPropertyList.rb
index 15301c3..9015aa7 100644
--- a/lib/cfpropertylist/rbBinaryCFPropertyList.rb
+++ b/lib/cfpropertylist/rbBinaryCFPropertyList.rb
@@ -166,9 +172,9 @@ module CFPropertyList
         when 1 # 2 byte float? must be an error
           raise CFFormatError.new("got #{length+1} byte float, must be an error!")
         when 2 then
-          buff.reverse.unpack("f")[0]
+          buff.reverse.unpack("e")[0]
         when 3 then
-          buff.reverse.unpack("d")[0]
+          buff.reverse.unpack("E")[0]
         else
           fail "unexpected length: #{length}"
         end
@@ -190,9 +196,9 @@ module CFPropertyList
         when 1 then # 2 byte CFDate is an error
           raise CFFormatError.new("#{length+1} byte CFDate, error")
         when 2 then
-          buff.reverse.unpack("f")[0]
+          buff.reverse.unpack("e")[0]
         when 3 then
-          buff.reverse.unpack("d")[0]
+          buff.reverse.unpack("E")[0]
         end,
         CFDate::TIMESTAMP_APPLE
       )
@@ -498,7 +504,7 @@ module CFPropertyList

     # Codes a real value to binary format
     def real_to_binary(val)
-      Binary.type_bytes(0b0010,3) << [val].pack("d").reverse
+      Binary.type_bytes(0b0010,3) << [val].pack("E").reverse
     end

     # Converts a numeric value to binary and adds it to the object table
@@ -545,7 +551,7 @@ module CFPropertyList
       val = val.getutc.to_f - CFDate::DATE_DIFF_APPLE_UNIX # CFDate is a real, number of seconds since 01/01/2001 00:00:00 GMT

       @object_table[@written_object_count] =
-        (Binary.type_bytes(0b0011, 3) << [val].pack("d").reverse)
+        (Binary.type_bytes(0b0011, 3) << [val].pack("E").reverse)

       @written_object_count += 1
       @written_object_count - 1

Where I am replacing the "native format" specifiers for "little-endian" and hence the test suite passes everywhere.

I thought I should also check what the format really should be, but checking Wikipedia 1 and sources 2 it seems this patch is wrong. But not only the patch, but also all the ".plist" files used in tests!!!

voxik commented 7 years ago

Let me put this explicitly, the .plist files used in tests seems to be little endian, where they should be big endian according to specs. It did not pop up earlier, since the "native format" was used ... I just can't explain that nobody noticed yet, but probably nobody is using the binary .plist files?

voxik commented 7 years ago

This should be the patch following the specifications but failing everywhere due to wrong .plist files:

diff --git a/lib/cfpropertylist/rbBinaryCFPropertyList.rb b/lib/cfpropertylist/rbBinaryCFPropertyList.rb
index 15301c3..9015aa7 100644
--- a/lib/cfpropertylist/rbBinaryCFPropertyList.rb
+++ b/lib/cfpropertylist/rbBinaryCFPropertyList.rb
@@ -166,9 +172,9 @@ module CFPropertyList
         when 1 # 2 byte float? must be an error
           raise CFFormatError.new("got #{length+1} byte float, must be an error!")
         when 2 then
-          buff.reverse.unpack("f")[0]
+          buff.reverse.unpack("g")[0]
         when 3 then
-          buff.reverse.unpack("d")[0]
+          buff.reverse.unpack("G")[0]
         else
           fail "unexpected length: #{length}"
         end
@@ -190,9 +196,9 @@ module CFPropertyList
         when 1 then # 2 byte CFDate is an error
           raise CFFormatError.new("#{length+1} byte CFDate, error")
         when 2 then
-          buff.reverse.unpack("f")[0]
+          buff.reverse.unpack("g")[0]
         when 3 then
-          buff.reverse.unpack("d")[0]
+          buff.reverse.unpack("G")[0]
         end,
         CFDate::TIMESTAMP_APPLE
       )
@@ -498,7 +504,7 @@ module CFPropertyList

     # Codes a real value to binary format
     def real_to_binary(val)
-      Binary.type_bytes(0b0010,3) << [val].pack("d").reverse
+      Binary.type_bytes(0b0010,3) << [val].pack("G").reverse
     end

     # Converts a numeric value to binary and adds it to the object table
@@ -545,7 +551,7 @@ module CFPropertyList
       val = val.getutc.to_f - CFDate::DATE_DIFF_APPLE_UNIX # CFDate is a real, number of seconds since 01/01/2001 00:00:00 GMT

       @object_table[@written_object_count] =
-        (Binary.type_bytes(0b0011, 3) << [val].pack("d").reverse)
+        (Binary.type_bytes(0b0011, 3) << [val].pack("G").reverse)

       @written_object_count += 1
       @written_object_count - 1
voxik commented 7 years ago

I can send PR with the patch, but not sure how to check the .plist files, since I am not Mac user ...

ckruse commented 7 years ago

All PList files have been tested to work with the macOS plist editor (and I just checked again). I'm not sure if big endian is really the right format, although I see your arguments; maybe I should do further investigation. Yes, please create a pull request!

voxik commented 7 years ago

So I'll create two PR, on LE and the other BE and you can pick the right one ;)

ckruse commented 7 years ago

Thank you very much!

Via Mobile, deshalb kurz. Nicht unhöflich gemeint!

Am 24.11.2016 um 12:30 schrieb Vít Ondruch notifications@github.com:

So I'll create two PR, on LE and the other BE and you can pick the right one ;)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

ckruse commented 7 years ago

Little endian shows the right results here…

ckruse commented 7 years ago

Yeah, it's definitely the fix-le branch…

ckruse commented 7 years ago

Thank you very much for your patch! I merged the pull request and closed the other and I just released a new version