rand: generate UTF-8 strings for the string ('s') type

[mrc0mmand]

So far we were generating string only in the ASCII range, but D-Bus supports UTF-8 strings as well, so let's fix that.

Resolves: #108

[mrc0mmand]

@evverx I wonder if we should limit this to only printable characters (as it was done for the ASCII range), or keep it as it is (by still omitting the bottom 32 control characters).

In the first case, I guess making use of g_unichar_isprint() would help. In the second case (i.e. using the almost whole 0x20 - 0x10FFFF range as it is now) we'd have to tweak the logging a bit, to avoid causing mayhem on the terminal or in the log files.

[mrc0mmand]

Ah, nevermind, g_variant_print() takes care of that:

   -- Signature: (isaaai(y(b(n(q(iua{ov})v)o))x(dh))a{t(bov)})
   -- Value: (540456405, "|􃌜i˫\U0009e7c3췴Ӫ\U0009d9cbຕX^z\U000a96fc鶁暐ࢬڠ䇖􅍼٩\U0003f443󵜽\U000d2975\U000dc1fcp铳ꅖa\U000ccbfaڝ,\U0007d0e9I\U00051bde@𰖶\U000dd924\U0008980e\U000df20d\U0008cb25ɼ\U000c6fe4쒿󸕎 uòw慈Į\U000e8cf7\U0008840b\U0006
04c4펅1ᅙ􆂲붏霖\U0003f52cbﶓ䫫N\U0007291f \U0008d630KǢៗ蟂,ۊ\u2e7bǢՅTL󼧰1蹬쉤[󷥆\U0003212b݇矌ח¦붣㐝밑¢\U00018ffe\U000495ab鞰\U0005a57fǞ\U0006daf3ʠύ䃿𤚧\U000386ba\ua63cÅⒽ\U000e6ca0􆪪&\U000139b5ޑ\U00055b9di\U0004eeb4蔄K0⌬悧쇗ﲂ\U000e2fa3m\U0004a6d
9璛\U0006a3bb\U0006fff8W碡5醟ʮ\U000ce2ec7-ᗻ\U000d1adbћ\U000a1f46\U0007a684\"鄖p6Ѭοyᕒʸ铤Л璛ﴴ⫸\U000c6260ѭధ晁ԝ\U0009453e\U00036e97z\U000e94fa&$\U000834ff칽闈\U00076909\u05fbʨﭪO㩝P̠?┾ɹվ궜v}@`eِ²\U0005b126r侦˿\U0005a97bT흈ƕ\U000c11cdﺕ⚉ߔ=钒ץ\U0
00d732a𪚜\U000e3ebf\U0006ff5a\U000d687d𰢍{ⷉ\U000ee6f3󵥴ΟəM˫\U000a8104ѻը`IW䲙\U000ed6e6dw\U000b9260\U000dae64噀󿭑⏼ޟ$I삺\u07b2Z\U00076736Ю1lɾ˦\U00093cf5ۖˆ%۲ݒН卍\U0004cc05𐫘[;ɠ󻽮\U0007bc4eҶߺጱ꩕w\U000c139aʟ`2ۼ%λ醢{`弶ޅbದ굱ש\"(愾y㦷㚟釅,吴ơ⮗Xвqiػ􍌊L
\U000ec438ǵ=8]줦\U000c0429\u05cbϚВYľĽ쥪\u07fcҀ\U000a08a5H糂шuZ4͂쨢体H雭􋏝\U000e6cadP...

so I guess we can leave it as is, for now at least.

[evverx]

Looks like with this patch applied dfuzzer generates only valid UTF-8 string. I think it should generate bogus strings as well.

[evverx]

I wonder if it would make sense to put bytes from [1..255] in those strings and hope some of them are UTF-8 :-)

Having said that it seems this patch should be good to go because it seems it kind of extends the current behavior of dfuzzer.

[mrc0mmand]

Looks like with this patch applied dfuzzer generates only valid UTF-8 string. I think it should generate bogus strings as well.

That would definitely make sense, but it might require some tinkering, thanks to all the UTF-8 validation glib does :-)

That said, I'll look into that later (and hopefully prepare a new PR for that).

[mrc0mmand]

Oh well, this might not be as easy as I originally anticipated.

I managed to tell glib to create a string variant with a corrupted UTF-8 string using

diff --git a/src/fuzz.c b/src/fuzz.c
index f53a49f..d34b537 100644
--- a/src/fuzz.c
+++ b/src/fuzz.c
@@ -153,6 +153,7 @@ GVariant *df_generate_random_basic(const GVariantType *type, guint64 iteration)
         else if (g_variant_type_equal(type, G_VARIANT_TYPE_DOUBLE))
                 return g_variant_new(ssig, df_rand_gdouble(iteration));
         else if (g_variant_type_equal(type, G_VARIANT_TYPE_STRING)) {
+                g_autoptr(GBytes) bytes = NULL;
                 g_autoptr(char) str = NULL;

                 if (df_rand_string(&str, iteration) < 0) {
@@ -160,7 +161,12 @@ GVariant *df_generate_random_basic(const GVariantType *type, guint64 iteration)
                         return NULL;
                 }

-                return g_variant_new(ssig, str);
+                /* Since we want to be able to create corrupted UTF-8 strings as well,
+                 * let's re-implement the internal g_variant_new_from_trusted() function
+                 * so we can bypass the internal consistency check (g_utf8_validate())
+                 * of the g_variant_new_string() function */
+                bytes = g_bytes_new(str, strlen(str) + 1);
+                return g_variant_new_from_bytes(G_VARIANT_TYPE_STRING, bytes, TRUE);
         } else if (g_variant_type_equal(type, G_VARIANT_TYPE_OBJECT_PATH)) {
                 g_autoptr(char) obj_path = NULL;

diff --git a/src/rand.c b/src/rand.c
index 0ece784..6999817 100644
--- a/src/rand.c
+++ b/src/rand.c
@@ -322,6 +322,7 @@ static char *df_rand_random_string(size_t size)
                 return NULL;

         g_autoptr(GString) str = NULL;
+        char *ret;
         size_t str_size;

         str_size = size - 1;
@@ -341,7 +342,15 @@ static char *df_rand_random_string(size_t size)

         /* "Steal" the internal C-string from the GString class to avoid another
          * unnecessary allocation */
-        return g_steal_pointer(&str->str);
+        ret = g_steal_pointer(&str->str);
+
+        if (rand() % 5 == 0) {
+                for (size_t i = 0; i < strlen(ret) - 1; i++)
+                        if (rand() % 10 == 0)
+                                ret[i] |= 0x31;
+        }
+
+        return ret;
 }

 /**

but unfortunately there are other consistency checks out of my reach:

$ build/dfuzzer -v  -n org.freedesktop.dfuzzerServer -o /org/freedesktop/dfuzzerObject -i org.freedesktop.dfuzzerInterface -t df_hello
Cannot open suppression file './dfuzzer.conf'
Cannot open suppression file '/home/mrc0mmand/.dfuzzer.conf'
Loading suppressions from file '/etc/dfuzzer.conf'
Found suppressions for bus: 'org.freedesktop.dfuzzerServer' 
Loaded suppression for method: *:*:df_hang (Intentionally hangs the server to test timeout handling)
[SESSION BUS]                                                                                                          
Error while activating 'org.freedesktop.dfuzzerServer': The name is not activatable.
Error while calling method 'GetConnectionUnixProcessID': GDBus.Error:org.freedesktop.DBus.Error.NameHasNoOwner: The connection does not exist
Couldn't get the PID of the tested process                                                                             
[SYSTEM BUS]                                    
[PROCESS: /usr/bin/dfuzzer-test-server]     
[CONNECTED TO PID: 2773983]
Object: /org/freedesktop/dfuzzerObject                                                                                 
 Interface: org.freedesktop.dfuzzerInterface                                                                           
  [M] df_hello...**                  
GLib-GIO:ERROR:../gio/gdbusmessage.c:2498:append_value_to_blob: assertion failed: (g_utf8_validate (v, -1, &end) && (end == v + len))
Bail out! GLib-GIO:ERROR:../gio/gdbusmessage.c:2498:append_value_to_blob: assertion failed: (g_utf8_validate (v, -1, &end) && (end == v + len))
Aborted (core dumped) 

#0  0x00007f9ee0fb0c3c in __pthread_kill_implementation () at /lib64/libc.so.6
#1  0x00007f9ee0f609c6 in raise () at /lib64/libc.so.6
#2  0x00007f9ee0f4a7f4 in abort () at /lib64/libc.so.6
#3  0x00007f9ee1141d18 in g_assertion_message_expr[cold] () at /lib64/libglib-2.0.so.0
#4  0x00007f9ee11a1bae in g_assertion_message_expr () at /lib64/libglib-2.0.so.0
#5  0x00007f9ee13cb9ec in append_value_to_blob () at /lib64/libgio-2.0.so.0
#6  0x00007f9ee13cedab in g_dbus_message_to_blob () at /lib64/libgio-2.0.so.0
#7  0x00007f9ee13c25ab in g_dbus_connection_send_message_unlocked.lto_priv () at /lib64/libgio-2.0.so.0
#8  0x00007f9ee13c2d27 in g_dbus_connection_send_message_with_reply_unlocked () at /lib64/libgio-2.0.so.0
#9  0x00007f9ee13c2f03 in g_dbus_connection_send_message_with_reply () at /lib64/libgio-2.0.so.0
#10 0x00007f9ee13c3193 in g_dbus_connection_send_message_with_reply_sync () at /lib64/libgio-2.0.so.0
#11 0x00007f9ee13d0e1f in g_dbus_connection_call_sync_internal () at /lib64/libgio-2.0.so.0
#12 0x00007f9ee13d99b6 in g_dbus_proxy_call_sync_internal () at /lib64/libgio-2.0.so.0
#13 0x00007f9ee13d9b68 in g_dbus_proxy_call_sync () at /lib64/libgio-2.0.so.0
#14 0x00000000004071ba in df_fuzz_call_method (method=0x7fffbb041db0, value=0x7f9ed4005960) at ../src/fuzz.c:576
#15 0x0000000000406d97 in df_fuzz_test_method
    (method=0x7fffbb041db0, name=0x7fffbb0430f1 "org.freedesktop.dfuzzerServer", obj=0x7fffbb043112 "/org/freedesktop/dfuzzerObject", intf=0x7fffbb043134 "org.freedesktop.dfuzzerInterface", pid=2773983, execute_cmd=0x0, iterations=64)
    at ../src/fuzz.c:480
#16 0x0000000000404224 in df_fuzz (dcon=0xe21140, name=0x7fffbb0430f1 "org.freedesktop.dfuzzerServer", object=0x7fffbb043112 "/org/freedesktop/dfuzzerObject", interface=0x7fffbb043134 "org.freedesktop.dfuzzerInterface")
    at ../src/dfuzzer.c:501
#17 0x0000000000403383 in df_process_bus (bus_type=G_BUS_TYPE_SYSTEM) at ../src/dfuzzer.c:186
#18 0x0000000000402ffd in main (argc=10, argv=0x7fffbb042048) at ../src/dfuzzer.c:113

I guess this could also be bypassed using some $LD_PRELOAD magic, but that's not very nice :-)

I also tried to use -Wl,-wrap,g_utf8_validate with corresponding __wrap_g_utf8_validate() function (which always returns TRUE), but to no avail. I guess this applies only to "our" stuff, not to the shared libraries as well.

[evverx]

I think on the bright side given that dfuzzer doesn't crash with this PR merged it's safe to say that it generates valid UTF-8 strings and that's what this PR is about as far as I understand :-)