Testing with cpp_ reference examples

[mingodad]

Using a Lua script shown bellow we can test cake against the examples in https://en.cppreference.com/w/c that can be downloaded for offline view here https://en.cppreference.com/w/Cppreference:Archives .

local base_dir = "/home/mingo/.local/share/Zeal/Zeal/docsets/C.docset/Contents/Resources/Documents/en.cppreference.com/w/c/";

local cCodeFName = "/tmp/cCode.c";
--local cxx_cmd = "../build/src/frontend/cxx -fsyntax-only -toolchain linux " .. cppCodeFName;
--local cxx_cmd = "/usr/bin/time ../build-release/src/frontend/cxx -fsyntax-only -toolchain linux " .. cppCodeFName;
--local cxx_cmd = "clang-17-env /usr/bin/time clang++ -std=c++23 -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "clang-17-env /usr/bin/time clang++ -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "gcc-13-env /usr/bin/time g++ -std=c++23 -fsyntax-only " .. cppCodeFName;
--local cxx_cmd = "gcc-13-env /usr/bin/time g++ -fsyntax-only " .. cppCodeFName;
local cxx_cmd = "/usr/bin/time timeout 3s src/cake -Isrc/include " .. cCodeFName;

--<div dir="ltr" class="mw-geshi t-example-code" style="text-align: left;"><div class="c source-c">
local code_re = "<div .-class=\"[^\"t]*t%-example%-[^\"]+\".->.-<div class=\"c source%-c\">(.-)</pre></div>";

local find_cmd = "find " .. base_dir .. " -name '*.html'";
local start_t = os.time();

local html_list = {};
for fname in io.popen(find_cmd, "r"):lines("l") do
    table.insert(html_list, fname);
end
table.sort(html_list);

local count_with_code = 0;
local fname_with_code_failed = {};
for idx, fname in ipairs(html_list) do
    print(fname);

    local html = io.open(fname, "r"):read("a");
    for code in html:gmatch(code_re) do
        code = code:gsub("%b<>", "");
        code = code:gsub("&nbsp;", " ");
        code = code:gsub("&lt;", "<");
        code = code:gsub("&gt;", ">");
        code = code:gsub("&amp;", "&");

        --print("//====Start");
        --print(code);
        --print("//====End");

        local fd = io.open(cCodeFName, "w");
        fd:write(code);
        fd:close();
        count_with_code = count_with_code + 1;
        print("==CHECK==")
        print(cxx_cmd);
        io.stdout:flush();
        local rc, rc_type, rc_code = os.execute(cxx_cmd);
        if rc_code == 0 then
            print("==rc:OK==")
        else
            table.insert(fname_with_code_failed, idx);
            print("==rc:FAILED==")
            print("//====Start");
            print(code);
            print("//====End");
        end
    end

end
local end_t = os.time();
print("elapsed time: ", os.difftime(end_t, start_t));
print("Total files scanned     :", #html_list);
print("Total files with code   :", count_with_code);
print("Total files failed code :", #fname_with_code_failed);
for idx, fname_idx in ipairs(fname_with_code_failed) do
    print(html_list[fname_idx]);
end

[mingodad]

Also trying to test with sqlite3.c we get this:

/usr/bin/time ./cake -Iinclude -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c

Cake 0.7.4

sqlite3-orig/sqlite3.c
sqlite3-orig/sqlite3.c:36582:29: note: you can use "adjacent" "strings"
 36582 |      __asm__ __volatile__ (
       |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:38167:49: error: file sys/types.h not found
 38167 |#include <sys/types.h>   /* amalgamator: keep */
       |                                                ^
sqlite3-orig/sqlite3.c:38167:49: note: dir = 'include'
 38167 |#include <sys/types.h>   /* amalgamator: keep */
       |                                                ^

 1 files in 0.41 seconds
 0 errors 0 warnings 0 notes 
0.32user 0.09system 0:00.42elapsed 99%CPU (0avgtext+0avgdata 212044maxresident)k
0inputs+0outputs (0major+52649minor)pagefaults 0swaps

>/usr/bin/time gcc -fsyntax-only -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c
0.38user 0.04system 0:00.42elapsed 99%CPU (0avgtext+0avgdata 78836maxresident)k
0inputs+0outputs (0major+19596minor)pagefaults 0swaps

[thradams]

The error here is because the include dir (gcc and system headers) are not configured

     error: file sys/types.h not found

Create a cakeconfig.h file at same path of cake executable.

//Edit this file adding inside pragma dir "" .. the result of // echo | gcc -E -Wp,-v -

pragma dir "add dir here..."

pragma dir "add dir here..."

...

Next time you run cake

/cake -Iinclude -DSQLITE_THREADSAFE=0 sqlite3-orig/sqlite3.c

Then the includes configured inside cakeconfig.h will be used. Otherwise each time you need add -I ...

(The same on windows)

See "Include directories" http://thradams.com/cake/manual.html

On windows. https://youtu.be/gO6DfsU7qj4

I have created this repository with real source code to be checked https://github.com/thradams/cake-tests but It has been a while I don't check.

[mingodad]

Using valgrind and cakeconfig.h:

valgrind ./cake -Iinclude sqlite3-orig/sqlite3.c
==27122== Memcheck, a memory error detector
==27122== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==27122== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==27122== Command: ./cake -Iinclude sqlite3-orig/sqlite3.c
==27122== 
==27122== Invalid read of size 4
==27122==    at 0x111ECC: pp_tokens_opt (tokenizer.c:2681)
==27122==    by 0x1141B6: control_line (tokenizer.c:3248)
==27122==    by 0x1166DB: group_part (tokenizer.c:4339)
==27122==    by 0x10FA31: group_opt (tokenizer.c:1818)
==27122==    by 0x116798: preprocessor (tokenizer.c:4364)
==27122==    by 0x116A6A: include_config_header (tokenizer.c:4443)
==27122==    by 0x13E5D7: compile_one_file (parser.c:6663)
==27122==    by 0x13F3E7: compile (parser.c:6974)
==27122==    by 0x14CA17: main (main.c:58)
==27122==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
==27122== 
==27122== 
==27122== Process terminating with default action of signal 11 (SIGSEGV)
==27122==  Access not within mapped region at address 0x0
==27122==    at 0x111ECC: pp_tokens_opt (tokenizer.c:2681)
==27122==    by 0x1141B6: control_line (tokenizer.c:3248)
==27122==    by 0x1166DB: group_part (tokenizer.c:4339)
==27122==    by 0x10FA31: group_opt (tokenizer.c:1818)
==27122==    by 0x116798: preprocessor (tokenizer.c:4364)
==27122==    by 0x116A6A: include_config_header (tokenizer.c:4443)
==27122==    by 0x13E5D7: compile_one_file (parser.c:6663)
==27122==    by 0x13F3E7: compile (parser.c:6974)
==27122==    by 0x14CA17: main (main.c:58)
==27122==  If you believe this happened as a result of a stack
==27122==  overflow in your program's main thread (unlikely but
==27122==  possible), you can try to increase the size of the
==27122==  main thread stack using the --main-stacksize= flag.
==27122==  The main thread stack size used in this run was 8388608.
==27122== 
==27122== HEAP SUMMARY:
==27122==     in use at exit: 77,033 bytes in 973 blocks
==27122==   total heap usage: 3,041 allocs, 2,068 frees, 157,981 bytes allocated
==27122== 
==27122== LEAK SUMMARY:
==27122==    definitely lost: 0 bytes in 0 blocks
==27122==    indirectly lost: 0 bytes in 0 blocks
==27122==      possibly lost: 0 bytes in 0 blocks
==27122==    still reachable: 77,033 bytes in 973 blocks
==27122==         suppressed: 0 bytes in 0 blocks
==27122== Rerun with --leak-check=full to see details of leaked memory
==27122== 
==27122== For lists of detected and suppressed errors, rerun with: -s
==27122== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

Then applying this fix:

------------------------------- src/tokenizer.c -------------------------------
index 38a4e81..8eb69c3 100644
@@ -2678,7 +2678,7 @@ struct token_list replacement_list(struct preprocessor_ctx* ctx, struct macro* m
 struct token_list pp_tokens_opt(struct preprocessor_ctx* ctx, struct token_list* input_list, int level)
 {
    struct token_list r = { 0 };
-   while (input_list->head->type != TK_NEWLINE)
+   while (input_list->head && input_list->head->type != TK_NEWLINE)
    {
        prematch_level(&r, input_list, level);
    }

Then:

valgrind ./cake -Iinclude sqlite3-orig/sqlite3.c
==27365== Memcheck, a memory error detector
==27365== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==27365== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==27365== Command: ./cake -Iinclude sqlite3-orig/sqlite3.c
==27365== 
Cake 0.7.4

sqlite3-orig/sqlite3.c
sqlite3-orig/sqlite3.c:36582:29: note: you can use "adjacent" "strings"
 36582 |      __asm__ __volatile__ (
       |                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:46561:58: warning: unnecessary line-slicing
 46561 |WINBASEAPI HANDLE WINAPI CreateFileMappingFromApp(HANDLE,
       |                                                         ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:46567:75: warning: unnecessary line-slicing
 46567 |WINBASEAPI HANDLE WINAPI CreateFileMappingA(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:46572:75: warning: unnecessary line-slicing
 46572 |WINBASEAPI HANDLE WINAPI CreateFileMappingW(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:129301:6: warning: unnecessary line-slicing
 129301 |    }
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:24249:22: error: unexpected
 24249 |    *va_arg(ap,int*) = val;
       |                     ^
sqlite3-orig/sqlite3.c:24249:22: error: expected TK_SEMICOLON
 24249 |    *va_arg(ap,int*) = val;
       |                     ^
sqlite3-orig/sqlite3.c:28818:3: error: not found 'mutexIsInit'
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:28818:3: error: expected TK_RIGHT_PARENTHESIS
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |          ^~~~~~~~~~~
sqlite3-orig/sqlite3.c:28818:3: error: expected TK_SEMICOLON
 28818 |  assert( GLOBAL(int, mutexIsInit) ) assert(mutexIsInit);
       |                     ^
sqlite3-orig/sqlite3.c:29331:3: error: member 'nRef' not found in struct 'sqlite3_mutex'
 29331 |  assert( p->nRef==0 ) assert(p->nRef==0);
       |             ^~~~
sqlite3-orig/sqlite3.c:29358:3: error: member 'id' not found in struct 'sqlite3_mutex'
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |             ^~
sqlite3-orig/sqlite3.c:29358:3: error: not found 'pthreadMutexNotheld'
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: right type is not scalar for or expression
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_RIGHT_PARENTHESIS
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                         ^
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                           ^
sqlite3-orig/sqlite3.c:29358:3: error: unexpected
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29358:3: error: expected TK_SEMICOLON
 29358 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29401:3: error: member 'id' not found in struct 'sqlite3_mutex'
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |             ^~
sqlite3-orig/sqlite3.c:29401:3: error: not found 'pthreadMutexNotheld'
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: right type is not scalar for or expression
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_RIGHT_PARENTHESIS
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                     ^~~~~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                         ^
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                           ^
sqlite3-orig/sqlite3.c:29401:3: error: unexpected
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29401:3: error: expected TK_SEMICOLON
 29401 |  assert( p->id==SQLITE_MUTEX_RECURSIVE || pthreadMutexNotheld(p) ) assert(p->id==1 || pthreadMutexNotheld(p));
       |                                            ^
sqlite3-orig/sqlite3.c:29457:3: error: not found 'pthreadMutexHeld'
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |          ^~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_RIGHT_PARENTHESIS
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |          ^~~~~~~~~~~~~~~~
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                          ^
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                            ^
sqlite3-orig/sqlite3.c:29457:3: error: unexpected
 29457 |  assert( pthreadMutexHeld(p) ) assert(pthreadMutexHeld(p));
       |                             ^
sqlite3-orig/sqlite3.c:29457:3: error: expected TK_SEMICOLON
 29457 |  assert( pthreadMutexHeld(p==27365== Invalid read of size 4
==27365==    at 0x126259: type_is_void (type.c:611)
==27365==    by 0x13C1E3: unlabeled_statement (parser.c:5558)
==27365==    by 0x13CC6C: block_item (parser.c:5894)
==27365==    by 0x13C906: block_item_list (parser.c:5788)
==27365==    by 0x13C6C7: compound_statement (parser.c:5702)
==27365==    by 0x13BE52: primary_block (parser.c:5446)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==    by 0x13BDDA: statement (parser.c:5434)
==27365==    by 0x13BF86: secondary_block (parser.c:5480)
==27365==    by 0x13D096: selection_statement (parser.c:6031)
==27365==    by 0x13BE7A: primary_block (parser.c:5450)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==  Address 0x23e28578 is 8 bytes inside a block of size 232 free'd
==27365==    at 0x4C35A6F: free (vg_replace_malloc.c:974)
==27365==    by 0x12295A: expression_delete (expressions.c:3565)
==27365==    by 0x122906: expression_delete (expressions.c:3560)
==27365==    by 0x1225B3: assignment_expression (expressions.c:3508)
==27365==    by 0x122990: expression (expressions.c:3577)
==27365==    by 0x13D9FE: expression_statement (parser.c:6314)
==27365==    by 0x13C1A1: unlabeled_statement (parser.c:5553)
==27365==    by 0x13CC6C: block_item (parser.c:5894)
==27365==    by 0x13C906: block_item_list (parser.c:5788)
==27365==    by 0x13C6C7: compound_statement (parser.c:5702)
==27365==    by 0x13BE52: primary_block (parser.c:5446)
==27365==    by 0x13C15F: unlabeled_statement (parser.c:5545)
==27365==  Block was alloc'd at
==27365==    at 0x4C37F23: calloc (vg_replace_malloc.c:1554)
==27365==    by 0x11C0B4: primary_expression (expressions.c:1165)
==27365==    by 0x11E2F4: postfix_expression (expressions.c:1943)
==27365==    by 0x11F906: unary_expression (expressions.c:2374)
==27365==    by 0x11FE3A: cast_expression (expressions.c:2498)
==27365==    by 0x11FEE0: multiplicative_expression (expressions.c:2533)
==27365==    by 0x120418: additive_expression (expressions.c:2664)
==27365==    by 0x120F4B: shift_expression (expressions.c:2875)
==27365==    by 0x12117F: relational_expression (expressions.c:2946)
==27365==    by 0x121685: equality_expression (expressions.c:3091)
==27365==    by 0x12198E: and_expression (expressions.c:3164)
==27365==    by 0x121B81: exclusive_or_expression (expressions.c:3218)
==27365== 
...
==27365== 
==27365== HEAP SUMMARY:
==27365==     in use at exit: 167,251,640 bytes in 4,385,178 blocks
==27365==   total heap usage: 4,755,509 allocs, 370,331 frees, 183,171,918 bytes allocated
==27365== 
==27365== LEAK SUMMARY:
==27365==    definitely lost: 0 bytes in 0 blocks
==27365==    indirectly lost: 0 bytes in 0 blocks
==27365==      possibly lost: 0 bytes in 0 blocks
==27365==    still reachable: 167,251,640 bytes in 4,385,178 blocks
==27365==         suppressed: 0 bytes in 0 blocks
==27365== Rerun with --leak-check=full to see details of leaked memory
==27365== 
==27365== For lists of detected and suppressed errors, rerun with: -s
==27365== ERROR SUMMARY: 179 errors from 19 contexts (suppressed: 0 from 0)
Segmentation fault (core dumped)

[thradams]

( Note: I never compiled sqlite on linux.. just on windows. So it is a good opportunity to fix some bugs. )

[mingodad]

Looking through the usage (see bellow) of struct expression::type it seems that it should be struct type owner * type; instead of struct type type; :

• https://github.com/thradams/cake/blob/d503e535afa6e9fd09f4cebc8487ee55c02d8511/src/expressions.c#L1354

Because here https://github.com/thradams/cake/blob/d503e535afa6e9fd09f4cebc8487ee55c02d8511/src/type.c#L1794 you are making a raw copy (memcpy) .

[mingodad]

Attached is the output of running test-c-reference.lua , it includes the extracted code for the ones that fail.

Total files scanned     :   611
Total files with code   :   432
Total files failed code :   129

test-c-reference.lua.log.zip

[thradams]

a zip with source code only (.c) would help. I don't have lua installed or lua experience. Initially I was using unittest with files, but then I changed to use "embedded" source code, because it was faster to check. But we can have both approaches like having a folder 100% with code that should compile ok.

On Fri, 23 Feb 2024 at 07:31, Domingo Alvarez Duarte < @.***> wrote:

Attached is the output of running test-c-reference.lua , it includes the extracted code for the ones that fail.

Total files scanned : 611 Total files with code : 432 Total files failed code : 129

test-c-reference.lua.log.zip https://github.com/thradams/cake/files/14384173/test-c-reference.lua.log.zip

— Reply to this email directly, view it on GitHub https://github.com/thradams/cake/issues/106#issuecomment-1961082212, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSMZLTVTAH4LONCA26PNRTYVBVXJAVCNFSM6AAAAABDRFHAL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRRGA4DEMRRGI . You are receiving this because you commented.Message ID: @.***>

-- www.thradams.com

[mingodad]

The zip file is the output of running the Lua script, you can see the errors and the source code for the failures direct there (no need to to run the script yourself).

[mingodad]

Here is a simplified code that cake can't handle but gcc can:

#include <stdio.h>
int main(void)
{
    printf("isgreater(2.0,1.0)      = %d\n",  ( ({int x=2; int y=3; x > y; })));

    return 0;
}

[mingodad]

In the zip file search for ==rc:FAILED== to go straight to a failure.

[mingodad]

This https://github.com/mingodad/cake/commit/9294ae781fe2b7c6b9269a9ce487c8857f6917a8 temporary fix allow more files to be parsed.

[mingodad]

I found this two minimal examples that show a bug in cake when handling wchar_t: This one parses fine:

#include <stdio.h>
#include <wchar.h>

int main(void)
{
    wchar_t *wstr =L"御休みなさい";
    int rc = fputws(wstr, stdout);
    return rc;
}

This one that do not use an intermediate variable fail:

#include <stdio.h>
#include <wchar.h>

int main(void)
{
    int rc = fputws(L"御休みなさい", stdout);
    return rc;
}

Output:

src/cake dad/test-wchar1.c
Cake 0.7.4

dad/test-wchar1.c

 1 files in 0.02 seconds
 0 errors 0 warnings 0 notes

src/cake dad/test-wchar2.c
Cake 0.7.4

dad/test-wchar2.c
type mismatch:
unsigned *
const int * __ws => wchar_t
dad/test-wchar2.c:6:21: error:  incompatible types at argument 1 [-W]
 6 |    int rc = fputws(L"御休みなさい", stdout stdout);
   |                    ^~~~~~~~~~~~~~~~~~~~~

 1 files in 0.03 seconds
 1 errors 0 warnings 0 notes 

[mingodad]

With this fix https://github.com/mingodad/cake/commit/603abc438ca210b0701955482724767571a5aa65 I could get cake to parse wchar_t literals type and parse several files: Before (lua "test-c-reference.lua"):

Total files scanned     :   611
Total files with code   :   432
Total files failed code :   129

After (lua "test-c-reference.lua"):

Total files scanned     :   611
Total files with code   :   432
Total files failed code :   107

[mingodad]

With this sample:

enum { SIZE = 5 };

int main(void)
{
    const double a[SIZE] = {1.0, 2.0, 3.0, 4.0, 5.0};
    unsigned int sz1 = sizeof a;
    unsigned int sz2 =  sizeof *a;
}

And with this fix https://github.com/mingodad/cake/commit/a4bcf34072a8e92f93871bf7323a09e50a94bd21 we get:

src/cake  dad/test-type-pointer.c
Cake 0.7.4

/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-type-pointer.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-type-pointer.c:7:32: error: indirection requires pointer operand [-W]
 7 |    unsigned int sz2 =  sizeof *a;
   |                               ^

 1 files in 0.00 seconds
 1 errors 0 warnings 0 notes 

Before it was:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
cake: type.c:1573: type_remove_pointer: Assertion `type_is_pointer(p_type)' failed.
timeout: the monitored command dumped core
Command terminated by signal 6

[mingodad]

With this fix https://github.com/mingodad/cake/commit/5e61573d2175d7ecee68311c822c2de81c47e4c9 : Sample:

#include <complex.h>
#include <stdio.h>

int main(void)
{
    double imaginary z = 3*I;
    z = 1 / z;
    printf("1/(3.0i) = %+.1fi\n", cimag(z));
}

Output:

src/cake  dad/test-double-free.c
>src/cake  dad/test-double-free.c
Cake 0.7.4

/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-double-free.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-double-free.c:6:22: error: expected TK_SEMICOLON [-W]
 6 |    double imaginary z = 3*I( 1.0iF);
   |                     ^
...
 1 files in 0.02 seconds
 14 errors 0 warnings 0 notes 

Before:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
free(): double free detected in tcache 2
timeout: the monitored command dumped core
Command terminated by signal 6

[mingodad]

With this temporary fix https://github.com/mingodad/cake/commit/0a59d95eba52c12f9ac9e3bc9644fac03aaadd97 : Sample:

#include <math.h>
#include <stdio.h>

// Possible implementation of the tgmath.h macro cbrt
#define cbrt(X) _Generic((X), \
              long double: cbrtl, \
                  xdefault: cbrt,  \
                    float: cbrtf  \
              )(X)

int main(void)
{
    double x = 8.0;
    const float y = 3.375;
    printf("cbrt(8.0) = %f\n", cbrt(x)); // selects the default cbrt
    printf("cbrtf(3.375) = %f\n", cbrt(y)); // converts const float to float,
                                            // then selects cbrtf
}

Output:

src/cake  dad/test-generic.c
>src/cake  dad/test-generic.c
Cake 0.7.4

/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c:5:9: note: use SCREAMING_CASE for macros
 5 |#define cbrt(X) _Generic((X), \
              long double: cbrtl, \
                  xdefault: cbrt,  \
                    float: cbrtf  \
              )(X)
   |        ^~~~
/home/mingo/dev/c/A_programming-languages/cake-dad2/dad/test-generic.c:15:32: error: unexpected [-W]
 15 |    printf("cbrt(8.0) = %f\n", cbrt(x) _Generic((x), long double: cbrtl, xdefault: cbrt, float: cbrtf )(x)); // selects the default cbrt
    |                                                                 ^~~~~~~~
...
 1 files in 0.03 seconds
 24 errors 0 warnings 0 notes 

Before:

/usr/bin/time timeout 3s src/cake /tmp/cCode.c
timeout: the monitored command dumped core
Command terminated by signal 11

This pattern in expressions.c need to be revised because several fixes seem to be by returning dangle pointers:

            {
                expression_delete(new_expression);
                throw;
            }

[mingodad]

This https://github.com/mingodad/cake/commit/aa07dbd3d04420de170b9dda9804140226157fa4 fix the UTF8 range assert that was failing with the sample bellow:

#include <errno.h>
#include <locale.h>
#include <stdio.h>
#include <stdlib.h>
#include <wchar.h>

int main(void)
{
    setlocale(LC_ALL, "en_US.utf8");

    errno = 0;
    if (fputwc(L'🍌', stdout) == WEOF)
    {
        if (errno == EILSEQ)
            puts("Encoding error in fputwc.");
        else
            puts("I/O error in fputwc.");
        return EXIT_FAILURE;
    }
}

[thradams]

hi I believe print_type_specifier_flags from you branch is wrong because we can have like unsigned int two specifiers. so if else is wrong.

On Sat, 24 Feb 2024 at 14:20, Domingo Alvarez Duarte < @.***> wrote:

This @.*** https://github.com/mingodad/cake/commit/aa07dbd3d04420de170b9dda9804140226157fa4 fix the UTF8 range assert that was failing with the sample bellow:

include

include

include

include

include

int main(void) { setlocale(LC_ALL, "en_US.utf8");

errno = 0;
if (fputwc(L'🍌', stdout) == WEOF)
{
    if (errno == EILSEQ)
        puts("Encoding error in fputwc.");
    else
        puts("I/O error in fputwc.");
    return EXIT_FAILURE;
}

}

— Reply to this email directly, view it on GitHub https://github.com/thradams/cake/issues/106#issuecomment-1962427681, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSMZLU2BTA2KM3DMTECRB3YVIOPDAVCNFSM6AAAAABDRFHAL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSGQZDONRYGE . You are receiving this because you commented.Message ID: @.***>

-- www.thradams.com

[mingodad]

Thank you for reply ! Yes you're right ! I've updated it now.

[mingodad]

This fix https://github.com/mingodad/cake/commit/b3c7c11c488ce331870e1b9f3c37feb4d2b4aefc memory use after free detected by valgrind when parsing this sample:

#include <stdio.h>
#include <complex.h>
#include <math.h>

int main(void)
{
    double complex z1 = 0.0 + INFINITY * _Imaginary_I;
    printf("z1 = %.1f%+.1fi\n", creal(z1), cimag(z1));

    double complex z2 = 0.0 + INFINITY * _Complex_I;
    printf("z2 = %.1f%+.1fi\n", creal(z2), cimag(z2));
}

Before the fix:

src/cake /tmp/cCode.c
free(): double free detected in tcache 2
timeout: the monitored command dumped core
Command terminated by signal 6

After the fix:

src/cake /tmp/cCode.c
Cake 0.7.4

/tmp/cCode.c
/tmp/cCode.c:7:42: error: not found '_Imaginary_I'
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: right * is not arithmetic
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: invalid type multiplicative expression
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~
/tmp/cCode.c:7:42: error: expected TK_SEMICOLON
 7 |    double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I;
   |                                                           ^~~~~~~~~~~~

[thradams]

I did a checkin with a pattern that I think it is better

with fails like malloc inside try blocks I want to return null to avoid ignoring half AST not begin reported. (unless we add a report error for out-of-memory.) then the AST could be half correct, because it reports an error.

    catch
    {
        if (p_expression_node)
        {
            expression_node_delete(p_expression_node);
            p_expression_node = NULL;
        }
    }

double free is fixed just puting

     new_expression->left = p_expression_node;
     p_expression_node = NULL; /*MOVED*/

after moving it

[thradams]

did you see the option -DTEST on build? When we fix some problem is good to add a test case in tests.c that covers the problems. then we avoid regressions.

On Sun, 25 Feb 2024 at 06:29, Domingo Alvarez Duarte < @.***> wrote:

This fix @.*** https://github.com/mingodad/cake/commit/b3c7c11c488ce331870e1b9f3c37feb4d2b4aefc memory use after free detected by valgrind when parsing this sample:

include

include

include

int main(void) { double complex z1 = 0.0 + INFINITY * _Imaginary_I; printf("z1 = %.1f%+.1fi\n", creal(z1), cimag(z1));

double complex z2 = 0.0 + INFINITY * _Complex_I;
printf("z2 = %.1f%+.1fi\n", creal(z2), cimag(z2));

}

Before the fix:

src/cake /tmp/cCode.c free(): double free detected in tcache 2 timeout: the monitored command dumped core Command terminated by signal 6

After the fix:

src/cake /tmp/cCode.c Cake 0.7.4

/tmp/cCode.c /tmp/cCode.c:7:42: error: not found '_Imaginary_I' 7 | double complex _Complex z1 = 0.0 + INFINITY 1e10000f _Imaginary_I; | ^~~~ /tmp/cCode.c:7:42: error: right is not arithmetic 7 | double complex _Complex z1 = 0.0 + INFINITY 1e10000f _Imaginary_I; | ^~~~ /tmp/cCode.c:7:42: error: invalid type multiplicative expression 7 | double complex _Complex z1 = 0.0 + INFINITY 1e10000f _Imaginary_I; | ^~~~ /tmp/cCode.c:7:42: error: expected TK_SEMICOLON 7 | double complex _Complex z1 = 0.0 + INFINITY 1e10000f * _Imaginary_I; | ^~~~

— Reply to this email directly, view it on GitHub https://github.com/thradams/cake/issues/106#issuecomment-1962870626, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSMZLRWX4Z2BSMYG5IB2ETYVMABFAVCNFSM6AAAAABDRFHAL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSHA3TANRSGY . You are receiving this because you commented.Message ID: @.***>

-- www.thradams.com

[thradams]

a zip with all .c would be useful. the same code can compile on linux and not compile on windows for instance. the includes and the behavior is different it also can work as regression test.

On Fri, 23 Feb 2024 at 09:35, Domingo Alvarez Duarte < @.***> wrote:

The zip file is the output of running the Lua script, you can see the errors and the source code for the failures direct there (no need to to run the script yourself).

— Reply to this email directly, view it on GitHub https://github.com/thradams/cake/issues/106#issuecomment-1961250233, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSMZLTRIIJKSZS2S75LJGTYVCEHTAVCNFSM6AAAAABDRFHAL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRRGI2TAMRTGM . You are receiving this because you commented.Message ID: @.***>

-- www.thradams.com

[mingodad]

Here it's a zip with all examples extracted. en-cpp-reference-c.zip

[thradams]

many thanks! I will try to use it as unit tests.

On Sun, 25 Feb 2024 at 09:14, Domingo Alvarez Duarte < @.***> wrote:

Here it's a zip with all examples extracted. en-cpp-reference-c.zip https://github.com/thradams/cake/files/14396325/en-cpp-reference-c.zip

— Reply to this email directly, view it on GitHub https://github.com/thradams/cake/issues/106#issuecomment-1962916587, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABSMZLV6YS3C7MGU6CD5OFDYVMTJ5AVCNFSM6AAAAABDRFHAL6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRSHEYTMNJYG4 . You are receiving this because you commented.Message ID: @.***>

-- www.thradams.com

[thradams]

ccode424.c is failing..but nullptr_t is a typedef.

void puts(const char*);

#define DETECT_NULL_POINTER_CONSTANT(e) \
    _Generic(e,                         \
        void* : puts("void*"),          \
        nullptr_t : puts("nullptr_t"),  \
        default : puts("other")       \
    )

int main()
{
    DETECT_NULL_POINTER_CONSTANT(((void*)0));
    DETECT_NULL_POINTER_CONSTANT(0);
    DETECT_NULL_POINTER_CONSTANT(nullptr);
}

I think cake is correct.

[mingodad]

Why do you have src/file.c and tests/en-cpp-reference-c/out checked in the repository ? Aren't those files temporary or generated files ?

[thradams]

The reason I kept tests/en-cpp-reference-c/out is to compare the code generation ouput. Lets say regression in code generation. file.c is not generated but it is the file I use as "temp" and generally it has the last sample I worked.

[thradams]

Most of errors now are from imaginary samples that where not implemented.

[thradams]

sqlite is compiling. one problem was assert as keyword. the other was __builtinoffset

assert assert can be fixed. it was used for instance

void flow_visit_ctx_destroy(struct flow_visit_ctx* obj_owner p)
{
    assert(p->tail_block == NULL);
}

it works at same time as runtime and static information. the problem is that when debug is not defined we cannot parse the innert part we need to use it as empty macro. the detection of GCC assert ((void)0) is missing;;

[thradams]

ouput .\cake sqlite.c on linux

I added cakeconfig.h at src

/home/tra/cake/src/sqlite3.c
/home/tra/cake/src/sqlite3.c:46447:58: warning: unnecessary line-slicing
 46447 |WINBASEAPI HANDLE WINAPI CreateFileMappingFromApp(HANDLE,
       |                                                         ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:46453:75: warning: unnecessary line-slicing
 46453 |WINBASEAPI HANDLE WINAPI CreateFileMappingA(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:46458:75: warning: unnecessary line-slicing
 46458 |WINBASEAPI HANDLE WINAPI CreateFileMappingW(HANDLE, LPSECURITY_ATTRIBUTES,
       |                                                                          ^~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:129074:6: warning: unnecessary line-slicing
 129074 |    }
        |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/tra/cake/src/sqlite3.c:40309:23: warning:  passing null as array [-Wnonnull]
 40309 |    utimes(zLockFile, NULL ((void *)0));
       |                      ^
/home/tra/cake/src/sqlite3.c:107205:55: warning: index 1 is past the end of the array [-Wout-of-bounds]
 107205 |            pExpr->iTable = exprProbability(pList->a[1].pExpr);
        |                                                      ^
/home/tra/cake/src/sqlite3.c:107475:61: warning: index 1 is past the end of the array [-Wout-of-bounds]
 107475 |          nRight = sqlite3ExprVectorSize(pExpr->x.pList->a[1].pExpr);
        |                                                            ^
/home/tra/cake/src/sqlite3.c:112595:60: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112595 |         sqlite3ExprCompare(0,pFarg->a[0].pExpr, pFarg->a[1].pExpr,-1),
        |                                                           ^
/home/tra/cake/src/sqlite3.c:112604:69: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112604 |         sqlite3ExprImpliesExpr(pParse,pFarg->a[0].pExpr, pFarg->a[1].pExpr,-1),
        |                                                                    ^
/home/tra/cake/src/sqlite3.c:112613:23: warning: index 1 is past the end of the array [-Wout-of-bounds]
 112613 |      pA1 = pFarg->a[1].pExpr;
        |                      ^
/home/tra/cake/src/sqlite3.c:113225:71: warning: index 1 is past the end of the array [-Wout-of-bounds]
 113225 |        pDef = sqlite3VtabOverloadFunction(db, pDef, nFarg, pFarg->a[1].pExpr);
        |                                                                      ^
/home/tra/cake/src/sqlite3.c:113831:43: warning: index 1 is past the end of the array [-Wout-of-bounds]
 113831 |    compRight.pRight = pExpr->x.pList->a[1].pExpr;
        |                                          ^
/home/tra/cake/src/sqlite3.c:114428:48: warning: index 1 is past the end of the array [-Wout-of-bounds]
 114428 |       || exprImpliesNotNull(pParse, pList->a[1].pExpr, pNN, iTab, 1)
        |                                               ^
/home/tra/cake/src/sqlite3.c:114618:55: warning: index 1 is past the end of the array [-Wout-of-bounds]
 114618 |                                   pExpr->x.pList->a[1].pExpr);
        |                                                      ^
/home/tra/cake/src/sqlite3.c:125624:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 125624 |      memcpy(&p1->a[1], p2->a, p2->nSrc*sizeof(SrcItem));
        |                     ^
/home/tra/cake/src/sqlite3.c:125626:50: warning: index 1 is past the end of the array [-Wout-of-bounds]
 125626 |      p1->a[0].fg.jointype |= (JT_LTORJ0x40 & p1->a[1].fg.jointype);
        |                                                     ^
/home/tra/cake/src/sqlite3.c:130152:40: warning: index 2 is past the end of the array [-Wout-of-bounds]
 130152 |    Expr *pEscape = pExpr->x.pList->a[2].pExpr;
        |                                       ^
/home/tra/cake/src/sqlite3.c:148965:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 148965 |    if( pTabList->a[1].fg.jointype & JT_CROSS 0x02 ) return 1;         /* (1b) */
        |                     ^
/home/tra/cake/src/sqlite3.c:149271:27: warning: index 1 is past the end of the array [-Wout-of-bounds]
 149271 |         || (pTabList->a[1].fg.jointype&(JT_OUTER0x20|JT_CROSS0x02))!=0)
        |                          ^
/home/tra/cake/src/sqlite3.c:159094:21: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159094 |  pLeft = pList->a[1].pExpr;
        |                    ^
/home/tra/cake/src/sqlite3.c:159281:22: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159281 |    pCol = pList->a[1].pExpr;
        |                     ^
/home/tra/cake/src/sqlite3.c:159322:32: warning: index 1 is past the end of the array [-Wout-of-bounds]
 159322 |          *ppRight = pList->a[1].pExpr;
        |                               ^
/home/tra/cake/src/sqlite3.c:160233:32: warning: index 1 is past the end of the array [-Wout-of-bounds]
 160233 |    pLeft = pExpr->x.pList->a[1].pExpr;
        |                               ^

[mingodad]

Great work ! About memory usage cake is using around 5x more than gcc-9.4 and 4x more than clang-17:

/usr/bin/time myvalgrind src/cake  sqlite3-orig/sqlite3.c
...
1 files in 0.94 seconds
 2415 errors 2 warnings 3 notes 
Command exited with non-zero status 1
0.77user 0.18system 0:00.96elapsed 99%CPU (0avgtext+0avgdata 382088maxresident)k
0inputs+0outputs (0major+95108minor)pagefaults 0swaps

/usr/bin/time gcc -fsyntax-only  sqlite3-orig/sqlite3.c
0.38user 0.05system 0:00.44elapsed 99%CPU (0avgtext+0avgdata 79596maxresident)k
0inputs+0outputs (0major+19793minor)pagefaults 0swaps

/usr/bin/time clang-17-env clang -fsyntax-only  sqlite3-orig/sqlite3.c
0.73user 0.08system 0:01.05elapsed 76%CPU (0avgtext+0avgdata 100064maxresident)k
151992inputs+0outputs (545major+9762minor)pagefaults 0swaps

[thradams]

Not sure, but probably it is because comments, spaces, inactive preprocessor blocks are preserved to be able to generate the "c backend" . But I think if cake is used as static analyzer only it could discard a lot of tokens. or when used with the option -no-ouput. It also preserve tokens inside includes because initially I was sure if I could use comments This also needs to be reviewed and discard tokens from included files.also with a option.

[mingodad]

Also for comparison https://github.com/rui314/chibicc uses more memory than cake when compiling sqlite3.c:

chibicc -c sqlite3.c 
2.59user 0.37system 0:02.97elapsed 99%CPU (0avgtext+0avgdata 525964maxresident)k
0inputs+6336outputs (0major+201446minor)pagefaults 0swaps

And tinycc uses almost nothing compared to the others:

/usr/bin/time tcc -c sqlite3.c  
0.09user 0.01system 0:00.11elapsed 93%CPU (0avgtext+0avgdata 16748maxresident)k
840inputs+3696outputs (2major+3726minor)pagefaults 0swaps

[thradams]

TCC  is one pass I think Cake has the ast used in second passa.But the main problem must be the tokens that are not discarded Em 25 de fev. de 2024, à(s) 16:24, Domingo Alvarez Duarte @.***> escreveu: Also for comparison https://github.com/rui314/chibicc uses more memory than cake when compiling sqlite3.c: chibicc -c sqlite3.c 2.59user 0.37system 0:02.97elapsed 99%CPU (0avgtext+0avgdata 525964maxresident)k 0inputs+6336outputs (0major+201446minor)pagefaults 0swaps

And tinycc uses almost nothing compared to the others: /usr/bin/time tcc -c sqlite3.c
0.09user 0.01system 0:00.11elapsed 93%CPU (0avgtext+0avgdata 16748maxresident)k 840inputs+3696outputs (2major+3726minor)pagefaults 0swaps

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[thradams]

A lot of samples using imaginary. Some errors like missing strdup is caused because cake uses windows headers.

[thradams]

nullptr_t is suppose to be a typedef and it is used as builtin type on samples

[thradams]

alignas error missing GCC asm. it is not the same of standar asm ccode358 char is typedef strdup not present on windows headers

[thradams]

individual errors will be created. asm, align errors already exist.