The C syntax is different from the C++ syntax

[jeff-hykin]

Right now the C++ and C syntax are being generated completely separately. I am familiar with C++ and most of my fixes have been exclusively for C++ even if they should be applicable to C.

There have already been issues posted about this difference.

• 46

• 47

This issue is a log and general discussion about how to keep the C syntax in sync with the C++ Syntax without breaking compatibility on either side.

update:

I've decided to fix all of C++ first and then fix C. If there's a major issue with the C syntax, then it will get fixed separately.

I'm going to use this post as a log to keep track of all the non-major issues with the C syntax.

1. Percent Escapes in C sscanf(user_input, "%c %[^\n]", &arg0, arg_str); // reads char into arg0, the remainder until \n to arg_str atom/language-c#222 atom/language-c#70
2. struct f foo is missing highlighting

   46

   
   /* Forward Declaration */
   struct A b;
   enum B c;
   union C d;

/ Function Declaration / void a(struct A b); void b(enum B c); void c(union C d); void d(A b);

/ Definition / struct A { enum B c; union C d; }; enum B { struct A d; union C d; }; union C { struct A b; enum B c; };

3. Trigraph support
#32 
`const char * a = "foo ??/" ??/??/";`

[jeff-hykin]

What is your opinion on temporarily using the C++ syntax as the C syntax since C++ is almost a perfect superset of C @matter123 ? We might want to consider this since changes to the theme (for C++) might break the colors in C since their tags have not been updated. (ex: keyword.control for memory)

Or maybe having it be a superset with the exception of replacing cpp_tokens with c_tokens

There is also the option of creating a shared C and C++ file that is imported into both, but that will take time to setup properly.

[matter123]

I have a very experimental way to "diff" grammars, let me compare the C and C++ on the linux kernel. Give me about 10 minutes to set that up. Sharing common patterns is probably the best though.

Edit: There are far more files than I thought. The relatively small file size is promising, however.

[matter123]

linux.log

Only a small fraction of the files as the diff tool crashed. However, most changes from the C++ grammar look harmless. Have a better source of C files?

Gettext: gettext.log

• C++ drops built-in from storage.type.built-in.primitive
• C++ adds -c to punctuation.section.parens-c
• C++ removes punctuation.section.parens.block and instead has meta.block.parens
• on line 71717 the scope changes from variable.object to variable.other.object
• The C++ grammar correctly tags the attribute on line 71785
• C++ drops punctuation.section.block.begin.bracket.curly from union YYSTYPE {
• C++ misses that on line 73016 there is a preprocessor directive
• whatever is happening here: 0222: token ' s' has differing scopes scopes for c++ source meta.block meta.block.parens + scopes for c source meta.block punctuation.section.parens.block - 0222: token 'izeof (buf)' has differing scopes scopes for c++ source meta.block meta.block.parens + meta.function-call + scopes for c source meta.block punctuation.section.parens.block - keyword.operator.sizeof -
• incorrect identification as operator? 0042: token 'HASHWORDBITS ' has differing scopes scopes for c++ source meta.block meta.block.parens + meta.block.parens + scopes for c source meta.block punctuation.section.parens.block - punctuation.section.parens.block - keyword.operator -

While its not impossiable to do there are several issues.

[jeff-hykin]

Sadly I don't but I might know some people that do. I'll check in with them. This is a good idea for testing.

[matter123]

Yeah, it can catch some slightly hard to find bugs like anonymous struct https://github.com/jeff-hykin/cpp-textmate-grammar/pull/57#issuecomment-481502001, but it is currently pretty rough.

[jeff-hykin]

So long as themers don't have an issue, it might be best to fully clean up the C++ code first and then once it is, we work on back-porting everything to C.

If themers do have an issue, then I think the best solution will be to use the C++ highlighting for both until it is cleaned up. I looked up the differences in the C syntax that were not in C++ and there are only a handful of cases.

[matter123]

C supports some pretty fancy designated initalizers int a[10] = { [5] =5,0} and {.a.v[0].c = {.a ={[1]='q'}} are both valid.

[jeff-hykin]

I guess we will have to see how the C++ syntax handles that. So long as it doesn't break the syntax it will probably still be an improvement over what it currently is highlighted with.

That is interesting though, I didn't see anything about that so I probably didn't dig deep enough.

[matter123]

C	C++

So currently C and C++ offer similar coloring of the provided designated initializer example. C++ does forget about the character literal, however.

[tristan957]

I have a C repo located at https://git.sr.ht/~tristan957/tllt-cp that might be able to serve as a good test for diffing the C/C++ grammars. About 3000 lines of C code that uses many parts of the language.

[matter123]

tllt-cp.log

So the first item I noticed if we are to share generators, the existing pattern tags that manually add, .cpp have to go.

• C++ drops meta.function, but being a meta scope is probably not all that important.
• in err->message message is tagged as variable.other.member in C but variable.other.property in C++
• C++ drops punctuation.section.parameters.end.bracket.round from []
• again the scope changes from variable.object to variable.other.object
• scope changes from punctuation.section.parens.block to meta.block.parens

Thanks @tristan957

[tristan957]

Cool glad I could be of assistance

[jeff-hykin]

I just realized #107 blocks copying the C++ syntax to C. It would completely wreck any language that is inheriting from the C language.

[matter123]

I am going to use this comment to list patterns that can be shared between C and C++

• [ ] identifiers
  • both C and C++ agree on the character set that can be in an identifier including universal character names
• [x] numbers
  • C has no support for user-defined literals or numeric separators but this should not be an issue as quoting from the C11 standard 6.4/5 (non normative)

    EXAMPLE 1 The program fragment 1Ex is parsed as a preprocessing number token (one that is not a valid floating or integer constant token), even though a parse as the pair of preprocessing tokens 1 and Ex might produce a valid expression (for example, if Ex were a macro defined as +1). Similarly, the program fragment 1E1 is parsed as a preprocessing number (one that is a valid floating constant token), whether or not E is a macro name.

tokenization treats any sequence of characters starting with a number as a number

• [ ] tri and digraphs
• [ ] string literals
  • should be compatible
  • C does not support Raw string literals, but they are different enough that they should be a different pattern
• [ ] the preprocessor should be identical
• [ ] comments (part of @std_space)
• [ ] C support patterns should be compatibe (std::)?malloc shoulden't over match in the C grammar, but maybe with Obj-C
• [ ] some keywords are shared but both languages have their own set of unique keywords.
• [ ] wordlike keywords and, or, etc are only defined in C upon inclusion of iso646.h
  • same goes for bool and stdbool.h
  • noreturn and stdnoreturn.h
  • static_assert and assert.h
  • these may need to not be marked specially or marked as support
• [ ] the lack of templates makes https://github.com/jeff-hykin/cpp-textmate-grammar/issues/81#issuecomment-482398279 unambigous so consideration is needed when sharing expression patterns
• [ ] union should be sharable
  • c++ unions do not support inheritance
• [ ] the typedef pattern as mentioned in https://github.com/jeff-hykin/cpp-textmate-grammar/issues/81#issuecomment-485662961 (see #196)
  • it will require injectable includes to be used for both
• [ ] enums
  • the 3 options of enum key should not be an issue in C as enum struct foo has no valid meaning.
• TODO

[tristan957]

Are you guys noticing that true, false, and NULL in C++ are not highlighted the same as in C?

[matter123]

Yeah that's a regressions

[matter123]

@jeff-hykin Is there a preferred way for shared patterns to add to the current grammars repository?

[jeff-hykin]

Ah this is really old, I was thinking the file that imports the shared thing would add it to it's grammar.

[jeff-hykin]

In general though, I think most of the sharing is going to happen once C++ has it's type system figured out. Then the common stuff will be extracted and a good sharing system will be implemented.

[jeff-hykin]

C syntax has finally been separated/moved to https://github.com/jeff-hykin/better-c-syntax !