search

dvidelabs / flatcc

FlatBuffers Compiler and Library in C for C
Apache License 2.0
646 stars 182 forks source link

MinGW support #155

Closed vaind closed 4 years ago

vaind commented 4 years ago

Trying to compile flatcc with MinGW (TDM GCC 64) and I am having two, seemingly small, issues:

1. Compiling RT lib

I'm not entirely sure what's wrong but I was able to make it compile (i.e. the following error wouldn't show), if I flipped the #ifndef FLATCC_USE_GENERIC_ALIGNED_ALLOC to ifdef... Couldn't find where that flag is being set though :man_shrugging: 

cmake /z/objectbox-generator/./third_party/flatcc/src -DCMAKE_BUILD_TYPE=Release -G 'MinGW Makefiles'
-- The C compiler identification is GNU 9.2.0
-- The CXX compiler identification is GNU 9.2.0
-- Check for working C compiler: C:/TDM-GCC-64/bin/gcc.exe
-- Check for working C compiler: C:/TDM-GCC-64/bin/gcc.exe - works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/TDM-GCC-64/bin/g++.exe
-- Check for working CXX compiler: C:/TDM-GCC-64/bin/g++.exe - works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- dist install dir Z:/objectbox-generator/third_party/flatcc/src
-- lib install dir Z:/objectbox-generator/third_party/flatcc/src/lib
-- Setting GNU C compiler options with c11 and Posix
-- Disabling GNU C compiler warnings: -Wstringop-truncation -Wno-format-overflow
-- GCC_VERSION: 9.2.0

-- Configured C_FLAGS:  -DFLATCC_REFLECTION=1 -std=c11 -pedantic -Wall -Wextra -Wno-stringop-truncation -Wno-format-overflow -Werror -Wno-unused-function -Wsign-conversion
-- Configuring done
-- Generating done
-- Build files have been written to: Z:/objectbox-generator/third_party/flatcc/build
+ cd /z/objectbox-generator
+ cmake --build ./third_party/flatcc/build --config Release --target flatccrt
Scanning dependencies of target flatccrt
[ 14%] Building C object src/runtime/CMakeFiles/flatccrt.dir/builder.c.obj
In file included from Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_flatbuffers.h:26,
                 from Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:68,
                 from Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:19:
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c: In function 'flatcc_builder_finalize_aligned_buffer':
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_alloc.h:116:36: error: implicit declaration of function 'aligned_alloc' [-Werror=implicit-function-declaration]
  116 | #define FLATCC_ALIGNED_ALLOC(a, n) aligned_alloc(a, n)
      |                                    ^~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:203:44: note: in expansion of macro 'FLATCC_ALIGNED_ALLOC'
  203 | #define FLATCC_BUILDER_ALIGNED_ALLOC(a, n) FLATCC_ALIGNED_ALLOC(a, n)
      |                                            ^~~~~~~~~~~~~~~~~~~~
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:1995:14: note: in expansion of macro 'FLATCC_BUILDER_ALIGNED_ALLOC'
 1995 |     buffer = FLATCC_BUILDER_ALIGNED_ALLOC(align, size);
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_alloc.h:116:36: error: incompatible implicit declaration of built-in function 'aligned_alloc' [-Werror]
  116 | #define FLATCC_ALIGNED_ALLOC(a, n) aligned_alloc(a, n)
      |                                    ^~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:203:44: note: in expansion of macro 'FLATCC_ALIGNED_ALLOC'
  203 | #define FLATCC_BUILDER_ALIGNED_ALLOC(a, n) FLATCC_ALIGNED_ALLOC(a, n)
      |                                            ^~~~~~~~~~~~~~~~~~~~
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:1995:14: note: in expansion of macro 'FLATCC_BUILDER_ALIGNED_ALLOC'
 1995 |     buffer = FLATCC_BUILDER_ALIGNED_ALLOC(align, size);
      |              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:21:1: note: include '<stdlib.h>' or provide a declaration of 'aligned_alloc'
   20 | #include "flatcc/flatcc_emitter.h"
  +++ |+#include <stdlib.h>
   21 |
In file included from Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_flatbuffers.h:26,
                 from Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:68,
                 from Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:19:
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c: In function 'flatcc_builder_aligned_alloc':
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_alloc.h:116:36: error: incompatible implicit declaration of built-in function 'aligned_alloc' [-Werror]
  116 | #define FLATCC_ALIGNED_ALLOC(a, n) aligned_alloc(a, n)
      |                                    ^~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:203:44: note: in expansion of macro 'FLATCC_ALIGNED_ALLOC'
  203 | #define FLATCC_BUILDER_ALIGNED_ALLOC(a, n) FLATCC_ALIGNED_ALLOC(a, n)
      |                                            ^~~~~~~~~~~~~~~~~~~~
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:2014:12: note: in expansion of macro 'FLATCC_BUILDER_ALIGNED_ALLOC'
 2014 |     return FLATCC_BUILDER_ALIGNED_ALLOC(alignment, size);
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_alloc.h:116:36: note: include '<stdlib.h>' or provide a declaration of 'aligned_alloc'
  116 | #define FLATCC_ALIGNED_ALLOC(a, n) aligned_alloc(a, n)
      |                                    ^~~~~~~~~~~~~
Z:/objectbox-generator/third_party/flatcc/src/include/flatcc/flatcc_builder.h:203:44: note: in expansion of macro 'FLATCC_ALIGNED_ALLOC'
  203 | #define FLATCC_BUILDER_ALIGNED_ALLOC(a, n) FLATCC_ALIGNED_ALLOC(a, n)
      |                                            ^~~~~~~~~~~~~~~~~~~~
Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:2014:12: note: in expansion of macro 'FLATCC_BUILDER_ALIGNED_ALLOC'
 2014 |     return FLATCC_BUILDER_ALIGNED_ALLOC(alignment, size);
      |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1.exe: all warnings being treated as errors
src\runtime\CMakeFiles\flatccrt.dir\build.make:82: recipe for target 'src/runtime/CMakeFiles/flatccrt.dir/builder.c.obj' failed
mingw32-make.exe[3]: *** [src/runtime/CMakeFiles/flatccrt.dir/builder.c.obj] Error 1
CMakeFiles\Makefile2:532: recipe for target 'src/runtime/CMakeFiles/flatccrt.dir/all' failed
mingw32-make.exe[2]: *** [src/runtime/CMakeFiles/flatccrt.dir/all] Error 2
CMakeFiles\Makefile2:539: recipe for target 'src/runtime/CMakeFiles/flatccrt.dir/rule' failed
mingw32-make.exe[1]: *** [src/runtime/CMakeFiles/flatccrt.dir/rule] Error 2
Makefile:148: recipe for target 'flatccrt' failed
mingw32-make.exe: *** [flatccrt] Error 2

2. When included in a simple main.c

After I hacked the ifdef above I got it compiled but got to the next error in asignment. Also seems to be fixed by a correct cast after indexing.

CMakeFiles\check-flatccrt.dir\build.make:82: recipe for target 'CMakeFiles/check-flatccrt.dir/main.cpp.obj' failed
        CMakeFiles\Makefile2:94: recipe for target 'CMakeFiles/check-flatccrt.dir/all' failed
        Makefile:102: recipe for target 'all' failed

        In file included from Z:/objectbox-generator/third_party/flatcc/include/flatcc/flatcc_flatbuffers.h:26,
                         from Z:/objectbox-generator/third_party/flatcc/include/flatcc/flatcc_builder.h:68,
                         from C:\Users\User\AppData\Local\Temp\check-flatccrtcmake566681682\conf\main.cpp:3:
        Z:/objectbox-generator/third_party/flatcc/include/flatcc/flatcc_alloc.h: In function 'void __flatcc_aligned_free(void*)':
        Z:/objectbox-generator/third_party/flatcc/include/flatcc/flatcc_alloc.h:106:26: error: invalid conversion from 'void*' to 'char*' [-fpermissive]
          106 |     raw = ((void **)p)[-1];
              |           ~~~~~~~~~~~~~~~^
              |                          |
              |                          void*
        mingw32-make.exe[2]: *** [CMakeFiles/check-flatccrt.dir/main.cpp.obj] Error 1
        mingw32-make.exe[1]: *** [CMakeFiles/check-flatccrt.dir/all] Error 2
        mingw32-make.exe: *** [all] Error 2

        exit status 2
mikkelfj commented 4 years ago

Flags are not necessarily defined. The headers allow for them to be defined. You can do it with -D FLATCC_USE_GENERIC_ALIGNED_ALLOC on the compiler command line, or you can customize flatcc_rtconfig.h

On the first error, maybe there is a missing header include, or maybe the compiler is between two chairs: aligned_alloc is a C11 feature that wasn't in glibc for the longest time, but as I recall it should not require any special headers. You could try compile with -std=c11 to see if that changes anything. Maybe something needs to be updated here - much of the logic is around dealing with glibc not doing what it should be doing here.

I can't really help more, since it requires the system that you are using. If you can figure out what needs to be patched, if you care enough, I can add it if it won't break other stuff.

As to the last error: 'void*' to 'char*' that sounds really odd, but with gcc all warnings and errors are odd these days. What compiler version are you using, and have you disabled -pedantic? Not sure what -fpermissive means, as it doesn't sound very permissive.

Again, I can't do much from here. It depends on the specific compiler, and gcc has a policy of being very obnoxious because they think everyone only compiles with gcc.

vaind commented 4 years ago

The compiler is GCC 9.2. I'll look into the rest of your comment, thanks

mikkelfj commented 4 years ago

doh, I could just read the CMake output to get the compiler version, but thanks :)

mikkelfj commented 4 years ago

Here is a 13 day old build on the flatcc ci-more branch. This branch isn't built regularly except up to a release, as it covers a lot of compiler versions. In this build a gcc 9.3 compiler passes on Ubuntu 18.04:

whole build: https://travis-ci.org/github/dvidelabs/flatcc/builds/696593748

subsystem job under build: gcc-9 (Ubuntu 9.3.0-11ubuntu0~18.04.1) 9.3.0 https://travis-ci.org/github/dvidelabs/flatcc/jobs/696593751

vaind commented 4 years ago

Yes, I'm aware this works fine on Ubuntu, it's certainly related to MinGW (windows). I'll try some more stuff shortly and let you know if I can get it working after all

mikkelfj commented 4 years ago

Is _WIN32 defined? That could potentially throw the system off if not using MSVC.

mikkelfj commented 4 years ago

I found this in your CMake log:

Configured C_FLAGS: -DFLATCC_REFLECTION=1 -std=c11 -pedantic -Wall -Wextra -Wno-stringop-truncation -Wno-format-overflow -Werror -Wno-unused-function -Wsign-conversion

You definitely need to remove -pedantic. That is officially not supported as of gcc 8.0 as it forces things to break, and too much maintenance each gcc release forward. As for other settings, see comments in the flatcc CMakeList.txt on gcc.

-Wsign-conversion support was recently added, at least for clang, but I'm not sure about gcc.

vaind commented 4 years ago

-pedantic is set up by default by the tdm-gcc but they're just warnings, no errors so won't help here.

Win32 is defined

I'm trying to track the code path being executed, it's getting here: image

however, that doesn't end up generating any code because it's not MSVC image

I'll check if I can get it to work by defining FLATCC_USE_GENERIC_ALIGNED_ALLOC

mikkelfj commented 4 years ago

Did you notice:

#if 0
#define PORTABLE_DEBUG_ALIGNED_ALLOC
#endif

If you flag that in, you should get more information.

vaind commented 4 years ago

Not really, it'll just error out on line 99 :/

portable/paligned_alloc.h:99:2: error: #error "DEBUG: C11_ALIGNED_ALLOC configured"
   99 | #error "DEBUG: C11_ALIGNED_ALLOC configured"
mikkelfj commented 4 years ago

I'm wondering why this doesn't trigger, given -stcd=c11

#if defined (_ISOC11_SOURCE)
/* glibc aligned_alloc detection. */
#define PORTABLE_C11_ALIGNED_ALLOC 1
#elif defined (__GLIBC__)
...

Or maybe it does trigger, but it doesn't work, because of MINGW?

mikkelfj commented 4 years ago

I posted at the same time as you. It confirms it. So, the key is to figure out how to either not detect C11 support in this case, or to figure out why the code breaks when there is support.

mikkelfj commented 4 years ago

I think the problem is on line 44

#if defined (_ISOC11_SOURCE)
/* glibc aligned_alloc detection. */
#define PORTABLE_C11_ALIGNED_ALLOC 1
#elif defined (__GLIBC__)

The _ISOC11_SOURCE might be defined which overrides the glibc version check and thereby claims there is a built-in aligned_alloc.

Maybe this is correct. But it still causes an error. I suspect it is correct since there is a mention of built-in aligned_alloc in your log.

Can you check if you have a path where stdlib.h is not included where it is needed?

Z:\objectbox-generator\third_party\flatcc\src\src\runtime\builder.c:21:1: note: include '<stdlib.h>' or provide a declaration of 'aligned_alloc'
   20 | #include "flatcc/flatcc_emitter.h"
  +++ |+#include <stdlib.h>
vaind commented 4 years ago

I'm wondering why this doesn't trigger, given -stcd=c11

#if defined (_ISOC11_SOURCE)
/* glibc aligned_alloc detection. */
#define PORTABLE_C11_ALIGNED_ALLOC 1
#elif defined (__GLIBC__)
...

Or maybe it does trigger, but it doesn't work, because of MINGW?

Well even after changing to #if defined (__STDC_VERSION__) && __STDC_VERSION__ >= 201112L, which should be the thing you were looking for, semantically (C 11), it still doesn't help of course, since PORTABLE_C11_ALIGNED_ALLOC isn't used later to define anything

mikkelfj commented 4 years ago

Well, <stdlib.h> is always included in paligned_alloc.h, so that doesn't make any sense.

vaind commented 4 years ago

Is PORTABLE_C11_ALIGNED_ALLOC meant to mean there IS already an existing aligned_alloc? Just trying to understand

mikkelfj commented 4 years ago

Exactly

vaind commented 4 years ago

looks like mingw doesn't have one: https://github.com/ebassi/graphene/issues/83

mikkelfj commented 4 years ago

Then mingw shouldn't define _ISOC11_SOURCE but there we go.

Can you propose a detection mechanism? This is fragile if MingW defines it later.

vaind commented 4 years ago

I've searched among system headers and aligned_alloc it isn't defined anywhere.

I've got it working for now using export CFLAGS=-DFLATCC_USE_GENERIC_ALIGNED_ALLOC=1 but am not really sure what would be the best option

mikkelfj commented 4 years ago

I'm gonna push a branch shortly for you to test

mikkelfj commented 4 years ago

Please have a look at https://github.com/dvidelabs/flatcc/tree/mingw

It is not tested on MingW of course, but it is a start.

vaind commented 4 years ago

Yep, that commit helps and RT lib now compiles.

mikkelfj commented 4 years ago

Thats great, I'll merge that to master. I just added a commit with a non-MingW line I accidentally deleted.

Note that you will need aligned_free for MingW (non-standard), but _aligned_malloc does not support free.

vaind commented 4 years ago

Thx, feel free to close this

mikkelfj commented 4 years ago

OK, what about the other warning/error?

And can you please recheck on master to ensure things are ok?

mikkelfj commented 4 years ago

Ah the other issue was aligned_free missing. You can close when master is confirmed. Thanks for raising this issue and debugging.

vaind commented 4 years ago

seems to work fine

vaind commented 4 years ago

thx again