cmake: flatcc_generate_sources + add_custom_command + crossbuild + ci

[madebr]

This pr does the following:

• use add_custom_command to generate the sources. Add all generated sources literally to OUTPUTS. The key to don't have sources recompile is to add the generated sources to add_executable. The flatcc maybe has an argument to return what files are generated, but this is work for #169. flatcc should have an option that returns the generated sources (and nothing else). e.g.:

  flatcc -a monster.fbs
  flatbuffers_common_builder.h flatbuffers_common_reader.h monster_builder.h monster_reader.h monster_verifier.h

  This information + flatcc -d monster.fbs allows to know the dependant sources + generated sources.

• Create aliases (flatcc::runtime, flatcc::libflatcc, flatcc::cli) for the 2 libraries + executable, and use it everywhere. make install will also create these 3 targets. This means that the script by #169 can be used by flatcc itself + users of flatcc (by using imported targets). So flatcc can be used as:

  find_package(flatcc REQUIRED)
  add_custom_command(OUTPUTS genrated.h
      COMMAND flatcc::cli input.fbs ...)
  add_executable(myexe source.c generated.h)
  target_link_libraries(myexe PRIVATE flatcc::runtime)

I have tested it on cmake 2.8.12.1 (with gmake and ninja).

[mikkelfj]

It is called fc.exe for file compare, and not "fc" because that means something else in cmd.exe interpreter.

[madebr]

Ok, I'll remove the feedback. It looks like github actions and appveyor have bash + diff. So python is not tested either way. Appveyor is giving problems with (I think) endlines (\r\n vs \n).

[madebr]

Btw, the reference files are now removed from this pr. I'll open a pr when this one is finished.

[mikkelfj]

Windows has bash and some other unix tools wherever there is git, but GH actions isn't the only environment, hence best to keep dependencies to a minimum.

[madebr]

Remind me if I've forgotten to add fc.exe support on Windows when that pr gets opened. :smile:

[mikkelfj]

But I still don't think it is necessary to rework all the tests. If I were rewriting all the JSON tests today, I would consider it. I use diff to test scripting languages because there are so many weird expressions to test that step by step test cases would explode.

[madebr]

You don't need to rewrite any test? What it does is capture the stdout/stderr of every test, and compare it to reference files. These reference files can be generated by a make target.

[mikkelfj]

The one test that would benefit from diff, is the one with the little whale. The cgen test. Currently it only tests that it is running, not that it is running correctly. And this is exactly a scripting language (or schema language) test.

[mikkelfj]

Well, let's take that in a separate PR.

[madebr]

Ok, can you look at #172? That is the only diff that I could detect between runs on my pc and on runs on CI.

[wdobbe]

I created a new PR to @madebr 's cmake_dep branch. It makes cross-compilation work.

Also updated the README.md:

• moved the cmake build info further down as requested by @mikkelfj
• updated the section about cross compilation

[mikkelfj]

@madebr if you merge master, travis should build again.

[madebr]

I wonder if this flag belongs in the main CMakeList.txt file since you could compile mingw natively on Windows too?

Let's keep in the back of our head when/if you test it on mingw@WIndows?

[mikkelfj]

Let's keep in the back of our head when/if you test it on mingw@WIndows?

right, good find btw.

[mikkelfj]

In the mingw cross build:

-- Installing: /home/runner/work/flatcc/flatcc/prefix_host/lib/libflatccrt.dll.a -- Installing: /home/runner/work/flatcc/flatcc/prefix_host/bin/libflatccrt.dll -- Installing: /home/runner/work/flatcc/flatcc/prefix_host/lib/libflatcc.dll.a -- Installing: /home/runner/work/flatcc/flatcc/prefix_host/bin/libflatcc.dll -- Installing: /home/runner/work/flatcc/flatcc/prefix_host/bin/flatcc.exe

is dll.a a new word for .lib?

[madebr]

On Windows, shared libraries (dll's) have an import library. The import library contains the exported symbols, users of the library can use.

MSVC default to use .lib for static libraries, and GNU .a. The convention for mingw is to use .dll.a for these import libraries. That way, simultaneous builds of static and shared builds (which is the default of libtool) don't clash.

[mikkelfj]

That way, simultaneous builds of static and shared builds (which is the default of libtool) don't clash.

OK

[madebr]

That way, simultaneous builds of static and shared builds (which is the default of libtool) don't clash.

OK

Note that using -lsome_library will match both libsome_library.dll.a and libsome_library.a. I think the linker prefers the import library, but I'm not sure. I guess using -Bshared and -Bstatic, you can select between the two.

[madebr]

Let's keep in the back of our head when/if you test it on mingw@WIndows?

right, good find btw.

So, gcc mingw 8 from chocolatey needs the workaround: https://github.com/madebr/flatcc/runs/1592328584?check_suite_focus=true

gcc mingw 10 from msys2 does not need the workaround: https://github.com/madebr/flatcc/runs/1592230351?check_suite_focus=true

The proper way to add this flag is to do a try_compile. Or the simpler way is to just add the flag when MINGW is true-ish.

This is the result of the current cmake_dep branch: https://github.com/madebr/flatcc/actions/runs/437106453

[prince-chrismc]

If this gets merged the would would benefit from a release ❤️

[mikkelfj]

The proper way to add this flag is to do a try_compile. Or the simpler way is to just add the flag when MINGW is true-ish.

Adding the flag is the right way. No need to not set it on more recent compilers since no-one in their right mind would add MSVC specific formatters to the source tree outside of PRI macros.

FlatCC has not had the need of a single try_compile despite being used on rather different platforms, and I'd like to keep it that way.

[mikkelfj]

@prince-chrismc

If this gets merged the would would benefit from a release ❤️

Yes, it is about time for a release, also because of new optional scalar feature. But I try to not release immediately after making larger changes.

[madebr]

I rebased on top of master.

@mikkelfj To avoid bitrot, can you please take a(nother) look at this?

[mikkelfj]

Is it ready? I thought you had abandoned it.

[madebr]

Is it ready? I thought you had abandoned it.

I think so, I stopped pushing changes because I ran out of ideas 😄 I'm looking at the travis error atm.

[mikkelfj]

OK, I'll have a look at some point. But I'll need to make a release before merging. Looking forward to CI fixes.

[madebr]

I messed up my latest force push, but was able to recover using c723a5e.

[mikkelfj]

Sorry for not following up on this - we know have a functioning CI build again via #250. As discussed in #217, https://github.com/dvidelabs/flatcc/pull/217#issuecomment-1145862231 this PR is probably doing too much at a time.

A new PR #258 has been made. It also addresses the rebuild too much issue.

@madebr and @midokura-xavi92 would mind having a look at this PR? It seems to be a fairly simple change, although it does depend on all output of a directory.

[mikkelfj]

I finally got some time to work on flatcc. I'm going to close this PR because we have several build updates going on including PR #258. You are still welcome to comment on that PR and introduce your ideas, but I can't have two major PR on the build system. The plan is make a release soon, before major updates to the build system.