Thoughts about separate tools build-step (was: Need help with fips-flatbuffers)

[mgerhardy]

It would be nice if you could support me with https://github.com/mgerhardy/fips-flatbuffers

The problem is that flatc can't specify the filename - but only the path - also flatc isn't found properly.

[mgerhardy]

used this line btw. fips_generate(FROM monster.fbs TYPE FlatC HEADER monster_generated.h)

but i still get src/server/monster.cc and src/server/monster.h generated.

https://github.com/google/flatbuffers/blob/master/samples/monster.fbs

[mgerhardy]

and another btw. flatc is put into a directory like this: /home/mattn/dev/fips-deploy/Project/linux-make-debug/flatc

fips_project(flatbuffers) doesn't help - a whole fips_setup() fips_finish() cycle inside a dependency doesn't work of course, too.

[floooh]

I'll play around with this a bit in the next days. I haven't thought of the case that a tool needed for code generation needs to be compiled first, but the bgfx shader compiler is a similar situation. I think the best solution is a new fips target type (fips_begin_tool / fips_end_tool or so...), and then either compile this during the cmake run (cmake does this too for checking whether the C compiler works), or run it as a pre-build-step before code generation...

[code-disaster]

I have some experimental build running, because I've seen your module and want to learn more about flatbuffers.

So far I changed CMakeLists.txt to build flatc as a fips project if run from ./fips-flatbuffers:

if (NOT FIPS_PROJECT_DIR)
    message("Building 'flatc' compiler ...")
    cmake_minimum_required(VERSION 2.8)
    get_filename_component(FIPS_ROOT_DIR "../fips" ABSOLUTE)
    include("${FIPS_ROOT_DIR}/cmake/fips.cmake")
    set(FIPS_EXCEPTIONS ON)
    fips_setup()
    fips_project(flatbuffers)
    fips_begin_app(flatc cmdline)
... includes and source files omitted ...
    fips_end_app()
    fips_finish()
else()
    # todo: setup for projects *using* fips-flatbuffers
endif()

What's missing is how to point the generator to the flatc binary. Right now I hacked the generator to point to the fips-deploy folder.

Building a code generation tool as part of the project itself would be very nice though. In this case, I'm not sure if fips already supports to enable/disable exceptions per application (flatc requires them).

[floooh]

I've been thinking about this a bit but haven't done any 'coding' yet. The advantage of building tools as pre-build-step is that it just happens automatically for the first build. But there are a few problems:

1. as you found out: specific compile settings which are needed for the build tool, but not for the project
2. cross-compiling: when cmake is started with a toolchain-file for cross-compiling, but the tools must still be compiled natively for the host platform (don't even know whether this is possible with cmake)
3. a good place for the compiled tools would be the ${CMAKE_BINARY_DIR}, which is the top-level build directory, this is the same for the entire project

If this is too tricky to solve (esp (2)), then a separate './fips build-tools' action would be a fairly clean solution. You would still define tools inside a fips_begin_tool()/fips_end_tool() macro, but this would be ignored during normal builds, and vice versa, when building tools, all other fips_begin_xxx/fips_end_xxx would be ignored.

Currently in oryol, I simply have checked in precompiled tools for Windows/OSX/Linux, this would be the least elegant solution for fips-flatbuffers.

[mgerhardy]

Maybe the best would be to build tools 'outside' of fips - just triggering the cmake based build and put the resulting binary into CMAKE_INSTALL_PREFIX/bin

The point here is that this should even be extended to tools that don't use cmake as build system, but e.g. are Makefile based.

fips itself would then 'only' be used for cloning the repo and prodiving the py generators.

[mgerhardy]

I changed this to add precompiled flatc binaries for linux and windows. Would be glad if someone could maybe provide a mac binary for flatc and maybe provide a pull request.

[mgerhardy]

thanks for the osx binary - should this stay open? I mean, the points is imo still valid.

[floooh]

I would prefer a new ticket since I'm having trouble figuring out what exactly is needed :) Is it a better way to integrate tool compilation, or a better way to integrate pre-built tools, or more freedom in specifying the target file locations?

[floooh]

Some new discussion here: https://github.com/floooh/fips-bgfx/pull/4, had the idea of separate subprojects, with the build targets below... not sure yet whether this overcomplicates things though.