lake doesn't handle source files with the same name in different directories

[bogdanm]

In one of my projects, I have an executable file that builds from a number of C source files. Amongst these are are two 'main.c' files. lake compiles only one of them, then includes it twice in the linker step, which results in an error:

gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD mux_src/main.c -o .build/mux/main.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/server.c -o .build/mux/server.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/log.c -o .build/mux/log.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/deskutils.c -o .build/mux/deskutils.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/rfs_transports.c -o .build/mux/rfs_transports.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/os_io_posix.c -o .build/mux/os_io_posix.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/serial_posix.c -o .build/mux/serial_posix.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD rfs_server_src/net_posix.c -o .build/mux/net_posix.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD src/remotefs/remotefs.c -o .build/mux/remotefs.o gcc -c -O2 -Wall -DRFS_UDP_TRANSPORT -DRFS_INSIDE_MUX_MODE -Imux_src -Irfs_server_src -Iinc -Iinc/remotefs -m32 -O0 -g -MMD src/eluarpc.c -o .build/mux/eluarpc.o gcc .build/mux/main.o .build/mux/main.o .build/mux/server.o .build/mux/log.o .build/mux/deskutils.o .build/mux/rfs_transports.o .build/mux/os_io_posix.o .build/mux/serial_posix.o .build/mux/net_posix.o .build/mux/remotefs.o .build/mux/eluarpc.o -lm -m32 -Wl,-s -o .build/mux/mux.temp .build/mux/main.o: In function main': /home/bogdanm/work/elua/elua/mux_src/main.c:154: multiple definition ofmain' .build/mux/main.o:/home/bogdanm/work/elua/elua/mux_src/main.c:154: first defined here collect2: error: ld returned 1 exit status lake: failed with code 256

(my other main.c is in rfs_server_src/main.c)

The obvious way to fix this is to avoid duplicate source file names in the project (which is a good idea anyway), but sometimes this isn't a good option (for example when using external libraries). One solution that I'm using in my builder is to make the object name derive from both the source name and its relative directory. For example:

mux_src/main.c -> mux_srcmain.o rfs_server_src/main.c -> rfs_server_srcmain.o

This should make the object name unique in the project. The other obvious idea is to replicate the source tree structure in the output directory (mux_src/main.o, rfs_server_src/main.o), but I find the previous solution easier to implement.

[stevedonovan]

Having an option to have a non-flat output directory is a good way (and could be yet another user-selectable option), but I like your idea; a build tool should track object files with the same basename and do something to make the result unambiguous. The first step is to check whether any generated target names so collide, and patch the name them if so. But let me do this carefully, so no Sunday night hacks ;)

[bogdanm]

Thank you for your reply. Personally, I would make the non-flat output directory implementation default, otherwise the user might get some weird errors. While working on compiling eLua with lake today, I kept on hitting this issue. For example, some of the drivers for the various peripherals have the same name as the corresponding eLua module (uart.c was the first that I found) and that led to linking errors, the main problem being that it's not obvious at all from these linking error what the actual problem is. About the implementation, that's obviously up to you, but in my builder the default is to always use the first solution that I proposed earlier (make the object file name reflect both the source file name and its directory). Even if there are no actual collisions, having an output file named 'srclualobject.o' instead of just 'lobject.o' still gives you a bit more information about the origin of the object file. I've actually tried to hack lake to do just that :), but I still have to get more familiar with the sources before I'm able to do that.

[stevedonovan]

The obvious point of customization is where the target name of a compile rule is created, at lake:1258.

    -- by default, target is created in cwd, but it can be put into a subdirectory
    if lang and lang.output_in_same_dir then
        -- this is used by compilers like javac which generate output files
        -- within the same directory as the source files
        target_name = replace_extension(input,r.out_ext)
    elseif  r.output_dir then
        local dir,name = r.output_dir,target_name
        if FULLNAME then
            name = dir:gsub('[/\\]','__'):gsub('%.','_')..'__'..name
        end
        target_name = join(dir,name)
    end

And everything works as expected for this simple case (building Lua from the directory below lua-5.1.5 using base and odir='release'.)

But then it occured to me that using base actually might solve our problem here, because then each source directory specified with base gets its own 'release' directory. Different values for base should not be a problem. There's a little problem with specifying base and odir=true (had to say odir='release' explicitly) which I'll track down.

The interactions between all these parameters get hairy, which is a classic documentation problem. We should aspire to produce software which is easy to document, I think. If a feature takes too long to explain it might well be a mis-feature?

[stevedonovan]

Latest push provides a solution, depending on global FULL_OUTPUTNAME. Have a look at the new examples/nameclash test:

lake.set_flags { FULL_OUTPUTNAME = true } c.program{'main',src='foo/main boo/main',defines='DEBUG'}

Which is not such a hacky solution as the groups solution (which is groups.lake in same dir)

[bogdanm]

Thanks a lot for your prompt fix. I guess I can mark this one as "closed".