How to add files created by a rule as a src to a c.program?

[devurandom]

I need to generate a C file by means of a rule. This file then needs to be set as the src of a c.program.

The code (lakefile) I currently have is:

RE2C = 'utils/re2c/re2c/build/re2c'
LEMON = 'utils/lemon/lemon'

lemon_rule = rule('.y', '.c', '$(LEMON) -s $(INPUT)')
re2c_rule = rule('.re.c', '.c', '$(RE2C) -o $(TARGET) $(INPUT)')

test = c.program{
    'test',
    src={
        'src/main.c',
        lemon_rule 'src/parser.y',
        re2c_rule 'src/scanner.re.c',
    }
}

default{test}

However, this results in:

lake: ./utils/lake/lake:518: attempt to call method 'gsub' (a nil value)

How do I write this properly?

[stevedonovan]

Currently it is a little awkward to express this. Here's a simple scenario that works:

-- rule that converts .l to .c files
L = rule('.l','.c','lua prepro.lua $(INPUT) $(TARGET)')

-- add targets to the rule
L 'lib.l'
L 'util.l'

-- (can also say L '*.l')

SRC = L:get_targets()
SRC[#SRC+1] = 'main'

prog = c.program{'main', src = SRC }

default {prog}

It's awkward because you have to explicitly get the generated targets, and add the other source file.

Another approach is to keep the files as a string and explicitly mention them again:

-- defining a new 'language'
L = {ext='.l',obj_ext='.c'}
L.compile = 'lua prepro.lua $(INPUT) $(TARGET)'
lake.add_group(L)

LFILES = 'lib util'
lfiles = L.group{src=LFILES}

prog = c.program{'main', src = 'main '..LFILES }

default {prog}

I'm thinking of extending the definition of src so that it can contain things other than source files. Then they can also be targets, or rules (and then the get_targets is implicit).

What do you think? I'm coming out with a new version soon, so I'm interested to know what other pain points you've hit!

[devurandom]

It would definitely be nice if the src parameter could receive other targets directly, yes.

I am still trying to get my first Lakefile to work, so I have not yet done much with Lake.

Anyway, I have some more questions:

1. My program consists of C and C++ sources which create one program. Do I need to create a c and a cxx library and link them together into a cxx program?
2. Then I have to create a rule which executes multiple commands. Is there any simpler way than having to create a function that calls os.execute and utils.substitute? (s.b.)
3. How do I specify CFLAGS? It seems defining the variable alone is not sufficient, and the cpp.program does not take a cflags parameter either…
4. I think that the extension replacement code for rules has a bug where it does not replace the whole extension. rule('.b.c', '.c', …) applied to file 'a.b.c' creates a file 'a.b.c' instead of 'a.c'.

Anyway, this is the code I currently have (which does not work):

RE2C = 'utils/re2c/re2c/build/re2c'
LEMON = 'utils/lemon/lemon'

CFLAGS = '-I src'

lemon_c_rule = rule('.y', '.c', '$(LEMON) -s $(INPUT)')
lemon_cpp_rule = rule('.y', '.cpp', function(t)
    os.execute(utils.substitute('$(LEMON) -s $(INPUT)', {LEMON=LEMON, INPUT=t.deps[1]}))
    os.execute(utils.substitute('$(MV) $(TARGET_).c $(TARGET_).cpp', {MV='mv -f', TARGET_=t.target:gsub('\.cpp$', ''):gsub('^', 'src/')}))
end)
re2c_rule = rule('.re.c', '.c', '$(RE2C) -o $(TARGET) $(INPUT)')

lemon_cpp_rule 'src/parser.y'
re2c_rule 'src/scanner.re.c'

test_src = {'src/main'}
for i,v in ipairs(lemon_c_rule:get_targets()) do v,_ = v:gsub('^', 'src/') table.insert(test_src,v) end
for i,v in ipairs(lemon_cpp_rule:get_targets()) do v,_ = v:gsub('^', 'src/') table.insert(test_src,v) end
for i,v in ipairs(re2c_rule:get_targets()) do table.insert(test_src,v) end

test = cpp.program{
    'test',
    src=test_src,
}

default{
    test,
}

[stevedonovan]

I think it's easier to work on a slightly lower level, directly with targets:

LEMON='lemon'
MV='mv'
RES2C='res2c'

c_parser = target('src/parser.c','src/parser.y','$(LEMON) -s $(DEPENDS)')
cpp_parser  = target('src/parser.cpp',c_parser.target,'$(MV) $(DEPENDS) $(TARGET)')
res2c = target('src/scanner.c','src/scanner.re.c','$(RES2C) -o $(TARGET) $(DEPENDS)')

src = {cpp_parser.target, res2c.target, 'src/main'}

test = cpp.program{'test',src=src,incdir='src'}

default {test}

And this gives me with lake -t the following output (-t because I don't actually have a lemon)

lemon -s src/parser.y
mv src/parser.c src/parser.cpp
g++ -c -O1 -Wall -Isrc  -MMD  src/parser.cpp -o parser.o
res2c -o src/scanner.c src/scanner.re.c
g++ -c -O1 -Wall -Isrc  -MMD  src/scanner.c -o scanner.o
g++ -c -O1 -Wall -Isrc  -MMD  src/main.cpp -o main.o
g++ parser.o scanner.o main.o  -Wl,-s -o test.exe

Note how the include directory is specified. You can pass flags directly, but the field name is 'flags', which I admit is confusing (better to have also an alias 'cflags')

This approach is ok if you only have one of any particular kind - rules are useful for doing lots of operations that match a pattern. I suppose the moral is: when in doubt, think like Make ;)

I've had to explicitly get the target file with 't.target' but the src list will soon also understand targets directly.

As for mixed C/C++, no problem. The compiler knows what to do and just compiles everything as C++.

steve d.

[devurandom]

c_parser = target('src/parser.c','src/parser.y','$(LEMON) -s $(DEPENDS)')

What I just noticed: rule() and target() have their parameters in opposite order, which confused me originally when I changed from target() to rule() and the lakefile worked even less than before.

As for mixed C/C++, no problem. The compiler knows what to do and just compiles everything as C++.

That might be an issue, when e.g. mixing in C99, which afaik a C++03 compliant compiler does not need to understand. In addition: Won't compiling it as C++ mess up symbol names?

[stevedonovan]

What I just noticed: rule() and target() have their parameters in opposite order

Ouch. That's not a nice inconsistency.

Ah, and yes, sometimes you do need to separately compile C and C++. (there is a c99 available that forces GCC to compile as C99)

So we can partition the build in two, like so:

cfiles = c99.group{'cfiles',src='src/one src/two',incdir='src'}
cppfiles = cpp.group('cppfiles',src='src/three src/four',incdir='src'}
prog = cpp.program('test',input={cfiles,cppfiles},libs='foo'}

This separates out the compile and the link stage and allows them to be separate, with their own flags etc.