lex, yacc, and tar tools not compatible with nonGNU versions

[bdbaddog]

This issue was originally created at: 2003-04-21 23:05:00. This issue was reported by: aegis. aegis said at 2003-04-21 23:05:00

I thought about filing these as separate bugs, but it can be split later if need be. I would suggest either making specific bison and flex tools and converting the current lex and yacc ones to use the less-flexible standard UNIX tools.

I'm just filing this bug to track the issue. It's not high priority for me at this point.

cruncher:~/open/scons
$ python runtest.py test/LEXFLAGS.py
/usr/freeware/bin/python 
/mnt/rigby/home13/users/aegisk/open/scons/test/LEXFLAGS.py
/mnt/rigby/home13/users/aegisk/open/scons/src/script/scons.py
returned 2
STDOUT ============
scons: Reading SConscript files ...
scons: done reading SConscript files.
scons: Building targets ...
lex -b -t bar.l > bar.c
scons: done building targets.

STDERR ============ lex: ERROR: Illegal option -- b Usage: lex [-ctvnV] [-Q(y/n)] [files...] scons: *** [bar.c] Error 1

Traceback (innermost last): File "/mnt/rigby/home13/users/aegisk/open/scons/test/LEXFLAGS.py", line 118, in ? test.run(arguments = 'bar' + _exe) File "/mnt/rigby/home13/users/aegisk/open/scons/etc/TestSCons.py", line 141, in run raise TestFailed TestSCons.TestFailed

cruncher:~/open/scons $ python runtest.py test/YACC.py /usr/freeware/bin/python /mnt/rigby/home13/users/aegisk/open/scons/test/YACC.py /mnt/rigby/home13/users/aegisk/open/scons/src/script/scons.py returned 2 STDOUT ============ scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... yacc -d -o foo.c foo.y scons: done building targets.

STDERR ============ UX:yacc: ERROR: Illegal option -- o UX:yacc: TO FIX: Usage: yacc [-vVdlt] [-Q(y/n)] [-p driver_file] [-P driver_file] [-b file_prefix] file scons: *** [foo.c] Error 1

Traceback (innermost last): File "/mnt/rigby/home13/users/aegisk/open/scons/test/YACC.py", line 142, in ? test.run(arguments = 'foo' + _exe, stderr = None) File "/mnt/rigby/home13/users/aegisk/open/scons/etc/TestSCons.py", line 141, in run raise TestFailed TestSCons.TestFailed

cruncher:~/open/scons $ python runtest.py test/TAR.py /usr/freeware/bin/python /mnt/rigby/home13/users/aegisk/open/scons/test/TAR.py /mnt/rigby/home13/users/aegisk/open/scons/src/script/scons.py returned 2 STDOUT ============ scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... tar -c -f foo.tar file10 file11 file12 scons: done building targets.

STDERR ============ tar: archive file /dev/tape does not exist or is a regular file or invalid blocksize scons: *** [foo.tar] Error 1

Traceback (innermost last): File "/mnt/rigby/home13/users/aegisk/open/scons/test/TAR.py", line 124, in ? test.run(arguments = 'foo.tar', stderr = None) File "/mnt/rigby/home13/users/aegisk/open/scons/etc/TestSCons.py", line 141, in run raise TestFailed TestSCons.TestFailed

htgoebel said at 2003-07-08 05:03:37
>This is an related issus, so I'm fileing it into this bugreport: 
> 
>Support for byacc (Berkley yacc) is not working 
> 
>YACCCOM is defined as: 
```python
    YACCCOM="$YACC $YACCFLAGS -o $TARGET $SOURCES" 

but byacc does not accept an option '-o', but always creates 'y.tab.c' (and if -d is given, y.tab.h). These filenames do not have any relation to the input filenames (but see option -b below).

Accorrding to 'man yacc', these files may be generated:

y.tab.c     always 
y.tab.h     if option -d 
y.code.c    if option -r 
y.output    if option -v  # human readable 

The option '-b ' may be used to change the 'y' into something different. I suggest using this option if possible, since this would allow concurrent builds.

GNU make has the rule:

%.c: %.y 
#  commands to execute (built-in): 
$(YACC.y) $< 
mv -f y.tab.c $@ 

So it renames y.tab.c into the target file.

I think, there is a need to add something like this (make-like syntax, but array-indexes invented by me :-):

%.c %.h: %.y 
$(YACC.y) -d $< 
mv -f y.tab.c $@[0] 
mv -f y.tab.h $@[1] 

BTW: Shouldn't the target be deleted prior to the 'mv ... $@'? Otherwise this could become a security leak in conjunction with softlinks.

issues@scons said at 2003-07-08 05:03:37

Converted from SourceForge tracker item 725465

gregnoel said at 2008-04-14 00:07:29

For the most part, lex/flex (resp. yacc/bison) have compatible sets of options. Those that differ are mostly GNU two-dash flags parallel to the one-dash set.

The major difference is that flex (bison) allow the specification of an output file, while lex (yacc) place their output in files with fixed names in the current directory. GNU autoconf solves this by using command sequences when it can't find flex (bison): the processing command followed by move(s) to change the name of the output file(s). And the potential race condition of having two lexers (parsers) running at the same time is simply ignored.

Scons can do better. When using lex (bison) it can use SideEffect() to prevent race conditions and it can rename the output file(s) internally without needing an external command. But what it needs to do is recognize when it is using lex (yacc) and reconfigure itself internally to automatically Do The Right Thing.

If we really wanted to be clever, we could scan the options and deal with any incompatible flags, but autoconf doesn't try to be that smart, so I don't see it as necessary. Also, flex (bison) recognize options that cause the output to be modified so that multiple scanners (parsers) can be used in one program; these are simple string-to-string transforms that could be done by SCons at the time as file(s) are moved to their destination. Given that flex (bison) are so ubiquitous nowdays, this is also probably unnecessary.

Compatibility with tar is (literally) another issue (#1890), where the external tar command is being replaced by Python's tarfile module.

Hope this helps you.

gregnoel said at 2008-04-18 18:47:24

We shouldn't have UNCONFIRMED issues; changing them to NEW.

gregnoel said at 2008-05-30 13:38:28

Issue 1762 has been marked as a duplicate of this issue.

gregnoel said at 2008-05-30 13:40:38

Bug party triage.

gregnoel said at 2008-12-26 13:30:38

Adjust triage of issues.

russel said at 2009-04-15 01:43:10

Posted from message on the user email list:

By default, at least on Ubuntu, the "bison -d . . . " command transforms name.y to name.tab.c and name.tab.h. By default, SCons when faced with:

Program('name.y', YACCFLAGS=['-d'])

produces name.c and name.h. The problem here is that the name of the header file has to be explicitly known in the lexer file, and the default is name.tab.h. This seems to imply that to follow the default, in SCons I have to do:

environment = Environment(tools=['gcc', 'link', 'lex', 'yacc'])

parserFiles = environment.CFile(['parser.tab.c', 'parser.tab.h'], 'parser.y', YACCFLAGS=['-d'])

scannerFile = environment.CFile('scanner.c', 'scanner.l')
Depends(scannerFile, parserFiles)

environment.Program(['main.c', scannerFile, parserFiles], LIBS=['y'])

which seems a bit yukky. Am I just missing something?

Thanks.

tortoise_74 said at 2009-04-17 04:12:23

Note also that flex support C++ when invoked as flex++. To use it this way I currently do something like:

env = Environment(LEX='flex++ -f',CC='g++',CFLAGS='-O2 -pg -g',LINKFLAGS='-pg -g')
env.Append(CXXFLAGS=env.get('CFLAGS'))
lexer = env.Program(target='lexer',source='lexer.l')

garyo said at 2012-09-01 10:04:01

de-assigning all tickets assigned to Greg Noel (no longer working on SCons)

gregnoel said this issue is duplicated by #1762 at 2008-05-30 13:38:28.

[mwichmann]

Years later, the Tar builder still doesn't work on a non-POSIX tar, because -f is hardcoded in TARCOM, and is not recognized by a non-POSIX tar. One could conceive of defining a platform specific consvar, say TARDEVFLAG and define that to -f or f appropriately, but I'll just put this as another vote for implementing #1890.