search

lexxmark / winflexbison

Main winflexbision repository
GNU General Public License v3.0
414 stars 120 forks source link

Bison 3.1 has been released #22

Closed donmac703 closed 5 years ago

donmac703 commented 6 years ago

See announcement at [https://savannah.gnu.org/forum/forum.php?forum_id=9225] (dated Tue 28 Aug 2018 04:28:44 AM UTC)

donmac703 commented 6 years ago

Hi, Looks like there is a lot of activity going on to bring the generated scanner up to Modern C++ principles. Depending on effort involved on your side, it may be better to wait a little. See https://github.com/akimd/bison for latest commits.

While I am very interested in the Typed Mid-rule actions introduced in Bison 3.1, I am also using the generated scanner in a C++17 target...

GitMensch commented 6 years ago

@donmac703 The 3.1 is the latest release and contains both bug-fixes and new features. I have no idea why we should wait and don't think an inofficial bison fork at Github provides any reasons to do so.

lexxmark commented 6 years ago

upgrading win_bison is not a big deal (in comparing with win_flex). just need some free time to do this. :)

lexxmark commented 6 years ago

@donmac703 Please try this out. I will test it a bit as well. winflexbison.zip

GitMensch commented 6 years ago

@lexxmark :-) Wouldn't it be reasonable to push your changes to master and point donmac703 to the CI-build?

Croydon commented 6 years ago

When it's experimental than better another branch as master :)

GitMensch commented 6 years ago

Correct, if @lexxmark feels it is experimental, then it would be good to create a short-term 3.1 branch, apply the update there and point others to the created artifacts (thanks to #27 comparable with full releases).

GitMensch commented 6 years ago

What do we see now?

https://github.com/lexxmark/winflexbison/tree/bison3.1 with binaries at https://ci.appveyor.com/project/lexxmark/winflexbison

lexxmark commented 6 years ago

I'm pushing bison3.1 branch with upgraded bison code

lexxmark commented 6 years ago

There is a new branch bison3.1 appveyor is passed https://ci.appveyor.com/project/lexxmark/winflexbison/build/1.0.18

GitMensch commented 6 years ago

Is there any time schedule for integrating the branch into master?

donmac703 commented 6 years ago

Hi, The provided zip file has errors when run against my 7.5K line grammar file, which previously was without errors. It flags lines such as:

%define api.token.constructor %define api.value.type variant

with error: %define variable 'api.token.constructor' redefined

commenting out the lines that get this redefined error will lead to error C000 0409.

This can be reproduced by using the variant.yy provided in the examples folder of the Bison 3.1 distro.

lexxmark commented 6 years ago

Ok, will look into it.

lexxmark commented 6 years ago

I accidentally override my modifications with original bison code in uniqstr.c/h files. Please check again. winflexbison3.1.zip

donmac703 commented 6 years ago

Hi, Thanks for the quick response. I can confirm that I am now getting the expected summary results.

GitMensch commented 6 years ago

@donmac703 I was partly wrong with the "inofficial bison fork at GitHub": it very likely is from the Bison main developer (even if not up-to-date with the official repo found at http://git.savannah.gnu.org/cgit/bison.git) I still think it is reasonable to get version 3.1 into win_bison soon.

GitMensch commented 6 years ago

@lexxmark I've tried to run the tests from bison 3.1 (which builds fine under MinGW, creating unistd.h along with other headers and needed functions in "lib" as needed).

Result with 3.0.5:

C:/dev/bison-3.1/examples/variant.yy:82.34: error: invalid character: '>'
C:/dev/bison-3.1/examples/variant.yy:91.3-8: error: invalid directive: '%empty'

This may because of a new feature or because win_bison 3.0.5 contains a bug. Retested with your most up-to-date win_bison 3.1:

$ make check BISON=/dev/win_flex_bison-3.1/win_bison.exe
  GEN      public-submodule-commit
make  check-recursive
make[1]: Entering directory `/dev/bison-3.1/build_msys'
Making check in po
make[2]: Entering directory `/dev/bison-3.1/build_msys/po'
make[2]: Nothing to be done for `check'.
make[2]: Leaving directory `/dev/bison-3.1/build_msys/po'
Making check in runtime-po
make[2]: Entering directory `/dev/bison-3.1/build_msys/runtime-po'
make[2]: Nothing to be done for `check'.
make[2]: Leaving directory `/dev/bison-3.1/build_msys/runtime-po'
Making check in .
make[2]: Entering directory `/dev/bison-3.1/build_msys'
make  examples/variant.exe examples/calc++/calc++.exe examples/mfcalc/mfcalc.exe examples/rpcalc/rpcalc.exe /dev/win_flex_bison-3.1/win_bison.exe tests/atconfig tests/atlocal
make[3]: Entering directory `/dev/bison-3.1/build_msys'
  YACC     examples/variant.cc
updating location.hh
updating position.hh
updating stack.hh
updating examples/variant.output
updating examples/variant.hh
  CXX      examples/variant-variant.o
examples/variant.cc:344:12: warning: extra tokens at end of #line directive
 #line 78 ""C:/dev/bison-3.1/build_msys/../examples/variant.yy"" // lalr1.cc:635
            ^
examples/variant.cc:351:12: warning: extra tokens at end of #line directive
 #line 78 ""C:/dev/bison-3.1/build_msys/../examples/variant.yy"" // lalr1.cc:635
            ^
examples/variant.cc:358:12: warning: extra tokens at end of #line directive
 #line 78 ""C:/dev/bison-3.1/build_msys/../examples/variant.yy"" // lalr1.cc:635
            ^
examples/variant.cc:365:12: warning: extra tokens at end of #line directive
 #line 78 ""C:/dev/bison-3.1/build_msys/../examples/variant.yy"" // lalr1.cc:635
            ^
  CXXLD    examples/variant.exe
  YACC     examples/calc++/parser.stamp
  CXX      examples/calc++/calc__-driver.o
  LEX      examples/calc++/scanner.cc
  CXX      examples/calc++/calc__-scanner.o
  CXX      examples/calc++/calc__-calc++.o
  CXX      examples/calc++/calc__-parser.o
examples/calc++/parser.cc:334:12: warning: extra tokens at end of #line directive
 #line 42 ""examples/calc++/parser.yy"" // lalr1.cc:635
            ^
examples/calc++/parser.cc:341:12: warning: extra tokens at end of #line directive
 #line 42 ""examples/calc++/parser.yy"" // lalr1.cc:635
            ^
examples/calc++/parser.cc:348:12: warning: extra tokens at end of #line directive
 #line 42 ""examples/calc++/parser.yy"" // lalr1.cc:635
            ^
  CXXLD    examples/calc++/calc++.exe
  YACC     examples/mfcalc/mfcalc.c
updating examples/mfcalc/mfcalc.output
updating examples/mfcalc/mfcalc.h
  CC       examples/mfcalc/mfcalc-mfcalc.o
examples/mfcalc/mfcalc.c: In function 'yy_symbol_value_print':
examples/mfcalc/mfcalc.c:648:12: warning: extra tokens at end of #line directive
 #line 27 ""C:/dev/bison-3.1/build_msys/examples/mfcalc/mfcalc.y"" /* yacc.c:684  */
            ^
examples/mfcalc/mfcalc.c:654:12: warning: extra tokens at end of #line directive
 #line 25 ""C:/dev/bison-3.1/build_msys/examples/mfcalc/mfcalc.y"" /* yacc.c:684  */
            ^
examples/mfcalc/mfcalc.c:660:12: warning: extra tokens at end of #line directive
 #line 26 ""C:/dev/bison-3.1/build_msys/examples/mfcalc/mfcalc.y"" /* yacc.c:684  */
            ^
examples/mfcalc/mfcalc.c:666:12: warning: extra tokens at end of #line directive
 #line 27 ""C:/dev/bison-3.1/build_msys/examples/mfcalc/mfcalc.y"" /* yacc.c:684  */
            ^
  CCLD     examples/mfcalc/mfcalc.exe
  YACC     examples/rpcalc/rpcalc.c
updating examples/rpcalc/rpcalc.output
updating examples/rpcalc/rpcalc.h
  CC       examples/rpcalc/rpcalc-rpcalc.o
  CCLD     examples/rpcalc/rpcalc.exe
make[3]: Nothing to be done for `/dev/win_flex_bison-3.1/win_bison.exe'.
make[3]: Nothing to be done for `tests/atconfig'.
make[3]: `tests/atlocal' is up to date.
make[3]: Leaving directory `/dev/bison-3.1/build_msys'
make  check-TESTS check-local
make[3]: Entering directory `/dev/bison-3.1/build_msys'
make[4]: Entering directory `/dev/bison-3.1/build_msys'
PASS: examples/variant.test
PASS: examples/calc++/calc++.test
PASS: examples/mfcalc/mfcalc.test
PASS: examples/rpcalc/rpcalc.test
============================================================================
Testsuite summary for GNU Bison 3.1
============================================================================
# TOTAL: 4
# PASS:  4
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

That looks not bad, I guess the extra " is an error in win_bison? @lexxmark: Can you investigate this, please? If it is a general bison issue I guess you can find and fix it + provide a patch for upstream Bison patches.

After the examples are built the real testsuite follows but all but the skipped tests fail because win_bison doesn't use binary mode for output and the output is verified.

@lexxmark: Is there an option and/or an environment value you check to use binary mode already? If not I'll try to create a PR for this as adding something like this allows to use the normal autoconf generated testsuite scripts. GnuCOBOL uses an environment value COB_UNIX_LF so we could add WINFLEXBISON_UNIX_LF to both binaries. Opinions?

lexxmark commented 6 years ago

That looks not bad, I guess the extra " is an error in win_bison?

Have you tested the last bison3.1 artifact win_flex_bison-dev-2017-Win32-Release.zip? I have experienced this problem because had "/data" folder outdated (it remained from master).

After the examples are built the real testsuite follows but all but the skipped tests fail because win_bison doesn't use binary mode for output and the output is verified.

I've changed fopen mode from "w"/"w+" to "wb"/"wb+" and it seems output files with unix line endings now. Please check. Note that "b" mode is non standard.

If users will complain about line-endings we will think of WINFLEXBISON_UNIX_LF solution.

GitMensch commented 6 years ago

Something got broken on the way, that's the current output from the examples:

=====================================
   GNU Bison 3.1: ./test-suite.log
=====================================

# TOTAL: 4
# PASS:  2
# SKIP:  0
# XFAIL: 0
# FAIL:  2
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: examples/variant
======================

Assertion failed: yytypeid_, file examples/variant.hh, line 196
variant: FAIL: 1 (expected status: 0, effective: 3)
FAIL examples/variant.test (exit status: 1)

FAIL: examples/calc++/calc++
============================

--(end of buffer or a NUL)
--accepting rule at line 59 ("toto")
--accepting rule at line 40 (" ")
--accepting rule at line 49 (":=")
--accepting rule at line 40 (" ")
--accepting rule at line 51 ("1")
Assertion failed: yytypeid_, file examples/calc++/parser.hh, line 195
calc++: FAIL: 1 (expected status: 0, effective: 3)
Assertion failed: yytypeid_, file examples/calc++/parser.hh, line 195
calc++: FAIL: 2 (expected status: 0, effective: 3)
Starting parse
Entering state 0
Reducing stack by rule 2 (line 49):
-> $$ = nterm assignments (1.1: )
Stack now 0
Entering state 2
Reading a token: Next token is token "identifier" (-:1.1: a)
Shifting token "identifier" (-:1.1: a)
Entering state 5
Reading a token: Next token is token ":=" (-:1.3-4: )
Shifting token ":=" (-:1.3-4: )
Entering state 11
Reading a token: Assertion failed: yytypeid_, file examples/calc++/parser.hh, line 195
calc++: FAIL: 3 (expected status: 0, effective: 3)
Assertion failed: yytypeid_, file examples/calc++/parser.hh, line 195
calc++: FAIL: 4 (expected status: 0, effective: 3)
Assertion failed: yytypeid_, file examples/calc++/parser.hh, line 195
calc++: FAIL: 5 (expected status: 1, effective: 3)
FAIL examples/calc++/calc++.test (exit status: 1)

The other tests got better, but many are still failing. The ones I've tested mainly for EOL in stdout/stderr - did you fix this, too? Example of the test failure:

-input-c.y:1.7: error: %code qualifier 'q' is not used
-input-c.y:2.7-9: error: %code qualifier 'bad' is not used
-input-c.y:3.7-9: error: %code qualifier 'bad' is not used
-input-c.y:4.7-12: error: %code qualifier 'format' is not used
+input-c.y:1.7: error: %code qualifier 'q' is not used
+input-c.y:2.7-9: error: %code qualifier 'bad' is not used
+input-c.y:3.7-9: error: %code qualifier 'bad' is not used
+input-c.y:4.7-12: error: %code qualifier 'format' is not used

Note: Those failures are also seen when building with MinGW-GCC. Can you please add a configurable switch for this both in flex and bison) as I'd like to suggest this as a patch upstream.

GitMensch commented 6 years ago

That looks not bad, I guess the extra " is an error in win_bison?

Have you tested the last bison3.1 artifact win_flex_bison-dev-2017-Win32-Release.zip? I have experienced this problem because had "/data" folder outdated (it remained from master).

The current version doesn't have a double-quote there, so this part worked.

lexxmark commented 6 years ago

Something got broken on the way, that's the current output from the examples: Assertion failed: yytypeid, file examples/variant.hh, line 196 Assertion failed: yytypeid, file examples/calc++/parser.hh, line 195

Could you please give me "examples/variant.hh" and "examples/calc++/parser.hh" files to see asserting code.

The other tests got better, but many are still failing. The ones I've tested mainly for EOL in stdout/stderr - did you fix this, too?

Do you still have EOL mismatch errors? Please give me an example to reproduce.

-input-c.y:1.7: error: %code qualifier 'q' is not used

Have you ensured that tests are compatible with win_bison 3.1 Please give me steps to reproduce.

Note: Those failures are also seen when building with MinGW-GCC. Can you please add a configurable switch for this both in flex and bison) as I'd like to suggest this as a patch upstream.

For what? Wrap my changes about "wb+"/"wb" modes with some macro?

lexxmark commented 6 years ago

I've released bison 3.1

Croydon commented 6 years ago

And this allowed me to remove the patches from the Conan recipe, awesome :)

https://github.com/bincrafters/conan-winflexbison/commit/8fcce2addf0fe96864fe9d93c16c550c140284ff

https://bintray.com/bincrafters/public-conan/winflexbison%3Abincrafters

winflexbison

GitMensch commented 6 years ago

@Croydon - completely unrelated to the issue but the bintray package misses description, license, website, VCS, issue tracker, README, Release Notes...

You may use the update for winflexbison to add those information :-)

donmac703 commented 5 years ago

Closing an issue for a previous release that Alex built for the community and was successfully used.