innfeed inexplicable syntax errors after upgrade

[gwollman]

I recently upgraded (via binary package) from 2.6 to 2.7, after which innfeed now rejects innfeed.conf at the first peer block. For example, if innfeed.conf begins with (note first line is blank):

peer nntp.theworld.com {
        ip-name:        nntp.theworld.com
}

innfeed errors out with:

Aug 11 13:13:35 usenet innfeed[61136]: loading /usr/local/news/etc/innfeed.conf
Aug 11 13:13:35 usenet innfeed[61136]: ME config file error: line 2: syntax error

As this clearly should not be a syntax error, there seems to be something amiss with innfeed's configuration parser. I was able to get my site back feeding again by copying the old innfeed binary, along with libstorage and libinn shared libraries, from a snapshot of the filesystem.

The binary package was built locally with poudriere from the FreeBSD port news/inn at revision 14c8b9f26c9228b9eedce8289686328eee7fb4f6 and configured with --mandir=/usr/local/man --prefix=/usr/local/news --with-spool-dir=/usr/local/news/spool --with-log-dir=/var/log/news --with-tmp-dir=/usr/local/news/tmp --sysconfdir=/usr/local/news/etc --with-run-dir=/usr/local/news/run --with-db-dir=/usr/local/news/db --datarootdir=/usr/local/news/share --with-perl --without-blacklist --with-sqlite3=/usr/local --with-bdb=/usr/local --with-bdb-lib=/usr/local/lib --with-bdb-include=/usr/local/include/db5 --without-krb5 --enable-keywords --enable-largefiles --with-openssl=/usr/local.

Not sure if I should be filing the issue here on with the FreeBSD port, but FreeBSD includes only one trivial patch which doesn't look to be related to innfeed at all.

[rra]

Argh, sorry, I misread your report originally. You're talking about innfeed.conf, not incoming.conf. Yes, ip-name is correct. That's very odd. Did you check the line endings on the file to make sure nothing weird happened?

[gwollman]

Standard newlines. Note that the error points to the line with peer on it, not the ip-name line. hd shows no unusual characters:

00000db0  0a 70 65 65 72 20 6e 6e  74 70 2e 74 68 65 77 6f  |.peer nntp.thewo|
00000dc0  72 6c 64 2e 63 6f 6d 20  7b 0a 09 69 70 2d 6e 61  |rld.com {..ip-na|
00000dd0  6d 65 3a 09 6e 6e 74 70  2e 74 68 65 77 6f 72 6c  |me:.nntp.theworl|
00000de0  64 2e 63 6f 6d 0a 7d 0a  0a 70 65 65 72 20 6e 65  |d.com.}..peer ne|

[Julien-Elie]

I've just tested an innfeed.conf file beginning like you with \npeer ... and see no error. innfeed works well (and I've properly restarted it so that it uses that modified configuration file).

I don't know why you get this error. Do you know which version of flex and bison were used to build the FreeBSD package? I'm using bison 3.7.5 and flex 2.6.4 from Debian bullseye (stable). These programs generate the innfeed.conf configuration parser.

I'm not aware of any changes related to the innfeed.conf parser between INN 2.6.x and INN 2.7.0 except for a reformatting of the source code with clang-format and a few fixes of GCC warnings. I've just had a look again at these changes, and I do not see anything that would have caused that change of behaviour. So I'm a bit puzzled...

Incidentally, I see that you're building INN with --enable-keywords; do you use them? (keywords parameter in inn.conf)

[Julien-Elie]

If you remove the first empty line, is your innfeed.conf correctly parsed on your system? Otherwise, is it possible to attach here the file so that I can try it?

Does a run of inncheck report something?

[gwollman]

inncheck does not say anything about innfeed.conf. (Well, it complained about its ownership, but when I chowned it, inncheck didn't report anything else.)

innfeed errors out at the same statement whether it's on the first line in the file, the second, or indeed after the big long set of comments distributed in innfeed.conf.sample, which is how I first noticed it.

This package was built against the standard FreeBSD (b)yacc 1,9 and (f)lex.2.5.37. --enable-keywords is turned on by default in the port and I did not change this.

[Julien-Elie]

Thanks for your responses. I admit I do not know well how to investigate further this issue.

I've just tried a fresh install of INN 2.7.0 on FreeBSD 13.0-RELEASE-p3 (provided in the GCC Compile Farm as gcc303 machine):

wget https://ftp.isc.org/isc/inn/inn-2.7.0.tar.gz.asc
tar xvzf inn-2.7.0.tar.gz
cd inn-2.7.0
./configure --prefix=/usr/home/iulius/test-install --with-news-user=iulius --with-news-group=iulius
make
make tests
make install

As I'm not root in the server, I have to provide my own user/group and a local installation directory for the installation to work. All the tests pass fine:

All tests successful, 22 tests skipped.
Files=63,  Tests=3777,  14.06 seconds (2.77 usr + 6.62 sys = 9.40 CPU)

Afterwards, I add a domain in /usr/home/iulius/test-install/etc/inn.conf (otherwise the server is complaining it does not manage to resolve it), then enter /usr/home/iulius/test-install/bin and run:

$ ./innfeed -C
loading /usr/home/iulius/test-install/etc/innfeed.conf
ME config: adding default value for key article-timeout: 600
ME config: adding default value for key response-timeout: 300
ME config: adding default value for key close-period: 86400
ME config: adding default value for key initial-connections: 1
ME config: adding default value for key max-connections: 2
ME config: adding default value for key max-queue-size: 20
ME config: adding default value for key streaming: true
ME config: adding default value for key drop-deferred: false
ME config: adding default value for key min-queue-connection: false
ME config: adding default value for key no-check-high: 95.000000
ME config: adding default value for key no-check-low: 90.000000
ME config: adding default value for key no-check-filter: 50.000000
ME config: adding default value for key port-number: 119
ME config: adding default value for key backlog-limit: 0
ME config: adding default value for key force-ipv4: false
ME config: adding default value for key backlog-feed-first: false
ME config: adding default value for key dynamic-method: 3
ME config: adding default value for key dynamic-backlog-filter: 0.700000
ME config: adding default value for key dynamic-backlog-low: 20.000000
ME config: adding default value for key dynamic-backlog-high: 50.000000
config loading succeeded.

Works as expected with the sample file. I've also uncommented a peer definition in the sample file, and it still works. Even running innfeed directly without the -C flag properly creates the innfeed.status file with that peer inside.

Maybe you could also try to locally install INN 2.7.0 with the few commands I gave in this message, and see whether you're still facing a problem?

Also note that blacklist support was added in INN 2.7.0 and successfully tested by a contributor running FreeBSD, and on production at nyheter.lysator.liu.se; so normally INN 2.7.0 works fine on that system, at least when built from sources. May there be a problem with the port?

[gwollman]

I'll keep looking. (Our machines run 12.3, not 13, but that's unlikely to be the issue.)

[gwollman]

Sorry I haven't had time to pursue this in the last couple of weeks, getting ready for the start of the fall term + vacation.

[Julien-Elie]

Did you manage to find time for more investigation?

[flashfingaz]

got the same problem. did a freebsd update (12.something to 13.1)today and all hell broke loose with innfeed spamming logs with complains about innfeed.conf from trying around it seems that there is an syntax error on everything, regardless if peer definition or any other key. even on sample config file, when I only uncomment the log-file key

did also try to create a new config file with empty peer block, same result. same for using sampel config and deleting everything before and uncommenting log-file key to make sure first line actually contains something useful.

[Julien-Elie]

@kev009 Do you have any clue about what could be happening?

[kev009]

I have seen no issues on FreeBSD 13.x with my site (although I run the inn-current pkg, it was similar to 2.7.0 for some time). I do have some global parameters before any peer definitions. Someone with the symptom will need to edit the lexer/parser usage to provide more debug detail on what it is rejecting.

[flashfingaz]

Someone with the symptom will need to edit the lexer/parser usage to provide more debug detail on what it is rejecting.

would volunteer, just need some help how to. I tried enabling debugging by placing a innfeed.debug into pathlog, but that did not provide any more information.

[flashfingaz]

aha! I just grabbed the latest 2.7.0 release from github and compiled it on my site without touching anything (so just ./configure && make). used the resulting innfeed binary to check my config and it passed without problems

[gwollman]

@flashfingaz are you building with GNU Bison or the standard yacc?

[flashfingaz]

yacc according to config.log:

configure:15419: checking for bison
configure:15449: result: no
configure:15419: checking for byacc
configure:15435: found /usr/bin/byacc
configure:15446: result: byacc

$ byacc -V
byacc - 1.9 20200330

[gwollman]

I finally had a bit of time to look into this. I can confirm that if the news/inn port is built with GNU Bison (USES+= bison) rather than the base-system yacc, it correctly parses innfeed.conf.

[Julien-Elie]

There's something I do not understand: @flashfingaz confirms a working FreeBSD build with Berkeley Yacc (byacc), and @gwollman with GNU Bison. What is not working exactly with the news/inn FreeBSD port? (is it built with another Yacc implementation than byacc or bison which both are reported to work?). Couldn't the port just enforce bison to fix the issue @kev009?

[kev009]

@Julien-Elie it works with byacc in the port. Still need more details from the reporter.

[Julien-Elie]

@gwollman @flashfingaz Have you had a chance to try the new INN 2.7.1 build for FreeBSD? Is the issue solved?

[gwollman]

I just added USES+=bison to the port Makefile and since that worked, I haven't thought any more about it. The port hasn't been updated to 2.7.1 yet (or at least hadn't as of the last time I did a package build, in early May).

[Julien-Elie]

@gwollman @flashfingaz Have you tried the 2.7.1 port?

[gwollman]

@gwollman @flashfingaz Have you tried the 2.7.1 port?

I am currently running 2.7.1_1 but our workflow for package builds means that my workaround to always use bison is carried forward through updates to the port's makefile until there is a merge conflict, so I have no idea if the unmodified port works with byacc.

[Julien-Elie]

@gwollman A new INN 2.7.2 version has just been released. If there's no longer any issue with the FreeBSD build, I propose to close this ticket in a few months.

[kev009]

I recall seeing some bugs fixed in C stdio buffering "recently" that may be fixed in i.e. FreeBSD 14.1 and 13.3 which could have been relevant.

Unfortunately, the small number who experienced this bug were unwilling to dig into the issue and myself and others are running without issue so I think it is safe to close it.