Supporting configuration by SQLite3 database, as an option

[AaronNGray]

I am suggesting the modernization of INN by moving configuaration files into SQLite3 or SQL database.

My first reasoning in this is the limitation of line length in readers.conf in relation to users and newsgroup names.

I have been looking at putting a REST based API on INN so it can be maintained from a web based UI. Using OpenAPI and possibly in Python using Flask.

If anyone is interested in this direction we could start another repo to design the API ?

[pmetzger]

I personally hate having configs that are not in text files. It makes administration much harder, especially automated administration. I've never understood why this is appealing to people or what the benefit is. You can fix things like line length limitations without needing to put things into a format that you can't process with easy command line tools.

[AaronNGray]

It makes administration much harder, especially automated administration.

I find the total opposite :) There is nothing with having it as an option on a flag, and maybe even on a compiler flag so to keep the overheads to a minimum. And the amount of information you are dealing with for modifying the configuration especially from a remote web based REST interface is far far smaller for each configuration transaction. The readers.conf file with user authentification can potentially be massive as well with private newsgroups.

[pmetzger]

A REST interface opens up yet another opportunity to break your security because there's yet another thing that you need to secure. Pushing file updates over something that is already in place like SSH is a lot safer and easier to keep secure. Regardless, nothing prevents you from creating a database and generating the existing config file as a report out of it.

There are circumstances where relational databases are the correct way to maintain a large number of machines; I once was part of a group that administered many thousands of machines that way. However, the configs themselves were generated from the database; the database was never directly queried for configuration information by end systems.

[AaronNGray]

A REST interface opens up yet another opportunity to break your security because there's yet another thing that you need to secure. Pushing file updates over something that is already in place like SSH is a lot safer and easier to keep secure. Regardless, nothing prevents you from creating a database and generating the existing config file as a report out of it.

I have no problem securing REST API's we use 2FA these days.

There are circumstances where relational databases are the correct way to maintain a large number of machines; I once was part of a group that administered many thousands of machines that way. However, the configs themselves were generated from the database; the database was never directly queried for configuration information by end systems.

Modern ways of programming and programming techniques make this seamless.

Besides all this is would be optional for the individual instance.

[pmetzger]

I have no problem securing REST API's we use 2FA these days.

The 2FA or other authentication isn't the issue. Part of the issue is implementation failures that open up unexpected holes; the server itself is now a potential target. Implementation mistakes in that server could be used to get RCE.

Bringing up the authentication as though that was the relevant problem reveals a significant hole in your understanding of security.

Modern ways of programming and programming techniques make this seamless.

If you say so. I don't agree.

[AaronNGray]

The 2FA or other authentication isn't the issue. Part of the issue is implementation failures that open up unexpected holes; the server itself is now a potential target. Implementation mistakes in that server could be used to get RCE.

Modern coding techniques and frameworks are no longer subject to RCE's.

And as an attack vector anyway it's of little interest to anyone trying to make money other than SPAMMers.

Bringing up the authentication as though that was the relevant problem reveals a significant hole in your understanding of security.

I ran web servers for about 10 years snd am well aware if security concerns.

Anyway you don't have to use the interface if you do not wish to.

[pmetzger]

Modern coding techniques and frameworks are no longer subject to RCE's.

That's dangerously wrong. I'm not even going to argue any further.

[AaronNGray]

Modern coding techniques and frameworks are no longer subject to RCE's. That's dangerously wrong. I'm not even going to argue any further.

Most of the world are using these frameworks and its only zero days and poor security maintenance that are the issues.

[pmetzger]

Most of the world are using these frameworks and its only zero days and poor security maintenance that are the issues.

That's dangerously wrong. As I said, it's so completely wrong that there's no point in even having the discussion.

[Julien-Elie]

My first reasoning in this is the limitation of line length in readers.conf in relation to users and newsgroup names.

The current parser of readers.conf currently handles strings up to 8192 characters. Do you actually have newsgroup patterns longer than that?

We can easily increase the 8 KB limit if that is enough to solve your problem, though I am still unsure this is really needed:

• couldn't the perl_access or python_access parameters be used in your auth groups to generate a dynamic access group? (then the users parameter could be exactly the one of your authenticated user)
• couldn't the newsgroup pattern be somehow compressed, using wildcards? (or couldn't you use a python_dynamic hook)

See the parts about nnrpd hooks in the following documents about Perl and Python hooks.

I have been looking at putting a REST based API on INN so it can be maintained from a web based UI. Using OpenAPI and possibly in Python using Flask.

If anyone is interested in this direction we could start another repo to design the API ?

Adding this feature would be quite a bit of work and maintenance. I am unsure there are volunteers here to do that.

[AaronNGray]

On Sun, 4 Aug 2024, 13:15 Julien ÉLIE, @.***> wrote:

My first reasoning in this is the limitation of line length in readers.conf in relation to users and newsgroup names.

The current parser of readers.conf currently handles strings up to 8192 characters. Do you actually have newsgroup patterns longer than that?

It's potentially user lists as opposed to newsgroups.

Basically what I want is a user interface that shows either by users the newsgroups they have access to or by newsgroups which users have access. And yes the numbers for private groups will potentially be small. But I don't like arbitrary limits to one day bump into.

We can easily increase the 8 KB limit if that is enough to solve your problem, though I am still unsure this is really needed:

• couldn't the perl_access or python_access parameters be used in your auth groups to generate a dynamic access group? (then the users parameter could be exactly the one of your authenticated user)

Python possibly

• couldn't the newsgroup pattern be somehow compressed, using wildcards? (or couldn't you use a python_dynamic hook)

As I say it's user lists more than private newsgroup lists but could also include users with write access to public groups.

See the parts about nnrpd hooks in the following documents about Perl https://www.eyrie.org/~eagle/software/inn/docs/hook-perl.html and Python https://www.eyrie.org/~eagle/software/inn/docs/hook-python.html hooks.

I have been looking at putting a REST based API on INN so it can be maintained from a web based UI. Using OpenAPI and possibly in Python using Flask.

If anyone is interested in this direction we could start another repo to design the API ?

Adding this feature would be quite a bit of work and maintenance. I am unsure there are volunteers here to do that.

An alternative is to have the option to store the configuration in a SQL database like SQLite and then change that using a simple Python based OpenAPI defined REST API. I have yet to look at INN2's code based properly but did notice you have touched and reformmatted practically every source file. So yes this is a preliminary tentative requirement but it beats rewriting whole files every time you want to update a few bytes.

I will have a serious look at INN2's code base and if I feel I can code the SQLite modifications in a way they do not disturb the codebase unnecessary and can be on a build option as well as a run time flag then I may possibly proceed when I have time and the patch/pull request would obviously be up for scrutiny.

Hope I am not disturbing things too soon.

—

Reply to this email directly, view it on GitHub https://github.com/InterNetNews/inn/issues/307#issuecomment-2267521961, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAU4E6UFAGOBYUTVGRHDB3ZPYLNHAVCNFSM6AAAAABLTAWZ5KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRXGUZDCOJWGE . You are receiving this because you authored the thread.Message ID: @.***>

[Julien-Elie]

It's potentially user lists as opposed to newsgroups. Basically what I want is a user interface that shows either by users the newsgroups they have access to or by newsgroups which users have access. And yes the numbers for private groups will potentially be small.

Understood. I think you should definitely have a look at the python_access facility.

Basically, you could do something like that in readers.conf:

auth checkcredentials {
    auth: "ckpasswd -f /usr/local/news/db/passwords"
    python_access: "nnrpd_access"
    default: <public>
}

access public {
    users: "<public>"
    newsgroups: "comp.*"
}

See the nnrpd_access.py sample file to better understand how it works. In the access() function, just return a dictionary like {"read": "comp.*,private.*", "post": "comp.*,private.*"} where the newsgroups depend on the authenticated user. The username is set in the string attributes["user"].tobytes().decode(errors="backslashreplace") in Python 3 syntax.

No more issue of a limited length of 8 kB for the users parameter in readers.conf. You no longer need listing all the users.

Regarding the user interface, you may just list the users in /usr/local/news/db/passwords followed with the rule you defined in the _nnrpdaccess.py script (note that the .py extension is not present in the python_access parameter).

Moreover, I think you'll like the idea to just put all the information in an SQLite database: user, password, newsgroups for reading, newsgroups for posting, etc. and use an _nnrpdauth.py script to authenticate the users (sample here, you have to use the sqlite3 Python module to look for the user in the database) and _nnrpdaccess.py to grant the access. Then readers.conf only needs:

auth checkcredentials {
    python_auth: "nnrpd_auth"
    python_access: "nnrpd_access"
}

You can always return a successful auth, with user \<public> for instance when no users match, in which case the generated access group only allows public newsgroups.

I hope these suggestions have given you some ideas 😄 The SQLite database can be shared by both the _nnrpdauth.py and _nnrpdaccess.py scripts, and your web interface.

[AaronNGray]

@Julien-Elie Yes this does seem like a lot less work.