Request for Comments: System-wide installation

The discussion about changing Eggdrop's installation comes up often (see also #474 and #1107, but mainly on IRC) about a system-wide Eggdrop installation. Various distributions came up with their own solutions because Eggdrop lacks that functionality and instead recommends compilation and installation per-user.

If we install Eggdrop system-wide, it should conform to https://www.gnu.org/software/make/manual/html_node/Directory-Variables.html (for configure/make variables) and https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html (for user directories).

What we have to sort out is regarding the installation is:

Global binary: eggdrop binary (eggdrop symlink and eggdrop-1.x.x)
Global modules: eggdrop modules (*.so)
Global static files: eggdrop documentation (doc/), eggdrop help (help/), eggdrop language files (language/)
Global/Local scripts: eggdrop scripts (core scripts and user scripts)
Local runtime: chanfile, userfile, pidfile, filesys/, DEBUG, ssl-keys, logs/, text/
Local configuration: eggdrop.conf

An example with the default values:

XDG_DATA_HOME = $HOME/.local/share
XDG_CONFIG_HOME = $HOME/.config
INSTALL PREFIX = /usr/local

/usr/local/
├── bin/
│   ├── eggdrop -> eggdrop-1.9.0
│   └── eggdrop-1.9.0
├── lib/eggdrop/1.9.0/
│      ├── server.so
│      ├── irc.so
│      └── channels.so
├── share/doc/eggdrop/1.9.0/
│      ├── ABOUT
│      ├── BOTNET
│      └── html/*
└── share/eggdrop/1.9.0/
       ├── language/*
       ├── help/*
       ├── config/
       │  ├── eggdrop.example.conf
       │  └── eggdrop-basic.example.conf
       └── scripts/
          ├── alltools.tcl
          └── compat.tcl

$HOME/
├── .config/eggdrop/
│   ├── mybot1.conf
│   └── mybot2.conf
└── .local/share/eggdrop/
    ├── mybot1.key
    ├── mybot1.chan
    ├── mybot1.cert
    ├── mybot1.user
    ├── mybot2.pid
    ├── mybot2.key
    ├── mybot2.chan
    ├── mybot2.notes
    ├── mybot2.cert
    ├── mybot2.user
    ├── logs/
    │   ├── mybot1.log
    │   └── mybot2.log
    └── scripts/
       ├── news.tcl
       └── botnet.tcl

The required effort would be:

Install eggdrop system-wide like that on "make install" (or separate target, check if root, etc.)
Provide an eggdrop parameter to (interactively?) create a configuration (see znc -c) under ~/.config/eggdrop
Adjust autobotchk and eggdrop core to deal with this structure
Adjust configuration to load core scripts from one location, user scripts from another
Runtime directory for this should be ~/.local/share/eggdrop then almost all relative paths still work and Tcl files created work still
Provide an eggdrop parameter to create certificates from a config file (reading destination)
Modify eggdrop's module loading to search paths ourselves instead of using LD_LIBRARY_PATH etc.

Edit: moved doc into /usr/local/share/doc because most distributions seem to do it that way, versioned the share directory, introduced 1.9.0 subdirectories

We could also still enable the customization proposed in #1107, partially, by making a subdirectory under .local/share/eggdrop foreach bot (mybot1/eggdrop.{key,cert,chan,user,...} mybot1/logs/eggdrop.log etc.), then the configuration file can mostly stay the same and you just adjust the "runtime-directory" in it.

Example:

$HOME/
├── .config/eggdrop/
│   ├── mybot1.conf
│   └── mybot2.conf
└── .local/share/eggdrop
    ├── mybot1/
    │  ├── eggdrop.key
    │  ├── eggdrop.chan
    │  ├── eggdrop.cert
    │  ├── eggdrop.user
    │  └── logs/
    │      └── eggdrop.log
    ├── mybot2/
    │  ├── eggdrop.key
    │  ├── eggdrop.chan
    │  ├── eggdrop.cert
    │  ├── eggdrop.user
    │  └── logs/
    │      └── eggdrop.log
    └── scripts/
       ├── news.tcl
       └── botnet.tcl

Great start- I think decomposing the problem into 'types' of files like you did at the beginning is smart. One thing that may be missing as a category are 'local' modules (a user wants to use gseen or something, for instance- not installed by Eggdrop, but still needed).

Possibly ignorant questions I have:

Do these paths require root, to install globally (and follow-up, what if I don't want to install globally, I'm just a user on the shell- how do we keep both functionalities)
How do I , as a non-root user, compile a module I downloaded that isn't globally installed-
How hard will this be to transition from an old install, to a new install? (rhetorical, but something to think about)

Comments:

Putting the config in .config is 'different' than how we've done it; not that I'm against it because it is becoming a bit of the norm, but it may hinder new users (we do get a lot of n00bs), or make transitioning harder
I'm still forming an opinion, but may want to investigate more what goes in /usr/local/share vs $HOME/.local/share/eggdrop. Maybe we can do something with sourcing, but maintaining some of these files in two different places may become a bit of a headache

One more format-specific thought- maybe its worth investigating a switvh to the eggdrop.conf.d naming syntax (renaming eggdrop.conf to that, and eggdrop-basic to eggdrop.conf; or whatever makes sense there

How do I , as a non-root user, compile a module I downloaded that isn't globally installed-

We need to also install eggdrops header files into include/ or include/eggdrop.

Several interesting question.

Root is required to install globally in the system.
A non-root installation is done in the $HOME without root rights. The eggdrop searches for files in the paths in order of priority / presence. For a MYSQL example: https://dev.mysql.com/doc/refman/8.0/en/option-files.html

In this principle, we can change "source scripts/gseen.tcl" to "loadscript gseen (.tcl)" The loadscript look in the scripts path $HOME (/scripts), if not find in the global scripts path. For a specific/custom path: "loadscript ./path/gseen(.tcl)"

If the eggdrop was installed at the system level and I want to compile a personal / additional module. I am using "eggdrop-buildmod src" to compile it and have the .SO file and put in my $HOME (module). Same principle as znc-buildmod for znc https://wiki.znc.in/Compiling_modules

I think that the newcomers will not be confused (n00b or not) of its changes of paths. It becomes a normal norm. Former users may be surprised at first, but a structural logic will ultimately win for everyone and gain in use. Implementing this in a MAJOR version seems obvious rather than minor.

Hoping to have brought constructive ideas, MalaGaM

We could also still enable the customization proposed in #1107, partially, by making a subdirectory under .local/share/eggdrop foreach bot (mybot1/eggdrop.{key,cert,chan,user,...} mybot1/logs/eggdrop.log etc.), then the configuration file can mostly stay the same and you just adjust the "runtime-directory" in it.

Example:
$HOME/
├── .config/eggdrop/
│   ├── mybot1.conf
│   └── mybot2.conf
└── .local/share/eggdrop
    ├── mybot1/
    │  ├── eggdrop.key
    │  ├── eggdrop.chan
    │  ├── eggdrop.cert
    │  ├── eggdrop.user
    │  └── logs/
    │      └── eggdrop.log
    ├── mybot2/
    │  ├── eggdrop.key
    │  ├── eggdrop.chan
    │  ├── eggdrop.cert
    │  ├── eggdrop.user
    │  └── logs/
    │      └── eggdrop.log
    └── scripts/
       ├── news.tcl
       └── botnet.tcl

i would prefer this, like 1 bot, all files. but this means that the scripts dir could also be separated by the way. like one bot useing a script, the other an other one, but what is you want same script, but with different "db" files? also i would put the user files in a separate folder, cuz with the *.bak files it can be too much and annoying.

here is an example how i organize my stuff:

$ ls -ltrR eggdrop/confs/

eggdrop/confs/:
total 448
drwx------ 4 <myuser> <myuser>  4096 Jun  1 16:02 scripts
drwx------ 2 <myuser> <myuser>  4096 Jun  8 01:48 tls
drwx------ 2 <myuser> <myuser>  4096 Jun  8 10:12 pids
-rw------- 1 <myuser> <myuser> 70433 Jun  8 10:44 egg_bot1.conf
-rw------- 1 <myuser> <myuser> 70929 Jun  8 10:45 egg_bot2.conf
-rw------- 1 <myuser> <myuser> 70299 Jun  8 10:45 egg_bot3.conf
-rw------- 1 <myuser> <myuser> 70458 Jun  8 10:45 egg_bot4.conf
-rw------- 1 <myuser> <myuser> 70405 Jun  8 10:46 egg_bot5.conf
-rw------- 1 <myuser> <myuser> 70191 Jun  8 10:46 egg_bot6.conf
drwx------ 2 <myuser> <myuser>  4096 Jun 29 16:00 botfiles

eggdrop/confs/scripts:
total 12
-rw------- 1 <myuser> <myuser> 4017 Jun  1 16:02 pubcommands.tcl
drwxrwxr-x 2 <myuser> <myuser> 4096 Jun  1 16:02 egg_bot2
drwxrwxr-x 2 <myuser> <myuser> 4096 Jun  1 16:02 egg_bot1

eggdrop/confs/scripts/egg_bot2:
total 8
-rw------- 1 <myuser> <myuser> 6146 Jun  1 16:02 undernet_auth.tcl

eggdrop/confs/scripts/egg_bot1:
total 16
-rw------- 1 <myuser> <myuser> 15853 Jun  1 16:02 quakenet_auth.tcl

eggdrop/confs/tls:
total 44
-rw------- 1 <myuser> <myuser> 1914 Jun  1 16:02 ssl.conf
-rw------- 1 <myuser> <myuser> 3272 Jun  1 16:02 egg_bot1.key
-rw-rw-r-- 1 <myuser> <myuser> 2126 Jun  1 16:02 egg_bot1.crt
-rw------- 1 <myuser> <myuser> 3268 Jun  1 16:02 egg_bot3.key
-rw-rw-r-- 1 <myuser> <myuser> 2126 Jun  1 16:02 egg_bot3.crt
-rw------- 1 <myuser> <myuser> 3268 Jun  1 16:02 egg_bot4.key
-rw-rw-r-- 1 <myuser> <myuser> 2122 Jun  1 16:02 egg_bot4.crt
-rw------- 1 <myuser> <myuser> 3272 Jun  1 16:02 egg_bot6.key
-rw-rw-r-- 1 <myuser> <myuser> 2122 Jun  1 16:02 egg_bot6.crt
-rw------- 1 <myuser> <myuser> 3268 Jun  8 00:48 egg_bot5.key
-rw-rw-r-- 1 <myuser> <myuser> 2065 Jun  8 00:49 egg_bot5.crt

eggdrop/confs/pids:
total 24
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  1 16:05 pid.egg_bot6
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  1 16:05 pid.egg_bot3
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  1 16:05 pid.egg_bot1
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  1 16:05 pid.egg_bot4
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  1 16:05 pid.egg_bot2
-rw-rw-r-- 1 <myuser> <myuser> 7 Jun  8 10:12 pid.egg_bot5

eggdrop/confs/botfiles:
total 136
-rw------- 1 <myuser> <myuser> 1078 Jun 29 00:00 egg_bot2.user~bak
-rw------- 1 <myuser> <myuser> 2653 Jun 29 00:00 egg_bot2.chan~bak
-rw------- 1 <myuser> <myuser>  642 Jun 29 00:00 egg_bot6.user~bak
-rw------- 1 <myuser> <myuser> 1397 Jun 29 00:00 egg_bot6.chan~bak
-rw------- 1 <myuser> <myuser> 1304 Jun 29 00:00 egg_bot1.user~bak
-rw------- 1 <myuser> <myuser> 8240 Jun 29 00:00 egg_bot1.chan~bak
-rw------- 1 <myuser> <myuser>  613 Jun 29 00:00 egg_bot5.user~bak
-rw------- 1 <myuser> <myuser> 3095 Jun 29 00:00 egg_bot5.chan~bak
-rw------- 1 <myuser> <myuser>  511 Jun 29 00:00 egg_bot3.user~bak
-rw------- 1 <myuser> <myuser> 2530 Jun 29 00:00 egg_bot3.chan~bak
-rw------- 1 <myuser> <myuser>  464 Jun 29 00:00 egg_bot4.user~bak
-rw------- 1 <myuser> <myuser> 1892 Jun 29 00:00 egg_bot4.chan~bak
-rw------- 1 <myuser> <myuser> 1078 Jun 29 16:00 egg_bot2.user
-rw------- 1 <myuser> <myuser>  108 Jun 29 16:00 egg_bot2.notes
-rw------- 1 <myuser> <myuser> 2653 Jun 29 16:00 egg_bot2.chan
-rw------- 1 <myuser> <myuser>  613 Jun 29 16:00 egg_bot5.user
-rw------- 1 <myuser> <myuser>  112 Jun 29 16:00 egg_bot5.notes
-rw------- 1 <myuser> <myuser> 3095 Jun 29 16:00 egg_bot5.chan
-rw------- 1 <myuser> <myuser> 1304 Jun 29 16:00 egg_bot1.user
-rw------- 1 <myuser> <myuser>  110 Jun 29 16:00 egg_bot1.notes
-rw------- 1 <myuser> <myuser> 8240 Jun 29 16:00 egg_bot1.chan
-rw------- 1 <myuser> <myuser>  642 Jun 29 16:00 egg_bot6.user
-rw------- 1 <myuser> <myuser>  112 Jun 29 16:00 egg_bot6.notes
-rw------- 1 <myuser> <myuser> 1397 Jun 29 16:00 egg_bot6.chan
-rw------- 1 <myuser> <myuser>  470 Jun 29 16:00 egg_bot4.user
-rw------- 1 <myuser> <myuser>  112 Jun 29 16:00 egg_bot4.notes
-rw------- 1 <myuser> <myuser> 1892 Jun 29 16:00 egg_bot4.chan
-rw------- 1 <myuser> <myuser>  511 Jun 29 16:00 egg_bot3.user
-rw------- 1 <myuser> <myuser> 2530 Jun 29 16:00 egg_bot3.chan
-rw------- 1 <myuser> <myuser>  112 Jun 29 16:00 egg_bot3.notes

bot1,2,3,4,5 obviously a fictional name :) having the main confs folder under the eggdrop folder is great, cuz it's not existing in the build, so it will never be overwritten by accident. this structure is rly make sense and easy to understand if you run more bots like me, cuz everything which matters is separated. under scripts, you can see script(s) which are loaded in all bots, that went to scripts source, but there i created again the botnames so separate each bots. (probably would be easier to have a bot folder and make the subfolder under it though.. but i would never mix or use a share folder for everything (like for/with the botfiles), cuz it's 4 files (+bak) for 1.

for the logs i use the default folder, having there also the name of the file (egg_), but i keep chanlogs in a diff place. (but logs wont be overwritten ever with an other update/upgrade either.

however, i do think that we are all diff and we are all thinking differently. :) so a decision has to be made on your side.

ed: just saw in a nother ticket (#1107) a folder named "data" which could be also a nice one instead of "confs". that's also make sense, maybe even better, cuz under confs i also have diff stuff, not just confs.. but meh. haven't thought about it that much before. :D

eggheads / eggdrop

Request for Comments: System-wide installation #1122