SSL cleanups (with a few BSD odds & ends)

[dougnazar]

Various patches cleaning up the SSL handling:

• Free SSL structure on failed handshake (memory leak).
• Ensure we can compile without any errors or deprecations.
• Generate DH params natively on 3.x if needed (auto recommended)
• Fix several warnings about using non-length versions of strcat, strcpy, sprintf, etc.
• Move duplicated SSL code from nrpe & check_nrpe to separate ssl.c file

This has been compile tested on the following configurations:

• Gentoo Linux (system)
  • openssl-3.0.8
• Gentoo Linux (local installation)
  • openssl-1.0.0t
  • openssl-1.0.1u
  • openssl-1.0.2u
  • openssl-1.1.0l
  • openssl-1.1.1s
  • openssl-3.0.8
  • openssl-3.1.0
• FreeBSD 13.2
  • libressl-3.6.2
  • openssl-1.1.1t
  • openssl30-3.0.8
  • openssl31-3.1.0
• OpenBSD 7.3
  • LibreSSL 3.7.2
  • openssl-1.1.1t
  • openssl-3.0.8
  • openssl-3.1.0

It's been running in production (along with my other patches) on one of my Gentoo boxes. The others have been compile (and minimally run) tested only.

[tsadpbb]

Bike shedding a little bit, but could you add the ssl.o and the utils.o to the gitignore? Also, I noticed ssl.o doesn't get removed from make clean.

Edit: Could you also add generate_dh_params to the gitignore?

[tsadpbb]

With the Makefile command

check_nrpe: 
        $(srcdir)/check_nrpe.c utils.o $(SRC_INCLUDE)/utils.h $(CFG_INCLUDE)/common.h $(CFG_INCLUDE)/config.h $(SSL_OBJS)
    $(CC) $(CFLAGS) -o $@ $(srcdir)/check_nrpe.c utils.o $(SSL_OBJS) $(LDFLAGS) $(SOCKETLIBS) $(SNPRINTF_O) $(OTHERLIBS)

Do you think you would ever need to specify the directory that utils.o is in? Something like this

check_nrpe: $(srcdir)/check_nrpe.c $(srcdir)/utils.o $(SRC_INCLUDE)/utils.h $(CFG_INCLUDE)/common.h $(CFG_INCLUDE)/config.h $(srcdir)/$(SSL_OBJS)
    $(CC) $(CFLAGS) -o $@ $(srcdir)/check_nrpe.c $(srcdir)/utils.o $(srcdir)/$(SSL_OBJS) $(LDFLAGS) $(SOCKETLIBS) $(SNPRINTF_O) $(OTHERLIBS)

This applies to the nrpe command as well

[tsadpbb]

Should the auto_dh be enabled by default or should it be disabled by default to conform with backwards compatibility?

[dougnazar]

Bike shedding a little bit, but could you add the ssl.o and the utils.o to the gitignore? Also, I noticed ssl.o doesn't get removed from make clean.

Edit: Could you also add generate_dh_params to the gitignore?

I did most of my testing in separate build directories so I didn't notice. I had just assumed all *.o files were ignored. I'll add a wild card and generate_dh_params.

I missed the ssl.o on clean. Easy enough to fix.

Do you think you would ever need to specify the directory that utils.o is in? Something like this

No. utils.o is in the build (current) directory, we only need to refer back to the source directory for *.c, *.h or any other shipped file.

Should the auto_dh be enabled by default or should it be disabled by default to conform with backwards compatibility?

It's a bit of an effort to remember, but I don't think I changed backwards compatibility here. It defaulted to always generating a DH key, I just switched to auto keying on setups that supported it. It should support all the same ciphers suites and connections to older versions as before.

I believe (sorry, it's been awhile) I tested various settings of ssl_use_adh as well as a few older versions of nrpe to check the legacy cases, but my testing definitely wasn't exhaustive.