search

proftpd / proftpd

ProFTPD source code
http://www.proftpd.org
GNU General Public License v2.0
525 stars 199 forks source link

Unable to set session keys in mod_sftp #735

Closed mbStavola closed 6 years ago

mbStavola commented 6 years ago

What I Did

Trying to connect to my local test server via SFTP using both Filezilla and Cyberduck, but neither succeeded.

Checking the mod_sftp logs, it seems as though there is some issue with OpenSSL? Seems related to https://github.com/proftpd/proftpd/issues/547.

2018-08-02 01:28:15,208 mod_sftp/1.0.0[21]: sent server version 'SSH-2.0-mod_sftp'
2018-08-02 01:28:15,209 mod_sftp/1.0.0[21]: received client version 'SSH-2.0-FileZilla_3.33.0'
2018-08-02 01:28:15,209 mod_sftp/1.0.0[21]: handling connection from SSH2 client 'FileZilla_3.33.0'
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session key exchange: curve25519-sha256@libssh.org
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session server hostkey: ssh-rsa
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session client-to-server encryption: aes256-ctr
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session server-to-client encryption: aes256-ctr
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session client-to-server MAC: hmac-sha2-256
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session server-to-client MAC: hmac-sha2-256
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session client-to-server compression: none
2018-08-02 01:28:15,210 mod_sftp/1.0.0[21]:  + Session server-to-client compression: none
2018-08-02 01:28:15,244 mod_sftp/1.0.0[21]: error re-initializing aes256-ctr cipher for decryption:
  (1) error:0607B083:digital envelope routines:EVP_CipherInit_ex:no cipher set
2018-08-02 01:28:15,244 mod_sftp/1.0.0[21]: error setting session keys, disconnecting
2018-08-02 01:28:15,244 mod_sftp/1.0.0[21]: disconnecting 172.19.0.1 (Application error)

2018-08-02 01:37:36,043 mod_sftp/1.0.0[28]: sent server version 'SSH-2.0-mod_sftp'
2018-08-02 01:37:36,043 mod_sftp/1.0.0[28]: received client version 'SSH-2.0-Cyberduck/6.6.2.28219 (Mac OS X/10.13.1) (x86_64)'
2018-08-02 01:37:36,043 mod_sftp/1.0.0[28]: handling connection from SSH2 client 'Cyberduck/6.6.2.28219'
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session key exchange: curve25519-sha256@libssh.org
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session server hostkey: ssh-rsa
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session client-to-server encryption: aes128-cbc
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session server-to-client encryption: aes128-cbc
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session client-to-server MAC: hmac-sha1
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session server-to-client MAC: hmac-sha1
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session client-to-server compression: zlib@openssh.com
2018-08-02 01:37:36,046 mod_sftp/1.0.0[28]:  + Session server-to-client compression: zlib@openssh.com
2018-08-02 01:37:36,062 mod_sftp/1.0.0[28]: error re-initializing aes128-cbc cipher for decryption:
  (1) error:0607B083:digital envelope routines:EVP_CipherInit_ex:no cipher set
2018-08-02 01:37:36,063 mod_sftp/1.0.0[28]: error setting session keys, disconnecting
2018-08-02 01:37:36,063 mod_sftp/1.0.0[28]: disconnecting 172.19.0.1 (Application error)

What I Expected/Wanted

Logging in.

ProFTPD Version and Configuration

I made an image for Docker where I built ProFTPD and select modules from source. Here is what I got:

OpenSSL version: OpenSSL 1.1.0f (25 May 2017) ProFTPD version: 1.3.6 @ 5d19bfa Modules:

We can also see that mod_sftp is using OpenSSL 1.1.0f when we check the logs on startup:

2018-08-06 23:38:06,015 465161af99e7 proftpd[1]: mod_sftp/1.0.0: using OpenSSL 1.1.0f  25 May 2017
2018-08-06 23:38:06,015 465161af99e7 proftpd[1]: mod_sftp/1.0.0: using libsodium-1.0.11

ProFTPD config file:

#-----------------------------------------------------------------------
# Server Configuration: those parameters cannot be elsewhere
# http://www.proftpd.org/docs/howto/ConfigurationTricks.html
# http://www.proftpd.org/docs/howto/NAT.html
# http://www.proftpd.org/docs/howto/AWS.html
#-----------------------------------------------------------------------
ServerName                  "Test FTP Server"
ServerType                  standalone
UseIPv6                     off

# Bar use of SITE CHMOD by default
<Limit SITE_CHMOD>
        DenyAll
</Limit>

SystemLog                   none
LogFormat                   authentication "%{%F %T}t %P  from: %a to: %{protocol}:%H:%p  user: %U       msg: %S"
LogFormat                   transfer       "%{%F %T}t %P  from: %a to: %{protocol}:%H:%p  user: %U       file: %f        cmd: %m %J"

TLSProtocol                 SSLv3 TLSv1

<Global>
    #-----------------------------------------------------------------------
    # Generic Configuration
    #-----------------------------------------------------------------------
    DefaultRoot             ~
    CreateHome              on
    PassivePorts            %{env:PASSIVE_PORT_LOW} %{env:PASSIVE_PORT_HIGH}
    Umask                   022
    allowOverwrite          on
    User                    ftpuser
    Group                   ftpgroup
    ExtendedLog             /var/log/proftpd/auth.log AUTH,EXIT,SEC authentication
    ExtendedLog             /var/log/proftpd/xfer.log READ,WRITE transfer
    AuthOrder               mod_sql.c

    #-----------------------------------------------------------------------
    # Virtual Root Configuration
    # https://htmlpreview.github.io/?https://github.com/Castaglia/proftpd-mod_vroot/blob/master/mod_vroot.html
    #-----------------------------------------------------------------------
    VRootEngine off

    #-----------------------------------------------------------------------
    # Case-Sensitivity Configuration
    # http://www.castaglia.org/proftpd/modules/mod_case.html
    #-----------------------------------------------------------------------
    CaseEngine on
    CaseIgnore on

    #-----------------------------------------------------------------------
    # Unique ID Configuration
    # http://www.proftpd.org/docs/contrib/mod_unique_id.html
    #-----------------------------------------------------------------------
    UniqueIDEngine on

    #-----------------------------------------------------------------------
    # Ban Configuration
    # http://www.proftpd.org/docs/contrib/mod_ban.html#Usage
    #-----------------------------------------------------------------------
    BanEngine off

    #-----------------------------------------------------------------------
    # TLS Configuration
    # http://www.proftpd.org/docs/contrib/mod_tls.html
    #-----------------------------------------------------------------------
    TLSEngine                       off
    <IfDefine ENVIRONMENT="production">
        TLSRSACertificateFile           /usr/local/proftpd/etc/proftpd.cert.pem
        TLSRSACertificateKeyFile        /usr/local/proftpd/etc/proftpd.key.pem
        TLSLog                          none
        TLSVerifyClient                 off
        TLSRenegotiate                  none
        TLSRequired                     off
    </IfDefine>
    #-----------------------------------------------------------------------
    # SFTP Configuration
    # http://www.proftpd.org/docs/contrib/mod_sftp.html
    #-----------------------------------------------------------------------
    SFTPEngine off

    #-----------------------------------------------------------------------
    # MySQL Configuration
    # http://www.proftpd.org/docs/contrib/mod_sql.html
    #-----------------------------------------------------------------------
    SQLEngine               auth
    SQLAuthenticate         users
    SQLBackend              mysql
    SQLAuthTypes            plaintext # TODO: Change
    SQLConnectInfo          %{env:MYSQL_DATABASE}@%{env:MYSQL_HOST}:%{env:MYSQL_PORT} %{env:MYSQL_USER} %{env:MYSQL_PASSWORD}
    SQLUserInfo             users User Password Uid Gid Dir NULL
    SQLMinID                500
    SqlLogFile              /var/log/proftpd/sql.log

    SQLNamedQuery           get-user-authorized-keys FREEFORM "SELECT `key` FROM users WHERE User='%U'"

    SQLNamedQuery           get-host-authorized-keys FREEFORM "SELECT `key` FROM hosts WHERE hostname='${0}'"

    #-----------------------------------------------------------------------
    # SQL Password Configuration
    # http://www.proftpd.org/docs/contrib/mod_sql_passwd.html
    #-----------------------------------------------------------------------
    SQLPasswordEngine       off

    #-----------------------------------------------------------------------
    # AWS Configuration
    # https://htmlpreview.github.io/?https://github.com/Castaglia/proftpd-mod_aws/blob/master/mod_aws.html
    #-----------------------------------------------------------------------
    # AWSEngine off
</Global>

#-----------------------------------------------------------------------
# FTP Configuration
#-----------------------------------------------------------------------
<IfDefine ENVIRONMENT="development">
    <VirtualHost 0.0.0.0>
        Port                                    21
        TLSEngine                               off
    </VirtualHost>
</IfDefine>

#-----------------------------------------------------------------------
# FTPS Configuration
#-----------------------------------------------------------------------
<IfDefine ENVIRONMENT="production">
    <VirtualHost 0.0.0.0>
        Port                                    21
        TLSEngine                               on
        TLSOptions                              UseImplicitSSL
    </VirtualHost>
</IfDefine>

#-----------------------------------------------------------------------
# SFTP Configuration
# http://www.proftpd.org/docs/contrib/mod_sftp_sql.html#Usage
#-----------------------------------------------------------------------
<VirtualHost 0.0.0.0>
    Port                            22
    SFTPEngine                      on
    SFTPLog                         /var/log/proftpd/sftp.log
    SFTPAuthorizedUserKeys          sql:/get-user-authorized-keys
    SFTPHostKey                     /etc/proftpd/sftp_host_rsa
    SFTPCompression                 delayed
    MaxLoginAttempts                6
</VirtualHost>

<IfModule mod_facts.c>
    FactsAdvertise off
</IfModule>

(Apologies for the Frankenstein config file, I'm still finding my away around ProFTPD!)

mbStavola commented 6 years ago

Additionally, here are some relevant lines from the Dockerfile:

FROM debian:stretch

RUN apt-get -y update && \
    apt-get -y install \
        acl \
        git \
        curl \
        build-essential \
        libsodium-dev \
        libcurl4-openssl-dev \
        libxml2-dev \
        pkg-config \
        adduser \
        debianutils \
        libacl1 \
        libcap2 \
        libmemcached11 \
        libmemcachedutil2 \
        libncurses5 \
        libpam-runtime \
        libpam0g \
        libpcre3 \
        libtinfo5 \
        libwrap0 \
        lsb-base \
        netbase \
        sed \
        ucf \
        zlib1g \
        openssl \
        openbsd-inetd \
        libssl-dev \
        default-libmysqlclient-dev

RUN cd proftpd && ./configure --enable-openssl --enable-ctrls \
    --with-includes=/usr/include/libxml2 \
    --with-libraries=/usr/lib \
    --with-modules=mod_vroot:mod_case:mod_unique_id:mod_ban:mod_tls:mod_sftp:mod_sql:mod_sql_mysql:mod_sql_passwd:mod_sftp_sql:mod_aws \
    && make \
    && make install

If there's anything else I could provide, I'd be more than happy to do so.

Castaglia commented 6 years ago

Hmm. There were some mod_sftp changes needed for OpenSSL-1.1.x; you might see if using the latest proftpd source code, from the master branch, behaves any better.

mbStavola commented 6 years ago

Checked out master, same issue unfortunately.

Not super familiar with the codebase (or really C either), but I tried patching cipher.c with the assumption that this might be the problem (even though it should be fixed...?). Gets a little further than before after adjusting L475 and L664, but then breaks down because the client payload is too large?

mbStavola commented 6 years ago

Additionally, it seems that on master environment variable substitutions in config files don't work as they do in 1.3.6? I didn't see anything about this in RELEASE_NOTES, should I report this as a separate issue?

Castaglia commented 6 years ago

For the environment variable issue, please do open a separate ticket/issue for that.

Castaglia commented 6 years ago

Looks like OpenSSL Issue #4347 is indeed the root cause here; I checked the OpenSSL 1.1.0f source and verified that it does not contain the fix; OpenSSL 1.1.0g does have the fix.

Castaglia commented 6 years ago

I think that the fix for ProFTPD here is to check for OpenSSL support for EVP_CipherInit_ex, and use that instead of EVP_CipherInit, if present; that is the preferred direction of OpenSSL, and per the bug report, EVP_CipherInit_ex is not subject to this buggy behavior in OpenSSL 1.1.0f.

mbStavola commented 6 years ago

Awesome! Will verify shortly, going to fill out that separate issue for you regarding the environment variables.

mbStavola commented 6 years ago

It works! Thanks for all the help.

I assume this won't be backported to 1.3.6 since it's branched off of master?

Castaglia commented 6 years ago

Thanks for confirming that it works!

I was planning on backporting that change to the 1.3.6 branch, yes. I usually develop my PRs on master, than backport/cherry-pick the changes to the release branches as needed.

Castaglia commented 6 years ago

Merged to master, and backported to the 1.3.6 branch.