Build problems on Linux

[Ben-Voris]

Problems building on Linux

cmake is not listed as a prerequisite in BUILDING.md, but make -f Makefile.mingw BITS=64 fails without it. cmake is included in neither build-essential (12.4ubuntu1) nor mingw-w64 (5.0.3-1). Is the use of cmake intentional? If it is, I'm happy to create a PR to change BUILDING.md to state that it is necessary.

For reference, here is the output of the first build:

: make -f Makefile.mingw BITS=64
------------------------------------------------------------ openssl
Downloaded: openssl-1.0.2r.tar.gz [checksum OK]
    perl Configure --openssldir=/home/bvoris/NppFTP/x64/3rdparty --cross-compile-prefix=x86_64-w64-mingw32- no-shared no-asm mingw64
    make depend
    make
    make install_sw
------------------------------------------------------------ zlib
Downloaded: zlib-1.2.11.tar.gz [checksum OK]
    make -f win32/Makefile.gcc PREFIX=x86_64-w64-mingw32-
    cp zlib.h zconf.h /home/bvoris/NppFTP/x64/3rdparty/include
    cp libz.a /home/bvoris/NppFTP/x64/3rdparty/lib
------------------------------------------------------------ libssh
Downloaded: libssh-0.8.7.tar.xz [checksum OK]
    cmake -DCMAKE_SYSTEM_NAME=Windows                         -DCMAKE_C_COMPILER=x86_64-w64-mingw32-gcc -DCMAKE_CXX_COMPILER=x86_64-w64-mingw32-g++                         -DOPENSSL_INCLUDE_DIRS=/home/bvoris/NppFTP/x64/3rdparty/include -DOPENSSL_CRYPTO_LIBRARY=/home/bvoris/NppFTP/x64/3rdparty/lib/libcrypto.a                         -DWITH_STATIC_LIB=ON -DWITH_EXAMPLES=OFF -DWITH_SERVER=OFF -DCMAKE_INSTALL_PREFIX=/home/bvoris/NppFTP/x64/3rdparty -DCMAKE_PREFIX_PATH=/home/bvoris/NppFTP/x64/3rdparty /home/bvoris/NppFTP/x64/obj/libssh

/bin/sh: 1: cmake: not found

command failed: exit code 127
Makefile.mingw:77: recipe for target 'x64/3rdparty' failed
make: *** [x64/3rdparty] Error 1

After installing cmake, I get compiler notes/errors using this version of mingw.

: x86_64-w64-mingw32-gcc -v
Using built-in specs.
COLLECT_GCC=x86_64-w64-mingw32-gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-w64-mingw32/7.3-win32/lto-wrapper
Target: x86_64-w64-mingw32
Configured with: ../../src/configure --build=x86_64-linux-gnu --prefix=/usr --includedir='/usr/include' --mandir='/usr/share/man' --infodir='/usr/share/info' --sysconfdir=/etc --localstatedir=/var --disable-silent-rules --libdir='/usr/lib/x86_64-linux-gnu' --libexecdir='/usr/lib/x86_64-linux-gnu' --disable-maintainer-mode --disable-dependency-tracking --prefix=/usr --enable-shared --enable-static --disable-multilib --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --libdir=/usr/lib --enable-libstdcxx-time=yes --with-tune=generic --with-headers=/usr/x86_64-w64-mingw32/include --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libgomp --enable-languages=c,c++,fortran,objc,obj-c++,ada --enable-lto --with-plugin-ld --enable-threads=win32 --program-suffix=-win32 --program-prefix=x86_64-w64-mingw32- --target=x86_64-w64-mingw32 --with-as=/usr/bin/x86_64-w64-mingw32-as --with-ld=/usr/bin/x86_64-w64-mingw32-ld --enable-libatomic --enable-libstdcxx-filesystem-ts=yes
Thread model: win32
gcc version 7.3-win32 20180312 (GCC) 

Here is the output of the first make after installing cmake:

: make -f Makefile.mingw BITS=64
CXX tinyxml/src/tinystr.cpp
CXX tinyxml/src/tinyxmlerror.cpp
CXX tinyxml/src/tinyxml.cpp
CXX tinyxml/src/tinyxmlparser.cpp
CXX UTCP/src/ftp_c.cpp
UTCP/src/ftp_c.cpp: In member function ‘virtual int CUT_FTPClient::ReceiveFile(CUT_DataSource&, LPCSTR)’:
UTCP/src/ftp_c.cpp:332:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
     if ( m_nFirewallMode )
     ^~
UTCP/src/ftp_c.cpp:335:2: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
  m_wsData.SSLSetReuseSession(SSLGetCurrentSession());
  ^~~~~~~~
UTCP/src/ftp_c.cpp: In member function ‘virtual int CUT_FTPClient::ResumeReceiveFile(CUT_DataSource&, LPCSTR)’:
UTCP/src/ftp_c.cpp:467:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
     if ( m_nFirewallMode )
     ^~
UTCP/src/ftp_c.cpp:470:2: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
  m_wsData.SSLSetReuseSession(SSLGetCurrentSession());
  ^~~~~~~~
UTCP/src/ftp_c.cpp:517:77: error: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Werror=format=]
     _snprintf(m_szBuf,sizeof(m_szBuf)-1,"REST %d\r\n",dest.Seek (0,SEEK_END));
                                                       ~~~~~~~~~~~~~~~~~~~~~~^
UTCP/src/ftp_c.cpp: In member function ‘virtual int CUT_FTPClient::ResumeReceiveFilePASV(CUT_DataSource&, LPCSTR)’:
UTCP/src/ftp_c.cpp:698:77: error: format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘long int’ [-Werror=format=]
     _snprintf(m_szBuf,sizeof(m_szBuf)-1,"REST %d\r\n",dest.Seek (0,SEEK_END));
                                                       ~~~~~~~~~~~~~~~~~~~~~~^
UTCP/src/ftp_c.cpp: In member function ‘virtual int CUT_FTPClient::SendFile(CUT_DataSource&, LPCSTR)’:
UTCP/src/ftp_c.cpp:920:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
     if (m_nFirewallMode)
     ^~
UTCP/src/ftp_c.cpp:923:2: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
  m_wsData.SSLSetReuseSession(SSLGetCurrentSession());
  ^~~~~~~~
UTCP/src/ftp_c.cpp: In member function ‘virtual int CUT_FTPClient::GetDirInfo(LPCSTR)’:
UTCP/src/ftp_c.cpp:1960:5: error: this ‘if’ clause does not guard... [-Werror=misleading-indentation]
     if (m_nFirewallMode)
     ^~
UTCP/src/ftp_c.cpp:1963:2: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the ‘if’
  m_wsData.SSLSetReuseSession(SSLGetCurrentSession());
  ^~~~~~~~
cc1plus: all warnings being treated as errors
Makefile.mingw:41: recipe for target 'x64/obj/ftp_c.o' failed
make: *** [x64/obj/ftp_c.o] Error 1

These are easy enough to fix. Since the same errors occur in other files, I thought it best to check if you want them fixed or if the Linux build isn't meant to be used.

The "format ‘%d’ expects argument of type ‘int’, but argument 4 has type ‘long int’" errors are easy enough to fix, like so:

@@ -695,7 +697,7 @@ int CUT_FTPClient::ResumeReceiveFilePASV(CUT_DataSource & dest, LPCSTR sourceFil

     // if the file exist then we will send the REST command  with the size of the file we have
     // otherwise we just call retrieve as we do normally
-    _snprintf(m_szBuf,sizeof(m_szBuf)-1,"REST %d\r\n",dest.Seek (0,SEEK_END));
+    _snprintf(m_szBuf,sizeof(m_szBuf)-1,"REST %ld\r\n",dest.Seek (0,SEEK_END));^M
     dest.Close ();
     Send(m_szBuf);
     //check for a return of 100 or 200 code

The "misleadingly indented" note/error perplexes me because I don't see the misleading indent, but then I've never worked on a code base that does not always use braces to enclose blocks. (Particularly after Heartbleed.) Regardless, it appears that NppFTP prefers not enclosing single-line expressions with braces. One fix would be something like this:

@@ -1957,8 +1960,9 @@ int CUT_FTPClient::GetDirInfo(LPCSTR path){
     int     rt,loop,len;
     char    addr[32];

-    if (m_nFirewallMode)
+    if (m_nFirewallMode) {^M
         return GetDirInfoPASV(path);
+       }^M

        m_wsData.SSLSetReuseSession(SSLGetCurrentSession());

This code uses both tabs and spaces for whitespace, which can trigger this warning, so an alternative is to change all tabs to spaces or all spaces to tabs and then to enforce that policy. (I happen to be fan of Eclipse's code formatter, but any automated pre-checkin tool that enforces formatting rules is a goodness.) The downside is that the diff will show many changed lines. Of course, other than vc churn, there is no downside to doing both.

At minimum the same problem occurs in UTCP/src/ut_clnt.cpp and UTCP/src/utstrlst.cpp.

[chcg]

See:

Cross-compiling from Linux has been tested with Debian/Ubuntu (see Travis CI builds ) and Fedora. You will need to install the following packages:

Common: python3 cmake zip

Libssh uses it. I would say perl (necessary for openssl) is missing.

See Travis build output:

https://travis-ci.org/ashkulz/NppFTP/builds/503291260?utm_source=github_status&utm_medium=notification

at least there are no compiler warnings visible, but that one is a quite old version of the compiler.

[ashkulz]

@Ben-Voris: we've recently updated to Xenial, and there don't seem to be any warnings. Can you confirm if there's anything still pending?

[Ben-Voris]

I don't know what Xenial is. Does using it change the compiler version? If so, what version does it use?

As you noted, the later compilers produce more warnings. x86_64-w64-mingw32-g++ (GCC) 7.3-win32 20180312 does not like the mix of leading tabs and spaces.

[chcg]

@Ben-Voris Xenial is the name of http://releases.ubuntu.com/16.04/, so mingw for this version is:

g++-mingw-w64-x86-64 is already the newest version (5.3.1-8ubuntu3+17).

See https://travis-ci.org/ashkulz/NppFTP/jobs/558004748 + update to libssh 0.9.0.

I created also test build with not yet released bionic image of travis(http://releases.ubuntu.com/18.04/), see:

https://travis-ci.org/ashkulz/NppFTP/jobs/559124701

showing build errors.

[ashkulz]

I think @chcg fixed issues with gcc 7.4, I think only buster has a newer release (8.3.0). I think we should switch to that for CI if we want the very latest warnings instead of bionic.

Closing this issue for now, thanks for reporting @Ben-Voris!

[Ben-Voris]

The warnings caused by mixed tabs and spaces no longer occur.

But, running on x86_64-w64-mingw32-g++ (GCC) 7.3-win32 20180312 on Ubuntu 18.04.2, I see this block a few times:

/home/bvoris/NppFTP2/x64/obj/libssh/src/known_hosts.c: In function ‘ssh_is_server_known’:
/home/bvoris/NppFTP2/x64/obj/libssh/src/known_hosts.c:408:9: warning: ‘ssh_write_knownhost’ is deprecated [-Wdeprecated-declarations]
         ssh_write_knownhost(session);
         ^~~~~~~~~~~~~~~~~~~
In file included from /home/bvoris/NppFTP2/x64/obj/libssh/include/libssh/priv.h:171:0,
                 from /home/bvoris/NppFTP2/x64/obj/libssh/src/known_hosts.c:31:
/home/bvoris/NppFTP2/x64/obj/libssh/include/libssh/libssh.h:558:31: note: declared here
 SSH_DEPRECATED LIBSSH_API int ssh_write_knownhost(ssh_session session);

The note seems to be caused by this comment:

/**
 * @deprecated Please use ssh_session_export_known_hosts_entry()
 * @brief This function is deprecated.
 */

Then make fails with this:

[ 99%] Building C object src/CMakeFiles/ssh_shared.dir/external/curve25519_ref.c.obj
[100%] Linking C shared library libssh.dll
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x58): undefined reference to `pthread_mutex_init'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0xb4): undefined reference to `pthread_mutex_destroy'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0xf5): undefined reference to `pthread_mutex_lock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x116): undefined reference to `pthread_mutex_unlock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x129): undefined reference to `pthread_self'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x155): undefined reference to `pthread_mutex_lock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x195): undefined reference to `pthread_mutex_unlock'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ssh_shared.dir/build.make:1800: recipe for target 'src/libssh.dll' failed
[ 99%] Building C object src/CMakeFiles/ssh_shared.dir/external/curve25519_ref.c.obj
[100%] Linking C shared library libssh.dll
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x58): undefined reference to `pthread_mutex_init'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0xb4): undefined reference to `pthread_mutex_destroy'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0xf5): undefined reference to `pthread_mutex_lock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x116): undefined reference to `pthread_mutex_unlock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x129): undefined reference to `pthread_self'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x155): undefined reference to `pthread_mutex_lock'
CMakeFiles/ssh_shared.dir/objects.a(pthread.c.obj):pthread.c:(.text+0x195): undefined reference to `pthread_mutex_unlock'
collect2: error: ld returned 1 exit status
src/CMakeFiles/ssh_shared.dir/build.make:1800: recipe for target 'src/libssh.dll' failed

Simply running make again gives a different error (which, imho, means there is a problem in it). Here's the second failure.

: make -f Makefile.mingw BITS=64
CXX tinyxml/src/tinystr.cpp
CXX tinyxml/src/tinyxmlerror.cpp
CXX tinyxml/src/tinyxml.cpp
CXX tinyxml/src/tinyxmlparser.cpp
CXX UTCP/src/ftp_c.cpp
CXX UTCP/src/UTDataSource.cpp
CXX UTCP/src/ut_clnt.cpp
CXX UTCP/src/UT_StrOp.cpp
CXX UTCP/src/UT_Queue.cpp
CXX UTCP/src/utstrlst.cpp
CXX UTCP/src/uh_ctrl.cpp
CXX UTCP/src/utfile.cpp
CXX src/Monitor.cpp
CXX src/FTPCache.cpp
CXX src/DragDropSupport.cpp
CXX src/FTPClientWrapperSSL.cpp
In file included from src/FTPClientWrapperSSL.cpp:20:0:
src/FTPClientWrapper.h:24:10: fatal error: libssh/libssh.h: No such file or directory
 #include <libssh/libssh.h>
          ^~~~~~~~~~~~~~~~~
compilation terminated.
Makefile.mingw:44: recipe for target 'x64/obj/FTPClientWrapperSSL.o' failed
make: *** [x64/obj/FTPClientWrapperSSL.o] Error 1

Full make output

[ashkulz]

@Ben-Voris: you need to use the posix variant instead of win32, see b0b958a97c075a7bc070bc1ad3c611a681346cf0. Agreed about the second-time compilation, build_3rdparty.py should be run unconditionally.

[chcg]

@Ben-Voris Regarding "ssh_write_knownhost", this is causing compiler warnings within libssh build and could just be fixed there at the libssh code (created https://bugs.libssh.org/T164 and https://bugs.libssh.org/T165 therefore)

[Ben-Voris]

From my working copy:

: git log | grep b0b958a commit b0b958a97c075a7bc070bc1ad3c611a681346cf0

And, .travis.yml already contains these lines:

  1 language: cpp
  2 dist:     xenial
  3 sudo:     false
...
 20 - sudo update-alternatives --set x86_64-w64-mingw32-gcc /usr/bin/x86_64-w64-mingw32-gcc-posix
 21 - sudo update-alternatives --set x86_64-w64-mingw32-g++ /usr/bin/x86_64-w64-mingw32-g++-posix
 22 - sudo update-alternatives --set i686-w64-mingw32-g++   /usr/bin/i686-w64-mingw32-g++-posix
 23 - sudo update-alternatives --set i686-w64-mingw32-gcc   /usr/bin/i686-w64-mingw32-gcc-posix

This seems to be the contents of https://github.com/ashkulz/NppFTP/commit/b0b958a97c075a7bc070bc1ad3c611a681346cf0

[ashkulz]

@Ben-Voris, I meant -- you need to run the commands at your end as well. I wasn't able to get the pthread detection code to turn off in 0.9.0 so used this -- still need to check that the DLL is usable even if compiled in this mode.

[chcg]

@ashkulz Tested https://github.com/ashkulz/NppFTP/releases/tag/v0.29.0. It is working fine for me.

[ashkulz]

@Ben-Voris: I've fixed the rebuild issue with e8a5afd, which makes it a phony target (i.e. always run). I've also created #261 to use a even newer version of MinGW-w64 (with Debian 10, released recently).

If we do get issues in the build, let's continue the discussion on that PR. Thanks for your contributions!