Docs should reflect that pkgs may set additional env vars

[mithro]

Actual Behavior

conda build sets -std=c++17 in the CXXFLAGS build environment.

See the following output;

INFO:conda_build.metadata:Attempting to finalize metadata for gcc-lm32-elf-nostdc
Solving environment: ...working... done
Solving environment: ...working... done
Solving environment: ...working... done
BUILD START: ['gcc-lm32-elf-nostdc-5.4.0-0000.20180824231751.unknown.tar.bz2']
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac

The following NEW packages will be INSTALLED:

    libgcc-ng:    8.2.0-hdf63c60_1
    libstdcxx-ng: 8.2.0-hdf63c60_1

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Solving environment: ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env

The following NEW packages will be INSTALLED:

    binutils-lm32-elf:      2.31.1-h5cbe550_0 local
    binutils_impl_linux-64: 2.31.1-h6176602_1      
    binutils_linux-64:      2.31.1-h6176602_3      
    cloog:                  0.18.1-0               
    gcc_impl_linux-64:      7.3.0-habb00fd_1       
    gcc_linux-64:           7.3.0-h553295d_3       
    gmp:                    6.1.2-h6c8ec71_1       
    gxx_impl_linux-64:      7.3.0-hdf63c60_1       
    gxx_linux-64:           7.3.0-h553295d_3       
    isl:                    0.15-0                 
    libgcc-ng:              8.2.0-hdf63c60_1       
    libstdcxx-ng:           8.2.0-hdf63c60_1       
    mpc:                    1.1.0-h10f8cd9_1       
    mpfr:                   4.0.1-hdf1c602_3       

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
source tree in: /home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/work
INFO: activate-binutils_linux-64.sh made the following environmental changes:
+ADDR2LINE=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-addr2line
+AR=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar
+AS=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-as
+CXXFILT=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++filt
+ELFEDIT=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-elfedit
+GPROF=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-gprof
+HOST=x86_64-conda_cos6-linux-gnu
+LD_GOLD=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld.gold
+LD=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld
+NM=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-nm
+OBJCOPY=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-objcopy
+OBJDUMP=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-objdump
+RANLIB=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib
+READELF=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-readelf
+SIZE=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-size
+STRINGS=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-strings
+STRIP=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-strip
INFO: activate-gcc_linux-64.sh made the following environmental changes:
+CC=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc
+CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+_CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_cos6_linux_gnu
+CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2
+CPP=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-cpp
+DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og
+GCC_AR=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ar
+GCC=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc
+GCC_NM=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-nm
+GCC_RANLIB=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ranlib
+LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,-rpath,/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib -L/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/lib
INFO: activate-gxx_linux-64.sh made the following environmental changes:
+CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+CXX=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++
+DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
+GXX=/home/tansell/conda/conda-bld/gcc-lm32-elf-nostdc_1535152735807/_build_env/bin/x86_64-conda_cos6-linux-gnu-g++

Expected Behavior

conda doesn't force a C++ standard. Newer C++ standards are not 100% backwards compatible with older C++ standards which is why C++ compilers let you specify the C++ standard.

Steps to Reproduce

conda build a package using {{ compiler('cxx') }} with source code that is not C++17 compatible.

Output of conda info

     active environment : None
       user config file : /home/tansell/.condarc
 populated config files : 
          conda version : 4.5.10
    conda-build version : 3.13.0
         python version : 3.6.5.final.0
       base environment : /home/tansell/conda  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/tansell/conda/pkgs
                          /home/tansell/.conda/pkgs
       envs directories : /home/tansell/conda/envs
                          /home/tansell/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.10 requests/2.18.4 CPython/3.6.5 Linux/4.16.0-3rodete1-amd64 debian/rodete glibc/2.24
                UID:GID : 82487:89939
             netrc file : None
           offline mode : False

[mingwandroid]

conda doesn't force a C++ standard. Newer C++ standards are not 100% backwards compatible with older C++ standards which is why C++ compilers let you specify the C++ standard.

Our activation scripts are designed for building packages for AD and we want to use the most modern C++ standard whenever possible so we have decided to take a stance here. You cannot mix C++98 packages with C++17 packages due to the ABI differences so we explicitly do not want backwards compatibility here since it's not possible. Also we want packages that implement 'naive' auto-detection of C++ language features (#if __cplusplus >= 201103L) to compile that codepath

If you are not able to update the package to C++17 support then you should filter that out from CXXFLAGS. When I have to do this I do it in build.sh as:

if [[ ${target_platform} =~ .*linux.* ]]; then
    CXXFLAGS="${CXXFLAGS//-std=c++17/}"
fi

or:

re='(.*[[:space:]])\-std\=[^[:space:]]*(.*)'
if [[ "${CXXFLAGS}" =~ $re ]]; then
    export CXXFLAGS="${BASH_REMATCH[1]}${BASH_REMATCH[2]}"
fi

Another alternative that I do not recommend is to depend on gxx-impl_linux-64 and gxx-impl_linux32 instead but there are numerous downsides to that (lack of run_exports probably).

We will not be changing the CXXFLAGS our compilers set though so I am closing this.

[mingwandroid]

Also, nowhere in your output do I see any error.

[mithro]

Forgive me, I'm no expert here but it was my understanding the C++ ABI is not necessarily tied to the C++ standard being used? Does C++17 and the packages standard requires a specific ABI?

Looking at https://gcc.gnu.org/onlinedocs/libstdc++/manual/abi.html it seems what you actually want is -fabi-version?

(On the actual issues causing problems was running into with -std=c++17 was around bools and ++ operator.)

[mingwandroid]

No it is. I'd you are trying to limk to stuff compiled with an old pre-C++11 ABI and mix that with one using the new ABI (SSO change) you're in for a bad time. If the old module loads first it'll load the system libstdc++ instead of ours then the module that uses C++11 ABI will fall to find it's symbols. We decided to make the jump to the new ABI in July to September last year.

But regardless you get the new ABI with our compilers and the default std for GCC >6 is c++11 nowadays I believe.

Building everything with the latest standard available is something we will strive to continue doing.

[mithro]

From what I can understand, you can compile code with -std=c++98 and -fabi-version=0 -fabi-compat-version=8 and it will link fine against C++ code compile with -std=c++17 and -fabi-version=0 -fabi-compat-version=8?

It might be good to document the workaround for removing the -std=c++17 flag?

[mingwandroid]

The worry is if someone uses an old system compiler (where those flags do not even exist, nor any concept of the 'new C++ ABI') and will use the system libstdc++ and tries to mix it with stuff compiled with our compilers as I stated above. Using our compilers you can reduce or remove flags as appropriate.

A PR would be welcome but I don't really know to which repository it'd be applied (the compilers and compiler activation scripts are just packages like any other, they're not a feature of conda or conda-build as such) also I'd consider if pretty obvious that this is what needs to be done; I even print out the flag changes made so a package builder can easily see what we've set.

[mingwandroid]

https://gcc.gnu.org/onlinedocs/gcc-7.3.0/gcc/C_002b_002b-Dialect-Options.html#C_002b_002b-Dialect-Options

To sum up:

We target the cutting edge here, we stay current with the latest compiler features so will not contemplate reducing the default or messing with -fabi-version=0 -fabi-compat-version=8 which are not the latest versions.

For your own package builds feel free to play around but we are not going to risk compatibility by playing with these flags in our compiler activation packages.

[mithro]

Two things;

The C++ ABI version the compiler uses != C++ standard -- these are separate concepts (and settings).

If your goal is to have a specific ABI version then you should be explicitly setting the -fabi-version=n and -fabi-compat-version=n or -Wabi flags. By not providing the ABI version flags, you are using a "floating" ABI. IE You will always get the latest ABI. From the page you linked;

The default is version 0. Version 0 refers to the version conforming most closely to the C++ ABI specification. Therefore, the ABI obtained using version 0 will change in different versions of G++ as ABI bugs are fixed.

I can't find any documentation which says that -std=c++17 implies any given C++ ABI (but some C++17 features need at least a certain ABI version). As you are only setting -std=c++17, then you are not getting the static, well known C++ ABI it sounds like you are after.

---

For the documentation changes, I would recommend something like https://github.com/conda/conda-docs/commit/779ad11463f8b1f85fffd965e0842b3433cfa767#diff-2165fd095fe8f92aea74fe3ad04e1659R199

I agree it's a little complicated. I was looking at the Environment variables set during the build process section of the docs here -> https://conda.io/docs/user-guide/tasks/build-packages/environment-variables.html#environment-variables-set-during-the-build-process and was a bit tripped up by the lack of pointers to the fact that the compiler packages "extend" the build environment further.

---

[mingwandroid]

We explicitly want the latest ABI.

GCC attempted to implement -fabi-compat-version during the GCC 6 series but according to the clang developers it never worked right. Again, we are not interested in playing with this stuff or mixing any system GCC with our GCC.

We build hundreds of packages (actually thousands if you consider the different python and R variants) with these settings and the advised way of dropping back the --std=c++?? setting. I don't see any value in continuing this discussion.

[msarahan]

I'm going to reopen this, but strictly as a docs issue. Conda-build does not set these variables - the compiler activation scripts do. The docs should reflect that "in addition to these variables, packages installed in the host environment may also set environment variables. For example, the default compilers used by Anaconda, which are what you get when you use "{{ compiler() }}" expressions, set several compiler variables. If you need to change or unset those variables, you need to do so in your build script."

[mithro]

Sorry about going on about the ABI stuff - just trying to relay a more polite version of the long rant I got from my GCC developer friends when asked about the ABI topic.

Did the documentation change make any sense? Maybe a link to the Anaconda Compiler section from the environment would help? I was thinking of trying to update the environment page (https://conda.io/docs/user-guide/tasks/build-packages/environment-variables.html#) to more explicitly call out the conda-build environment around the activate.

[mingwandroid]

That's fine, and I don't claim to be an expert here, but I am happy with the job GCC has made of backwards compat. since GCC 6, but I do not want to risk GCC introducing functions that convert from the old ABI to the new one as I expect that will have performance issues and possibly tricky linking issues, but more importantly, AD doesn't provide packages built with the old ABI (except the free channel which contains legacy stuff built on a mix of CentOS5 and 6 using system compilers) and I do not want it to either. We made a huge effort to cross that line last year.

Adding the env var change information to that page would be good yes! I am in two minds as to whether or not to mention that this stuff only works with our compilers. The reason for that is that I believe using our compilers is definitely a better idea than using devtoolset (which statically links parts of the C++ standard library into every thing that uses C++) or system compilers on CentOS6 (which are then not able to support modern language features).

[mingwandroid]

Sorry about going on about the ABI stuff

Your questions are certainly going to be helpful for others who will have the same queries so no need to apologise. I will defend the decisions we made though as I am convinced they are right (at least from the perspective of building out AD).

[mingwandroid]

Since you also hack on crosstool-ng (which our compilers are based upon) I hope you may be able to help out on that if you find any issues and have time to look into them. I do a few icky things to fix issues in ctng and the various components. Currently we apply 31 patches on top of the upstream project (though some of the biggest ones were added for CentOS5 and those are not needed anymore).

[mithro]

As I just mentioned in https://github.com/conda/conda-docs/issues/607#issuecomment-416002384, I'm hoping to port our cross compilers to be based on your packages which used crosstool-ng. (I'm also hoping that it will make it easier for us to provide Mac + Windows packages too.)

Will definitely try and help here! My time availability is sadly very bursty which can make it hard to finish of things like this. :-/

[github-actions[bot]]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   • What OS and version you reproduced the issue on
   • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!