libc support for long long int and double in libc

[eriksl]

Hi

I am trying to get 64 bit and float working in libc (printf). It doesn't work at the moment, because apparently support is not compiled. This gives me the same functionality as ets(vsn)printf, so that doesn't help ;-)

I found these in the .config file, which suppose are the entries I'd need to change:

# CT_LIBC_NEWLIB_IO_C99FMT is not set
# CT_LIBC_NEWLIB_IO_LL is not set
# CT_LIBC_NEWLIB_IO_FLOAT is not set

Now this file is "git ignored" so very probably generated at some point. So if I change it here, my changes will get lost. Do you know where I can change this and have it working? Would you consider change this as well in (your) upstream version?

[eriksl]

Think I found it, this in Makefile right?

    # Append overrides
    cat ../crosstool-config-overrides >> .config
    # Build

[someburner]

Updated crosstool-ng to latest master with GCC 9.2 support. Had to change the config/sample config a bit to get things to work nicely. Look at the Makefile on my sdk2-lwip212-new-ctng branch for esp-open-sdk (new stuff under sdk: target).

Some notes:

• gcc 9.2 compile saves my project ~2K flash
• enabling newlib io options for me results in ~20K additional flash. I actually save nearly all of that by leaving the options disabled and using this nodemcu vsprintf (printf wraps vsprintf, see header here)

[eriksl]

Cool VERY COOL. I am definitely going to see what you did and apply it to my build environment.

:-)

[eriksl]

Yes long and long long work as expected.

I think the nodemcu vsprintf has limitations as to use it less code. For instance the version supplied with the NONOS-SDK and the default version of newlib don't handle floats or long long types, I can't use that.

A lot of DRAM is used by some "impure data" struct, which is used by newlib. I cannot really find out what it's used for and how to shrink it. It seems to have something to do with non-reentrant libc functions (like localtime()) and handling of errno. I don't need that. If I could get rid of that, I could save another 1k of DRAM.

Size of code in flash isn't that important to me, all devices have at least 2 x 1 Mbyte flash available.

[eriksl]

Done it now, thx!

I found it doesn't give me extra DRAM (which is what I am after), only less. BUT if I tweak some flags, I can get the DRAM usage indeed below the previous mark.

This may be interesting:

• I added "-Wl,--size-opt" to "CT_TARGET_LDFLAGS", which may save some bytes from newlib itself
• same for "CT_LIBC_NEWLIB_TARGET_CFLAGS" and also added "-ffunction-sections -fdata-sections" there
• enabled "CT_LIBC_NEWLIB_EXTRA_SECTIONS" (which probably does the same as "-ffunction-sections -fdata-sections"

I also tried enabling "LTO" linking in crosstoll, but I can't get that working, I always get tons of undefined symbols when linking my project, regardless if I enable LTO there or not.

And also I tried to remove all multithreading from newlib (which we're not using anyway), but then I end up with some undefined *_r symbols ("reentrant"), so apparantly the multithreading support overlaps with the reentrant support and we can't disable it.

Also I enabled a few stdio options in newlib, for formatting long longs, long doubles etc. and I removed support for some specific stdio optimisations which we won't use anyway (stdio output buffering, seek optimisation, etc.)

The last think I'd like to optimise someway, is the 1 k of "impure data" area claimed by newlib. It seems related to reentrant functions using __errno and other global variables. But I don't need it.

[eriksl]

You may find it interesting to set this as well:

CT_LIBC_NEWLIB_REENT_SMALL=y

It will large solve the issue I mentioned above, where 1 kbyte of "impure" data, for multithreading, was allocated (but never used). This option allocates the space dynamically. Saves me 800 bytes of precious DRAM.

[someburner]

Saves me 800 bytes of precious DRAM.

Nice! That is a worthwhile amount of ram. I'll see how I fare.

[eriksl]

Yeah, I'd say that (in contrary to I think most of the world ;-))

You can see I've programmed ATmel stuff for years, with 500 bytes - 2000 bytes RAM ;-)

Thanks for taking the effort to update to latest GCC and share it with us!

[someburner]

I tried enabling those options but my project doesn't seem to get any additional DRAM savings. Are you using newlib unmodified or have you made changes? If so can you share them?

Or are those savings only seen with the IO options enabled?

Edit: Actually I save exactly 160 bytes. Better than nothing but 800 would be nice..

[eriksl]

Yes I think I have some options active that you're not using. My full patch against the default config is here:

--- .config 2019-12-12 19:42:27.109713273 +0100
+++ .config 2019-12-12 20:03:08.165932076 +0100
@@ -44,18 +44,18 @@
 #
 # Paths
 #
-CT_LOCAL_TARBALLS_DIR="${HOME}/src"
+CT_LOCAL_TARBALLS_DIR="${CT_TOP_DIR}/sources"
 CT_SAVE_TARBALLS=y
 # CT_TARBALLS_BUILDROOT_LAYOUT is not set
 CT_WORK_DIR="${CT_TOP_DIR}/.build"
 CT_BUILD_TOP_DIR="${CT_WORK_DIR:-${CT_TOP_DIR}/.build}/${CT_HOST:+HOST-${CT_HOST}/}${CT_TARGET}"
-CT_PREFIX_DIR="${CT_PREFIX:-${HOME}/x-tools}/${CT_HOST:+HOST-${CT_HOST}/}${CT_TARGET}"
+CT_PREFIX_DIR="${CT_TOP_DIR}/xtensa-lx106-elf"
 CT_RM_RF_PREFIX_DIR=y
 CT_REMOVE_DOCS=y
-CT_INSTALL_LICENSES=y
+# CT_INSTALL_LICENSES is not set
 CT_PREFIX_DIR_RO=y
 CT_STRIP_HOST_TOOLCHAIN_EXECUTABLES=y
-# CT_STRIP_TARGET_TOOLCHAIN_EXECUTABLES is not set
+CT_STRIP_TARGET_TOOLCHAIN_EXECUTABLES=y

 #
 # Downloading
@@ -66,7 +66,7 @@
 # CT_FORBID_DOWNLOAD is not set
 # CT_FORCE_DOWNLOAD is not set
 CT_CONNECT_TIMEOUT=10
-CT_DOWNLOAD_WGET_OPTIONS="--passive-ftp --tries=3 -nc --progress=dot:binary"
+CT_DOWNLOAD_WGET_OPTIONS="--tries=3 -nc --progress=dot:binary"
 # CT_ONLY_DOWNLOAD is not set
 # CT_USE_MIRROR is not set
 CT_VERIFY_DOWNLOAD_DIGEST=y
@@ -164,7 +164,7 @@
 CT_DEMULTILIB=y
 CT_ARCH_SUPPORTS_BOTH_MMU=y
 CT_ARCH_DEFAULT_HAS_MMU=y
-CT_ARCH_USE_MMU=y
+# CT_ARCH_USE_MMU is not set
 CT_ARCH_SUPPORTS_FLAT_FORMAT=y
 CT_ARCH_SUPPORTS_EITHER_ENDIAN=y
 CT_ARCH_DEFAULT_LE=y
@@ -178,8 +178,8 @@
 #
 # Target optimisations
 #
-CT_TARGET_CFLAGS="-mlongcalls"
-CT_TARGET_LDFLAGS=""
+CT_TARGET_CFLAGS="-mlongcalls -mno-target-align -mno-serialize-volatile"
+CT_TARGET_LDFLAGS="-Wl,--size-opt"
 CT_TARGET_USE_OVERLAY=y
 CT_OVERLAY_NAME="lx106"
 CT_OVERLAY_LOCATION="${CT_TOP_DIR}/overlays"
@@ -192,7 +192,7 @@
 # General toolchain options
 #
 CT_SHOW_CT_VERSION=y
-CT_TOOLCHAIN_PKGVERSION=""
+CT_TOOLCHAIN_PKGVERSION="lx106"
 CT_TOOLCHAIN_BUGURL=""

 #
@@ -367,25 +367,26 @@
 CT_NEWLIB_later_than_2_0=y
 CT_NEWLIB_2_0_or_later=y
 CT_NEWLIB_CXA_ATEXIT=y
-CT_LIBC_NEWLIB_TARGET_CFLAGS="-DMALLOC_PROVIDED"
+CT_LIBC_NEWLIB_TARGET_CFLAGS="-DMALLOC_PROVIDED -ffunction-sections -fdata-sections -Wl,--size-opt"
 # CT_LIBC_NEWLIB_IO_C99FMT is not set
-# CT_LIBC_NEWLIB_IO_LL is not set
-# CT_LIBC_NEWLIB_IO_FLOAT is not set
+CT_LIBC_NEWLIB_IO_LL=y
+CT_LIBC_NEWLIB_IO_FLOAT=y
+CT_LIBC_NEWLIB_IO_LDBL=y
 # CT_LIBC_NEWLIB_IO_POS_ARGS is not set
-CT_LIBC_NEWLIB_FVWRITE_IN_STREAMIO=y
-CT_LIBC_NEWLIB_UNBUF_STREAM_OPT=y
-CT_LIBC_NEWLIB_FSEEK_OPTIMIZATION=y
+# CT_LIBC_NEWLIB_FVWRITE_IN_STREAMIO is not set
+# CT_LIBC_NEWLIB_UNBUF_STREAM_OPT is not set
+# CT_LIBC_NEWLIB_FSEEK_OPTIMIZATION is not set
 CT_LIBC_NEWLIB_DISABLE_SUPPLIED_SYSCALLS=y
 # CT_LIBC_NEWLIB_REGISTER_FINI is not set
-CT_LIBC_NEWLIB_ATEXIT_DYNAMIC_ALLOC=y
+# CT_LIBC_NEWLIB_ATEXIT_DYNAMIC_ALLOC is not set
 # CT_LIBC_NEWLIB_GLOBAL_ATEXIT is not set
-# CT_LIBC_NEWLIB_LITE_EXIT is not set
+CT_LIBC_NEWLIB_LITE_EXIT=y
-# CT_LIBC_NEWLIB_REENT_SMALL is not set
+CT_LIBC_NEWLIB_REENT_SMALL=y
 CT_LIBC_NEWLIB_MULTITHREAD=y
-# CT_LIBC_NEWLIB_EXTRA_SECTIONS is not set
-CT_LIBC_NEWLIB_WIDE_ORIENT=y
+CT_LIBC_NEWLIB_EXTRA_SECTIONS=y
+# CT_LIBC_NEWLIB_WIDE_ORIENT is not set
 CT_LIBC_NEWLIB_ENABLE_TARGET_OPTSPACE=y
 # CT_LIBC_NEWLIB_LTO is not set
 CT_LIBC_NEWLIB_EXTRA_CONFIG_ARRAY=""
 CT_ALL_LIBC_CHOICES="AVR_LIBC BIONIC GLIBC MINGW_W64 MOXIEBOX MUSL NEWLIB NONE UCLIBC"
 CT_LIBC_SUPPORT_THREADS_NONE=y
@@ -400,12 +401,12 @@
 # C compiler
 #
 CT_CC_CORE_PASS_2_NEEDED=y
-CT_CC_SUPPORT_CXX=y
-CT_CC_SUPPORT_FORTRAN=y
-CT_CC_SUPPORT_ADA=y
-CT_CC_SUPPORT_OBJC=y
-CT_CC_SUPPORT_OBJCXX=y
-CT_CC_SUPPORT_GOLANG=y
+#CT_CC_SUPPORT_CXX is not set
+#CT_CC_SUPPORT_FORTRAN is not set
+#CT_CC_SUPPORT_ADA is not set
+#CT_CC_SUPPORT_OBJC is not set
+#CT_CC_SUPPORT_OBJCXX is not set
+#CT_CC_SUPPORT_GOLANG is not set
 CT_CC_GCC=y
 CT_CC="gcc"
 CT_CC_CHOICE_KSYM="GCC"
@@ -459,12 +460,12 @@
 CT_CC_GCC_CORE_EXTRA_CONFIG_ARRAY=""
 CT_CC_GCC_EXTRA_CONFIG_ARRAY=""
 # CT_CC_GCC_SYSTEM_ZLIB is not set
-CT_CC_GCC_CONFIG_TLS=m
+# CT_CC_GCC_CONFIG_TLS is not set

 #
 # Optimisation features
 #
-CT_CC_GCC_USE_GRAPHITE=y
+# CT_CC_GCC_USE_GRAPHITE is not set
 CT_CC_GCC_USE_LTO=y

 #
@@ -479,15 +480,15 @@
 # Misc. obscure options.
 #
 # CT_CC_CXA_ATEXIT is not set
-# CT_CC_GCC_DISABLE_PCH is not set
-# CT_CC_GCC_LDBL_128 is not set
+CT_CC_GCC_DISABLE_PCH=y
+CT_CC_GCC_LDBL_128=y
 # CT_CC_GCC_BUILD_ID is not set
 CT_CC_GCC_LNK_HASH_STYLE_DEFAULT=y
 # CT_CC_GCC_LNK_HASH_STYLE_SYSV is not set
 # CT_CC_GCC_LNK_HASH_STYLE_GNU is not set
 # CT_CC_GCC_LNK_HASH_STYLE_BOTH is not set
 CT_CC_GCC_LNK_HASH_STYLE=""
-CT_CC_GCC_DEC_FLOAT_AUTO=y
+# CT_CC_GCC_DEC_FLOAT_AUTO is not set
 # CT_CC_GCC_DEC_FLOAT_BID is not set
 # CT_CC_GCC_DEC_FLOAT_DPD is not set
 # CT_CC_GCC_DEC_FLOATS_NO is not set
@@ -545,21 +546,21 @@
 CT_GMP_later_than_5_0_0=y
 CT_GMP_5_0_0_or_later=y
 CT_GMP_REQUIRE_5_0_0_or_later=y
-CT_COMP_LIBS_ISL=y
-CT_COMP_LIBS_ISL_PKG_KSYM="ISL"
-CT_ISL_DIR_NAME="isl"
-CT_ISL_PKG_NAME="isl"
-CT_ISL_SRC_RELEASE=y
+# CT_COMP_LIBS_ISL is not set
+# CT_COMP_LIBS_ISL_PKG_KSYM="ISL"
+# CT_ISL_DIR_NAME="isl"
+# CT_ISL_PKG_NAME="isl"
+# CT_ISL_SRC_RELEASE=y
 # CT_ISL_SRC_DEVEL is not set
 # CT_ISL_SRC_CUSTOM is not set
-CT_ISL_PATCH_GLOBAL=y
+# CT_ISL_PATCH_GLOBAL=y
 # CT_ISL_PATCH_BUNDLED is not set
 # CT_ISL_PATCH_LOCAL is not set
 # CT_ISL_PATCH_BUNDLED_LOCAL is not set
 # CT_ISL_PATCH_LOCAL_BUNDLED is not set
 # CT_ISL_PATCH_NONE is not set
-CT_ISL_PATCH_ORDER="global"
-CT_ISL_V_0_21=y
+# CT_ISL_PATCH_ORDER="global"
+# CT_ISL_V_0_21=y
 # CT_ISL_V_0_20 is not set
 # CT_ISL_V_0_19 is not set
 # CT_ISL_V_0_18 is not set
@@ -567,25 +568,25 @@
 # CT_ISL_V_0_16 is not set
 # CT_ISL_V_0_15 is not set
 # CT_ISL_NO_VERSIONS is not set
-CT_ISL_VERSION="0.21"
-CT_ISL_MIRRORS="http://isl.gforge.inria.fr"
-CT_ISL_ARCHIVE_FILENAME="@{pkg_name}-@{version}"
-CT_ISL_ARCHIVE_DIRNAME="@{pkg_name}-@{version}"
-CT_ISL_ARCHIVE_FORMATS=".tar.xz .tar.bz2 .tar.gz"
-CT_ISL_SIGNATURE_FORMAT=""
-CT_ISL_later_than_0_18=y
-CT_ISL_0_18_or_later=y
-CT_ISL_later_than_0_15=y
-CT_ISL_0_15_or_later=y
-CT_ISL_REQUIRE_0_15_or_later=y
-CT_ISL_later_than_0_14=y
-CT_ISL_0_14_or_later=y
-CT_ISL_REQUIRE_0_14_or_later=y
-CT_ISL_later_than_0_13=y
-CT_ISL_0_13_or_later=y
-CT_ISL_later_than_0_12=y
-CT_ISL_0_12_or_later=y
-CT_ISL_REQUIRE_0_12_or_later=y
+# CT_ISL_VERSION="0.21"
+# CT_ISL_MIRRORS="http://isl.gforge.inria.fr"
+# CT_ISL_ARCHIVE_FILENAME="@{pkg_name}-@{version}"
+# CT_ISL_ARCHIVE_DIRNAME="@{pkg_name}-@{version}"
+# CT_ISL_ARCHIVE_FORMATS=".tar.xz .tar.bz2 .tar.gz"
+# CT_ISL_SIGNATURE_FORMAT=""
+# CT_ISL_later_than_0_18=y
+# CT_ISL_0_18_or_later=y
+# CT_ISL_later_than_0_15=y
+# CT_ISL_0_15_or_later=y
+# CT_ISL_REQUIRE_0_15_or_later=y
+# CT_ISL_later_than_0_14=y
+# CT_ISL_0_14_or_later=y
+# CT_ISL_REQUIRE_0_14_or_later=y
+# CT_ISL_later_than_0_13=y
+# CT_ISL_0_13_or_later=y
+# CT_ISL_later_than_0_12=y
+# CT_ISL_0_12_or_later=y
+# CT_ISL_REQUIRE_0_12_or_later=y
 # CT_COMP_LIBS_LIBELF is not set
 # CT_COMP_LIBS_LIBICONV is not set
 CT_COMP_LIBS_MPC=y
@@ -673,12 +674,12 @@
 # CT_GETTEXT_NEEDED is not set
 CT_GMP_NEEDED=y
 CT_MPFR_NEEDED=y
-CT_ISL_NEEDED=y
+# CT_ISL_NEEDED is not set
 CT_MPC_NEEDED=y
 CT_ZLIB_NEEDED=y
 CT_GMP=y
 CT_MPFR=y
-CT_ISL=y
+# CT_ISL is not set
 CT_MPC=y
 CT_ZLIB=y

[someburner]

Thanks for that- now seeing 720 additional DRAM.

[eriksl]

Interesting where that come from! I hope I didn't disable any stuff you're actually using at some point :-O