Feature Request: More support for freestanding environments

[tigran2008]

On freestanding environments, only a minimal set of headers are available. Yet ETL makes use of C standard library headers (e.g., and , to name a few) that are limited to hosted environments yet are not very hard to implement manually.

I am working on a hobbyist operating system kernel project for learning purposes and even though I'm only now hearing of ETL, it seemed to me like there's huge potential of making use of it in my project because of the convenient data structures that it provides.

Proposed fix to this problem:

• Create bare minimum header-only partial implementations of those libc headers which would try to include the default C implementations if ETL_FREESTANDING_LIBC is not defined, and otherwise manually implement the functionality of these headers.

Example of <etl/libc_ctype.h>:

#ifndef ETL_LIBC_CTYPE_INCLUDED
#define ETL_LIBC_CTYPE_INCLUDED

#include "platform.h"

#ifndef ETL_FREESTANDING_LIBC
  #include <ctype.h>
#else

#define isalnum  isalnum
#define isalpha  isalpha
#define iscntrl  iscntrl
#define isdigit  isdigit
#define isgraph  isgraph
#define islower  islower
#define isprint  isprint
#define ispunct  ispunct
#define isspace  isspace
#define isupper  isupper
#define isxdigit isxdigit

#define tolower  tolower
#define toupper  toupper

extern "C" {

static inline int isalnum(int ch) {
    return (('0' <= (unsigned char)(ch) && (unsigned char)(ch) <= '9')
         || ('A' <= (unsigned char)(ch) && (unsigned char)(ch) <= 'Z')
         || ('a' <= (unsigned char)(ch) && (unsigned char)(ch) <= 'z'));
}

// [...implementation of other functionality]

} // extern "C"

#endif // ETL_FREESTANDING_LIBC

#endif

P.S.: Of course, I am not suggesting the integration of a full-fledged libc implementation into ETL, but rather a very minimal one to make things "just work" out of the box.

[jwellbelove]

That sounds like a useful addition to the ETL.

[tigran2008]

etl/platform.h could also include something like this to automatically define the macro:

#if (!__STDC_HOSTED__)
  #define ETL_FREESTANDING_LIBC
#endif

Edit: Actually, probably not. Because a freestanding codebase could still choose to implement those headers manually. I guess the best choice would be to make the user define it manually in "profile.h"

[jwellbelove]

Would stdint.h clone need to be provided by the ETL?

[tigran2008]

Not really, no. See these webpages:

https://wiki.osdev.org/Implications_of_writing_a_freestanding_C_project

https://en.cppreference.com/w/cpp/freestanding

[tigran2008]

I was also thinking.. maybe ETL could instead provide headers similar to (e.g. <etl/cstring.h>) instead of the headers that I originally proposed (e.g. <etl/libc_cstring.h>) which would also add these symbols to the etl namespace (e.g. etl::memcpy)?

[jwellbelove]

I haven't yet investigated to see how much of the C library the ETL uses, but I'm pretty sure it is quite small. (memcpy is the inly one I can think if at the moment, and I was thinking of moving away from using it as it is not constexpr) It could well be fairly easy to make the ETL freestanding by default.

[jwellbelove]

The only issue I can see is running the unit tests, as setting the freestanding flag would probably break the unit test compilation.

[tigran2008]

It is small, actually. Well, admittedly I only really tried <etl/string.h> and the error handler, but it required me to:

1. Create an empty because <etl/hash.h> includes it but doesn't make use of it,
2. Create a and define only this: https://en.cppreference.com/w/cpp/numeric/math/HUGE_VAL (By the way, if I recall correctly, according to the error I got, HUGE_VAL was used to define the signalling NaN, but it has a positive infinite value, not a NaN value. Is this the intended behavior?)
3. Create a and implement its functions
4. Create a and implement its functions (partially)

I can't check if it required anything more at the moment, as I'm not near my computer right now, but I could check later today.

[tigran2008]

By the way, what do you plan to use instead of C memory manipulation functions?

[jwellbelove]

I would try to create a highly optimised constexpr (C++14) memcpy style function in C++.

[jwellbelove]

Actually, the restriction of creating a constexpr memcpy is that you cannot use reiniterpret_cast, so you would be relying on the compiler to optimise.

[tigran2008]

Well, there's also std::bit_cast (https://en.cppreference.com/w/cpp/numeric/bit_cast), but that's available only since C++20.

P.S. Also I don't think it'd be possible to make use of it without the STL anyway because I don't think one could reimplement it unless some black magic is involved (I don't really understand how it works but I think it involves working with the compiler a lot which makes me wonder how it's implemented in the STL)

[jwellbelove]

Notice the "constexpr support needs compiler magic"

There are many things in C++ that make constexpr very difficult if you are not using the STL. For example, it's very difficult for me to create constexpr containers in the ETL as std::construct_at has been given special status so that it can use placement new to construct objects and still be constexpr. User defined functions are not given this freedom, so the only options are to only enable this when using the STL, or create functions that exactly match the implementation in the STL supplied with the compiler, so that it treats your function in the same way as it's own.

[jwellbelove]

I use the technique of 'clone the compiler's implementation' already for std::initializer_list. If you invoke an initializer_list in your code then the compiler will expect to see an implementation, of the correct format, in the std namespace. As you can imagine, this could be a very brittle feature.

[tigran2008]

As a beginner to C++, that sounds outright stupid to me. Why just not make those parts of the STL freestanding?

[jwellbelove]

that sounds outright stupid to me Couldn't agree more.

The C++ standards committee have made use of the STL mandatory if you want to be able to use certain aspects of modern C++, which severely hampers embedded programming, and third party libraries.

[VzdornovNA88]

I would try to create a highly optimised constexpr (C++14) memcpy style function in C++.

What do you mean? what is the criterion for the high efficiency of the function for constexpr?

we really still have to take care of the number of template type instantiations, and that's where my fantasy ends for the constexpr world. As for the versatility of the algorithm itself, it seems that the very idea that you need to have the same code to copy memory in run-time and compile-time contradicts what the world of constexpr is aimed at, I mean that today we cannot think in terms of some bytes in core constant expressionon - these should always be types, which, in principle, the memcpy algorithm does not take into .

On the other hand, there is std::copy, which is constexpr, but it's not about memcpy basically. Below is an example with part of _Uninitialized_copy_unchecked of an implementation from microsoft stl with the removal of all unnecessary. In fact, for constexpr, this is a regular loop on iterators. Here is_constant_evaluated from C++20 helps us , for C++14 you can invent or search for something like noexcept(function(...)) checks as described here https://stackoverflow.com/questions/46919582/is-it-possible-to-test-if-a-constexpr-function-is-evaluated-at-compile-time/46920091#46920091 or other tricks that seem quite fragile

Well, C++ wouldn't be C++ if we couldn't have the same type erasure on void* in the future - https://github.com/cplusplus/papers/issues/1431

Which nevertheless still prohibits freely manipulating bytes in consxtexpr since there are not actually such bytes)))

if (! is_constant_evaluated()) { return memmove(_First, _Last, _Dest); } for (; _First != _Last; ++_First) { ... }

although there is also an example https://open-std.org/Jtc1/sc22/wg21/docs/papers/2016/p0202r1.html which is about the need for std::memmove and std::memcpy to be marked as constexpr for implementations of most algorithms but 8 years later, we still don't have anything like this, is it possible to draw conclusions from this... There is a little interesting discussion about this https://gcc.gnu.org/bugzilla/show_bug.cgi?id=94082

as for the ongoing discussions about the similarity of the functionality of bit_cast and memcpy and the relationship of the former with the constexpr context , then it puts all the dots on https://www.open-std.org/jtc1/SC22/wg21/docs/papers/2017/p0476r2.html , where , in general, the marking of consctexpr for memcpy is abandoned in favor of internal compiler extensions as far as I understand.

I quote: "Furthermore, it is currently impossible to implement a constexpr bit-cast function, as memcpy itself isn’t constexpr. Marking the proposed function as constexpr doesn’t require or prevent memcpy from becoming constexpr, but requires compiler support. This leaves implementations free to use their own internal solution (e.g. LLVM has a bitcast opcode)."

so the bit_cast opcode is what the compiler can actually insert when the user uses std.::bit_cast in clang

This is not the truth, but just my reasoning on the subject of your comment, if you have something new about this, it would be very interesting to know.

[jwellbelove]

What do you mean? what is the criterion for the high efficiency of the function for constexpr? I meant a high efficiency runtime function that can also be constexpr. I later remembered that it would require reinterpret_cast or bit_cast which are not constexpr compatible.

The best bet may be to just use replace the requirement for constexpr memcpy with constexpr etl::copy and let the compiler optimise to the best of its ability.

[VzdornovNA88]

The best bet may be to just use replace the requirement for constexpr memcpy with constexpr etl::copy and let the compiler optimise to the best of its ability.

as a variant

there is still an ability to hope for the best until the last moment for etl::copy and move std::memcpy to the second implementation with a loop and call it through several important conditions such as: the declaration of std::memcpy exists and no is_constant_evaluated() context, in the end, even if there is no way to call the native implementation from the vendor , you can provide to call of a user provided implementation , and if there is none - well , we did everything we could - get a loop and hope for compiler optimization

of course this is for those types that can be copied by memcpy