2022/tuple-iteration-basics/

[utterances-bot]

C++ Templates: How to Iterate through std::tuple: the Basics - C++ Stories

If you have a standard container, it’s easy to use a range-based for loop and iterate over its elements at runtime. How about std::tuple? In this case, we cannot use a regular loop as it doesn’t “understand” tuple’s compile-time list of arguments. That’s why in this article, I’ll show you a few techniques you can use to run through all tuple’s entries.

https://www.cppstories.com/2022/tuple-iteration-basics/

[PaulTopping]

That it is so much trouble to do this doesn't make C++20 look good. I'm really concerned about the complexity of C++ now. I'm thinking we need to design a new language from first principles that has the same goals as C++ and the requirement that all good C++ code be converted into it.

[wimalopaan]

Typically i use an immediately invoded closure generated by a lambda-expression together with an index_sequence object as argument. Then you can use a fold expression to iterate.

[fenbf]

thanks @wimalopaan - do you have a working example? happy to see the code :) What's your common use case for such a tuple iteration?

[wimalopaan]

Example - https://godbolt.org/z/vaqnr6hrn

#include <tuple>
#include <algorithm>
#include <type_traits>
#include <utility>
#include <iostream>

void f(const auto& e) {
    std::cout << e << '\n';
}
int main() {
    std::tuple t{1, 2.0, "abc"};

    [&]<auto... I>(std::index_sequence<I...>) {
        (f(std::get<I>(t)),...);
    }(std::make_index_sequence<std::tuple_size_v<decltype(t)>>{});

    return std::tuple_size_v<decltype(t)>;
}

[fenbf]

Thanks @wimalopaan ! that's super short and works nicely! I'll mention that in the next part of my article.

Here's the adapted version, with separators (I had to pass ID to the function) - https://godbolt.org/z/fWEnd7o4c

void printElem(const auto& e, size_t id) {
    if (id > 0) std::cout << ", ";
    std::cout << e;
}
int main() {
    std::tuple tp{1, 2.0, "abc"};

    [&tp]<auto... I>(std::index_sequence<I...>) {
        (printElem(std::get<I>(tp), I),...);
    }(std::make_index_sequence<std::tuple_size_v<decltype(tp)>>{});
}

[wimalopaan]

With some boilerplate it's even more readable:

#include <tuple>
#include <algorithm>
#include <type_traits>
#include <utility>
#include <iostream>

namespace etl {
    template<typename... E>
    consteval auto size(const std::tuple<E...>& t) {
        return sizeof...(E);
    }
    template<typename... E>
    constexpr auto index_sequence(const std::tuple<E...>&) {
        return std::make_index_sequence<sizeof...(E)>{};
    }
}

void f(const auto& e) {
    std::cout << e << '\n';
}
int main() {
    std::tuple t{1, 2.0, "abc"};

    [&]<auto... I>(std::index_sequence<I...>) {
        (f(std::get<I>(t)),...);
    }(etl::index_sequence(t));

    return etl::size(t);
}

It's a pity that _stdlibc++does not include an template-overload ofstd::sizeforstd::tuple<>`.

It is still hard to read, but if you get used to this pattern using IIFE (immediate invoked function expressions) it becomes convenient.

[PiotrNycz]

I know, it will be in next part:

#include <tuple>
#include <iostream>

int main() {
    std::tuple t{1, 2.0, "abc"};
    std::apply([](auto const& ...e) {
        ((std::cout << e <<'\n'), ...);
    }, t);

    return std::tuple_size_v<decltype(t)>;
}

[wimalopaan]

Yes, std::appy is the correct algo for this! Thanks for the hint! I just wanted to point out, how this could be implemented. But yes: we all should algorithms, and avoid explicit loops!

[fenbf]

Thanks @PiotrNycz ! looks nice... but how about printing with a separator like ", " nicely? :)

[PiotrNycz]

We can also transform tuple<T...> into array<variant<T*...>> to get "iterable view": Better would be to use std::variant<T&...> - but std::variant does not like references.

template <typename ...T>
auto for_tuple(std::tuple<T...> const& t) {
    return std::apply([](auto const& ...e) {
        return std::array<std::variant<const T*...>, sizeof...(T)>{ &e... };
    }, t);
}
int main() {
    std::tuple t{1, 2.0, "abc"};
    for (auto const& e: for_tuple(t))
    {
        std::visit([](const auto* e) { std::cout << *e << '\n'; }, e);
    }
} 

I was also thinking about coroutines, but I doubt it is possible to use coroutine lambda in std functions like apply:

template <typename ...T>
generator<std::variant<const T*...>>  
auto for_tuple(std::tuple<T...> const& t) {
    std::apply([](auto const& ...e) -> generator<variant<T const*>> {
        ((co_yield &e), ...);
    }, t);
}

int main() {
    std::tuple t{1, 2.0, "abc"};
    for (auto const& e: for_tuple(t))
    {
        std::visit([](const auto* e) { std::cout << *e << '\n'; }, e);
    }
} 

[cooky922]

@PiotrNycz you could wrap it in std::ref or std::cref. There may also be danger in std::variant types when some alternatives have exactly the same type (ex: std::variant<const int*, const int*>).

[fenbf]

for (auto const& e: for_tuple(t)) is super cool, but you pay the price for the creation of std::variant and that would use max space(all types) times the num of elements. But maybe compiler could optimize it... at least for constexpr case (std::variant will be constexpr soon in C++23)...

[PiotrNycz]

@fenbf - I am using pointers (or maybe std::ref/std::cref as @cooky922 suggested) -- so price for single element is comparable to pointer size (mostly 8 bytes nowadays) plus size of variant index. So the total memory price is about (std::tuple_size_v<t> * (sizeof(std::size_t)) + sizeof(void*)) and since no dynamic memory is needed, then it is possible to optimize it out completely by compiler.

Anyway this memory overhead is only related to number of elements in tuple, not to specific types in tuple. For coroutine way (which is hard to implement) - this could be only this variant of pointers size plus coroutine overhead (and that could be quite big)

And for commas between:

std::apply([](const auto& ...e) {
    auto print = [needComma=false](const auto& elem) mutable {
         if (needComma) std::cout << ',';
         else needComma = true;
         std::cout << elem;
   };
   ((print(e), ...));
}, t);

[PiotrNycz]

@cooky922 - yes you are right when tuple has duplicates -- we should use some meta-programming like boost::mp11::mp_unique to get unique list of types: std::variant<boost::mp11::mp_unique<boost::mp11::mp_list<const T*...>>> or std::reference_type<const T> in place of pointer

[pblxptr]

I was using exactly the same technique in order to implement a very simple enum mapper that maps one enum into another and vice-versa. https://godbolt.org/z/scMq83aW6

[wimalopaan]

Additional side not: if one uses a fold-expression, this never forms a loop in assembler. The runtime for-each-statement is unrolled only up to 18 elements (gcc), for for this is a assembler loop. On µC this might be of interest. The general rule for C++ and µC is: never do with runtime-statemenst that can be done in a compile-time version. That is the benefit of the std::apply or the raw version.

[PiotrNycz]

Just for fun -- the working version with coroutine: https://godbolt.org/z/9Yc3nb1Y8 Main part is to return coroutine from inside std::apply -- it looks it works:

template <typename ...T>
tuple_for<std::tuple<T...>> 
 for_tuple(std::tuple<T...> const& t) {
    return std::apply([](auto const& ...e) -> tuple_for<std::tuple<T...>> {
        ((co_yield e), ...);
    }, t);
}
int main() {
    std::tuple t{1, 2.0, "abc"};
    for (auto const& e: for_tuple(t))
    {
        std::visit([](const auto* e) { std::cout << *e << '\n'; }, e);
    }
}

This tuple_for is "classic" coro generator which you can find in any post about C++20 coroutines, where value_type is variant on pointers to tuple values

[reedhedges]

Thanks, good example of applying variadic template arguments etc.... I'm pretty new to (modern) template programming, can you explain the use of decltype and decay at some point? Thanks

[wimalopaan]

Maybe a little bit simpler:

#include <tuple>
#include <array>
#include <variant>
#include <algorithm>
#include <functional>
#include <type_traits>
#include <utility>
#include <iostream>

namespace etl {
    template<typename... E>
    consteval auto size(const std::tuple<E...>& t) {
        return sizeof...(E);
    }
    template<typename... E>
    constexpr auto index_sequence(const std::tuple<E...>&) {
        return std::make_index_sequence<sizeof...(E)>{};
    }
    template<typename... E>
    auto tuple_for(const std::tuple<E...>& t) {
        return [&]<auto... I>(std::index_sequence<I...>) {
            using elem_t = std::variant<std::function<const E*()>...>;
            std::array<elem_t, sizeof...(E)> a{[&](){return &std::get<I>(t);}...};
            return a;
        }(etl::index_sequence(t));
    }
}

template<bool Comma = true>
void f(const auto& e) {
    if constexpr(Comma) {
        std::cout << ", ";
    }
    std::cout << e;
}
int main() {
    std::tuple t{1, 2.0, "abc"};

//    [&]<auto... I>(std::index_sequence<I...>) {
//        (f<(I != 0)>(std::get<I>(t)),...);
//    }(etl::index_sequence(t));

    for(const auto& p: etl::tuple_for(t)) {
        std::visit([](const auto& ep){
            std::cout << *ep() << '\n';
        }, p);
    }

    return etl::size(t);
}

[PiotrNycz]

Yet another version:

• with get_nth(tuple, index) returning variant of pointers to value_types of tuple -- we can iterate over index, or like for (auto&& elem : iota(0, tuple_size_v()) | tranform( get_nth. )) ... // << pseudo-code

template <typename T>
struct get_variant_value_type_of_tuple;
template <typename ...T>
struct get_variant_value_type_of_tuple<std::tuple<T...>&>{
    using type = std::variant<T*...>;
};
template <typename ...T>
struct get_variant_value_type_of_tuple<std::tuple<T...> const &>{
    using type = std::variant<const T*...>;
};
template <typename T>
using variant_value_type_of_tuple = typename get_variant_value_type_of_tuple<T>::type;

template <std::size_t I = 0u, typename T>
auto get_nth(T&& t, std::size_t idx)
  -> variant_value_type_of_tuple<T>
{
    constexpr auto size = std::tuple_size_v<std::decay_t<T>>;
    if constexpr (I < size) {
        if (idx == I) return &std::get<I>(std::forward<T>(t));
        // use "binary search" to reduce recursive depth to log N
        constexpr auto middle = I + (size+1u-I)/2u;
        if (idx <= middle) return get_nth<I + 1u>(std::forward<T>(t), idx);
        else return get_nth<middle + 1u>(std::forward<T>(t), idx);
    }
    else
         throw std::out_of_range("Tuple index out of range: " + std::to_string(idx) + " >= " + std::to_string(size));
}
int main() {
    std::tuple t{1, 2.0, "abc", 200u};
    for (std::size_t i = 0u; i < std::tuple_size_v<decltype(t)>; ++i)
        std::visit([](const auto* e) { std::cout << *e << '\n'; }, 
                   get_nth(t, i));
} 

[wimalopaan]

Yes, this results in a run-time search for the element. So the simple for-loop becomes O(N^2). This I tried to avoid by encapsulating the pointer into a closure.

My first posted version (or std::apply) results in the shortest machine code (therefor preferable in µC contexts). My closure version gets much shorter code (maybe faster) then the coro version.

The runtime linear search of pointers I tried to avoid because of the aboves reason (runtime complexity).

[fenbf]

Thanks for all super cool solutions! Lot's of cool ideas to understand and review :)

@reedhedges - I've removed std::decay... and simplified some examples with tuple_size_v, and made a comment on decltype.

[PiotrNycz]

@wimalopaan I think the std::apply is correct and preferable solution.

The last one I provided has actually n*log n complexity - for typical tuples, their size is less than 20 -- this log n is close to 1.
The version with coroutine+std::apply might be ok in future, when compilers will be mature enough to optimize such code.

The version with transforming tuple to array of variants of pointers to tuple values - is, IMO, the easier tor compiler to optimize now and allow to use raw iteration.

All versions with std::function - it really depends if small-size-object optimization might be applied to it. If not - dynamic memory allocation and virtual function dispatching that are used by std::function with bigger capture are costly and hard to optimize by compilers. Actually I do not see any benefit of functor that will returns pointer when needed comparing to just returning pointer to value. This "when needed" happens at once.

[2kaud]

Why do the C++ standards committee make things so complicated? Why not something 'simple' like:

const std::tuple tp {42, 10.5, "hello"};

for ( typename tn val : tp)
    switch (tn) {
        case int:
            // Process val as int
           break;

        case double:
            // Process val as double
            break;

        case char*:
            // Process val as char*
            break;

       default:
           std:;cout << "Unknown type\n";
           break;
    }

and something similar for std::variant, std::any etc.

[wimalopaan]

That would be indeed a proposal, perhaps with a little modificytion:

const std::tuple tp {42, 10.5, "hello"};

for (const auto val : tp) {
   if constexpr(std::is_same_v<decltype(val), int>) {
      // ...
   }
}

This would be less intrusive to the standard, and it would only extend the use of the auto-type-deduction into the body of the loop. which effectively could be translated as a generic lambda. Want's to write a proposal?

[2kaud]

Part of my idea above was that switch() could work on a type. So perhaps: switch constexpr(decltype(val)) { case int: // ... }

??? If someone is prepared to provide a proposal that would be great - but I don't want any involvement.

Also, if you know the number of elements in the tuple don't overlook structured bindings:

const std::tuple tp {10, 20, 3.14, 42, "hello"};
const auto& [a1, a2, a3, a4, a5] {tp};
std::cout << a1 << ' ' << a2 << ' ' << a3 << ' ' << a4 << ' ' << a5 << '\n';

[PiotrNycz]

there was proposal to do something like that:

const auto& [...a] = t;

see http://open-std.org/JTC1/SC22/WG21/docs/papers/2018/p1061r0.html. Unfortunately - proposal was rejected AFAIK.

Frankly, I never got to the point where std::apply wouldn't be enough for generic tuple. But this proposal with "structure binding pack" would be great for iterating over members of struct. Maybe reflection helps here -- but I haven't time to read its current status/shape.

For those who want to play with struct like with tuple -- there is BOOST_FUSION_ADAPT_STRUCT.

[2kaud]

One of my 'gripes' with current C++ is that much of what you may want to do (eg printTuple() ) can be done - but not simply. As noted in this article, some things require a lot of 'scaffolding' in order to achieve. IMO much 'simplification' should be introduced.

Re the original question, I would want code something like:

#include <tuple>
#include <iostream>

void printTuple(const auto& tp) {
    for (const auto& e : tp)
        std::cout << e << " ";

    std::cout << '\n';
}

int main() {
    const std::tuple tp {10, 20, 3.14, 42, "hello"};

     printTuple(tp);
}

But I guess this is too easy for 'the committee'

[PiotrNycz]

@2kaud well, this is not complicated task for current C++:

int main() {
    const std::tuple tp {10, 20, 3.14, 42, "hello"};
    std::apply([](const auto& ...e) { ((std::cout << e << " "), ...); }, tp);
}

You have to find something that is not "easy" to do with std::apply and wanted by many, to make C++ std committee to add something to language that is already too big to learn it in one month.

[2kaud]

and put the std::apply line into a function with a passed param - without getting tied up with complicated templates? I confess I'm been retired for a couple of years and that I was never a template expert and didn't get fully into variadic templates and fold expressions... It just seems to me - and did for a while before I retired - that C++ is getting more and more complex to do what should, to me, be able to be coded simply.

[2kaud]

I don't believe it! Following on from PiotrNycz, this works (C++20):

#include <tuple>
#include <iostream>

void printTuple(const auto& tp) {
    std::apply([](const auto& ...e) { ((std::cout << e << " "), ...); }, tp);

    std::cout << '\n';
}

int main() {
    const std::tuple tp {10, 20, 3.14, 42, "hello"};

    printTuple(tp);
}

Instead of std::apply(), I would prefer a range-for (I think range-for is under-specified to what it could be). But still much improvement over the other code..

[PiotrNycz]

@2kaud a small correction: this is valid c++17 code, none of C++20 features is needed

[2kaud]

std::apply() was introduced in C++17, but wasn't unconstrained placeholder (auto) as used in printTuple() function definition introduced with concepts in C++20?

[PiotrNycz]

@2kaud - true, I overlooked that, because - well, signature of printTuple is irrelevant in the subject: "iterating over tuple elements". It is easy to change to pure C++17:

auto printTuple = [](const  auto& tp) {
 ...
}

OR more classic:

template <typename T>
void printTuple(const  T& tp) {
 ...
}