boostorg / tokenizer

Boost.org tokenizer module
http://boost.org/libs/tokenizer
Boost Software License 1.0
25 stars 42 forks source link

allow tokenizing string_view to string_view(s) #15

Open cedral opened 4 years ago

cedral commented 4 years ago

This requires getting around the dependency on std::string's assign function. I added the following specialization to assign_or_plus_equal to get it working but it could probably be done better.

    template<class Iterator, typename char_t>
    static void assign(Iterator b, Iterator e, std::basic_string_view<char_t>& t)
    {
        t = { std::addressof(*b), static_cast<size_t>(std::distance(b, e)) };
    }
poyenc commented 4 years ago

Cool! I think it's good point to start supporting basic_string_view