Rebase ranges on c++20

[MikeGitb]

I have no idea, if anyone has time for this, so feel free to close as won't fix, but as

a) not all of ranes-v3 has been merged into the c++20 standard b) c++20 finally got native concepts

I'd very much like to see a rebased version of this library, that just contains the functionality not provided by c++20 and makes full use of c++20 in its implementation to improve compile-times as much as possible.

Aside from the time required, another problem I see is that such a library would have to live in a separate repository, as I guess no one has interest in deprecating the existing pre c++20 ranges-v3 library anytime soon. This might be problematic from a maintenence point of view.

[david-bakin]

I would like to see this too ... but in the meantime, is there a document somewhere (here or elsewhere) that lists what parts of range-v3 did not make it into C++20? I just found out, for example, that the sinks to<containerType>() and to_vector() did not. What else is missing from C++20? It would help when using the (superior) ranges-v3 documentation with a C++20 compiler ...

[cjdb]

C++ Reference has you covered.

• Concepts library
• Iterators library (everything marked with C++20)
• Ranges library
• Algorithms library (everything in namespace std::ranges)

[david-bakin]

@cjdb - yes but how does that tell me directly what didn't make it into C++20?

For example, I couldn't figure out how to easily materialize a view into a container. There appeared to be nothing on cppreference.com relevant. There are no examples I found of that sort of thing in the web. I finally found a SO answer that said range-v3 had to<> and to_vector but they didn't make it into the standard. So now ... I'd like to know what else didn't make it into the standard.

Because, you know, I could just swipe the code I needed from here - subject to the license of course - if I was desperate for that functionality! If only I knew I'd need to before wasting time searching cppreference to discover its missing ...

[JohelEGP]

Hmm... not directly, but the range-v3 headers have a namespace cpp20 where things that got into the standard have aliases, like https://github.com/ericniebler/range-v3/blob/d098b9610ac2f182f667ae9274ac2fac7f1327f5/include/range/v3/view/common.hpp#L225-L236 Everything else didn't make it into the standard, yet.

[david-bakin]

@johelegp - Thank you, I will look at that.

[davidhunter22]

In general should I expect to be able to use the range-v3 stuff with ranges from a standard library? My particular interest is in using things like the to<> from range v3 with ranges from the latest MSVC range implementation.

[cjdb]

In general should I expect to be able to use the range-v3 stuff with ranges from a standard library?

No. Standard ranges offer no guarantees for compatibility with other ranges libraries in C++20. I don't know to what effort range-v3 will try to maintain compatibility with the major vendors.

[davidhunter22]

Thanks for the quick reply. I did find https://github.com/cor3ntin/rangesnext which looks perfect, just new proposed stuff for C++23.

[MikeGitb]

@cjdb : Shouldn't a view/range from the standard match the requirements from ranges-v3 and vice versa? So why would the libs not be compatible?

[JohelEGP]

IIRC, underspecification of range adaptors means there's no portable way for range-v3 or any other library to opt into the piping syntax.

[MikeGitb]

Are you telling me I won't be able to write my own views in a way that is guaranteed to be portable across different tolchains? that sucks :(

[david-bakin]

No. You can, if you restrict yourself to the defined, standard, APIs. But a library that supplies multiple moving parts is not so constrained. It can be implemented internally using private methods, using knowledge of its own implementation. Thus pieces of a library aren't necessarily mix&match with pieces of another library implementing the same public, standard, API. (AFAIK, this is true in general, although I could stand to be corrected if it isn't true of ranges in particular.)

On Fri, Mar 12, 2021 at 3:01 AM MikeGitb @.***> wrote:

Are you telling me I won't be able to write my own views in a way that is guaranteed to be portable across different tolchains? that sucks :(

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ericniebler/range-v3/issues/1565#issuecomment-797416904, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA7YLBEZH72NMQCGEDDGLJ3TDHX7RANCNFSM4QUQXH6A .

[MikeGitb]

@david-bakin : But wasn't the point of all the concept work done during the development of ranges (v3 or std) that you can e.g. write

<anything-that-satisfies-requirement-X> | v3::to<std::vector>(); and not just <ranges_v3_typeX> | v3::to<std::vector>();

In other words: ranges_v3 as well as the std version where AFAIK designed to be as generic as possible. The only reason I can imagine, why parts from the two libraries wouldn't match is because the concepts they use don't match up. E.g. (extreme example) one lib uses begin() and end() the other uses start()and end() to get the starte and end of a range.

[david-bakin]

@MikeGitb - Sure, and that'll work for you: any type that meets the contract will work in that way. But a library implementation can do better.

Consider: Someone builds a library to manipulate jpegs. It's got a nice API that lets you load up a jpeg and do all kinds of interesting things to it. It becomes popular. Someone else thinks he can do it better (faster, better image quality, whatever) so he writes another library with the exact same interface but, you know, his "gaussian blur" performs better. You wouldn't expect to be able to take a jpegish handle from the first library and pass it to the second library's gaussian blur function and expect it to work.

Not the same situation: Ok, consider this: People complain to the second guy that his library doesn't work with the jpegish handles from the first library. So he fixes that. When he gets a handle in he checks to see if it is one of his or if it belongs to the other. If it's one of his he goes to work. If it is from the other he calls the other's function to do the work. Now his library is interoperable but you know what? His gaussian blur is still faster on his own handles but performs just as poorly on the other's handles.

That's kind of the situation you've got here. Nobody is constraining the internals of a library. It's delivered in one piece, and you use it that way, Mix and match works on published interfaces - but you can do better if you know you're working on your own pieces.

Now to get back to C++. You start with some template class. Then you discover that large parts of the semantics and implementation of that class don't actually depend on the template parameters. So, to get additional benefits of correctness, maintainability, and performance, you refactor that template class so it derives from a protected base class that implements those template-parameter-invariant pieces. Externally there's no change. But internally it works better, and you can keep it going better. So now, let's have a related set of more than one such class. Externally, nothing changes. But internally they start using, directly, methods and data from the protected base classes to get, again, those benefits of correctness, maintainability, and performance. And there you are. If your classes now have to interoperate with some other set of classes implementing that exact same public surface, well, you can make sure they do. But that doesn't mean you can't take advantage of your knowledge of the internals of your classes to give better, say, performance. But in fact, I don't think the C++ standard says your objects have to fully interoperate that way with similarly named objects from some other library. (Someone please correct me if I'm wrong.)

On Fri, Mar 12, 2021 at 8:52 AM MikeGitb @.***> wrote:

@david-bakin https://github.com/david-bakin : But wasn't the point of all the concept work done during the development of ranges (v3 or std) that you can e.g. write

| v3::to(); and not just | v3::to(); In other words: ranges_v3 as well as the std version where AFAIK designed to be as generic as possible. The only reason I can imagine, why parts from the two libraries wouldn't match is because the concepts they use don't match up. E.g. (extreme example) one lib uses begin() and end() the other uses start()and end() to get the starte and end of a range. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe .

[cjdb]

@david-bakin : But wasn't the point of all the concept work done during the development of ranges (v3 or std) that you can e.g. write

<anything-that-satisfies-requirement-X> | v3::to<std::vector>(); and not just <ranges_v3_typeX> | v3::to<std::vector>();

In other words: ranges_v3 as well as the std version where AFAIK designed to be as generic as possible. The only reason I can imagine, why parts from the two libraries wouldn't match is because the concepts they use don't match up. E.g. (extreme example) one lib uses begin() and end() the other uses start()and end() to get the starte and end of a range.

No work has been done to make standard ranges a non-closed ecosystem. It's got nothing to do with concepts, and everything to do with the fact that the machinery is completely unspecified, which means third-party libraries can't rely on anything working across standard library versions, let alone different standard library implementations.

[MikeGitb]

No work has been done to make standard ranges a non-closed ecosystem. It's got nothing to do with concepts, and everything to do with the fact that the machinery is completely unspecified, which means third-party libraries can't rely on anything working across standard library versions, let alone different standard library implementations.

The underlying machinery is completely irrelvant for the question, whether or not I can combine a view/action from one library with a view from another

Ranges was explicitly designed, such that e.g.

 X | std::views::take(5)

works with any X that models a certain concept ( I believe std::range, but haven't checked). Why on earth should it not work with a v3_ranges::ints(100) as X and vice versa?

The various views in both libraries would be completely useles,, if they only worked with types from that library itself.

Consider: Someone builds a library to manipulate jpegs.

We are not talkign about a library that monipulates jpegs. We are talking about the ranges library.

What the two of you are claiming is effectively the same as: "because the implementation of classic algorithms, like std::sort isn't specified, the algorithm wouldn't work on any iterators that don't belong to non-std containers".

[cjdb]

The underlying machinery is completely irrelvant for the question, whether or not I can combine a view/action from one library with a view from another

The underlying machinery is the linchpin for everything related to this discussion.

Ranges was explicitly designed, such that e.g.

X | std::views::take(5)

works with any X that models a certain concept ( I believe std::range, but haven't checked). Why on earth should it not work with a v3_ranges::ints(100) as X and vice versa?

Sure, if X models the requirements of std::ranges::take_view, then X | std::views::take(5) will work. That's a simple example. X | stdv::take(5) | rv::drop(10), isn't as simple, and doesn't work since we're passing unspecified closure objects from adaptor to adaptor.

The various views in both libraries would be completely useles,, if they only worked with types from that library itself.

No, they just can't talk with each other. Both work perfectly fine as closed systems of their own. Interop is still possible at the expense of composition.

If you'll permit an analogy, consider the release of the PlayStation 4. Many people I knew owned PlayStation 3s, and were hoping that the PS4 would support PS3 games. Support without an emulation layer wouldn't be possible because games for the PS3 were compiled for a PowerPC architecture, and the PS4 is x86-64. That doesn't make the PS3 or the PS4 useless: it just meant that neither could play the other's games at the time of release.

What the two of you are claiming is effectively the same as: "because the implementation of classic algorithms, like std::sort isn't specified, the algorithm wouldn't work on any iterators that don't belong to non-std containers".

This is a false equivalence. The C++17 algorithms library is underpinned by the named requirements Cpp17Iterator through Cpp17RandomAccessIterator, which provides a very well-specified interface for iterators, which describe communication between algorithms and ranges.

Range adaptor closures have no such specification and are completely left up to the implementation; and are thus a different kettle of fish.

[MikeGitb]

So the problem is only with chaining views via pipe syntax then?

This is a false equivalence. The C++17 algorithms library is underpinned by the named requirements Cpp17Iterator through Cpp17RandomAccessIterator, which provides a very well-specified interface for iterators, which describe communication between algorithms and ranges.

Range adaptor closures have no such specification and are completely left up to the implementation; and are thus a different kettle of fish.

There needs to be an requirement about what X are valid for X | some_view no? Otherwise, how would you know that X | some_view works for your custom type X?

[MikeGitb]

Apparently, all you have to do in order to pipe a temporary view from ranges v3 to the std::ranges is (partially) specialize std::ranges::enable_borrowed_range. That seems to be a standardized mechanism for letting custom views be used with std::ranges. I'd guess ranges v3 offeres a similar mechanism?

https://godbolt.org/z/a55vs8 https://en.cppreference.com/w/cpp/ranges/borrowed_range

As I said before: The only reason std::ranges and ::rnages might not compose out of the box is because they rely on mismatching concepts. In this case std::ranges::borrowed_range? vs whatever is the v3 equivalent (viewable_range?) and has nothing to do with the internal implementation of the library types.

[cjdb]

So the problem is only with chaining views via pipe syntax then?

This is a false equivalence. The C++17 algorithms library is underpinned by the named requirements Cpp17Iterator through Cpp17RandomAccessIterator, which provides a very well-specified interface for iterators, which describe communication between algorithms and ranges. Range adaptor closures have no such specification and are completely left up to the implementation; and are thus a different kettle of fish.

There needs to be an requirement about what X are valid for X | some_view no? Otherwise, how would you know that X | some_view works for your custom type X?

[range.adaptor.object]/p1 states that X | some_view is a valid expression if the type of X models std::viewable_range, then X | some_view yields a view. [range.adaptor.object] makes no such guarantees about how R | C and R | C | D work, only that they do. This is supported by paragraph 2, which states that a range adaptor object is a customisation point object, but doesn't give any further details regarding its type (similarly to all other CPOs).

Apparently, all you have to do in order to pipe a temporary view from ranges v3 to the std::ranges is (partially) specialize std::ranges::enable_borrowed_range. That seems to be a standardized mechanism for letting custom views be used with std::ranges. I'd guess ranges v3 offeres a similar mechanism?

All enable_borrowed_range does is permits a range to model borrowed_range. That's it. It's got nothing to do with permitting range adaptor objects to be interoperable.

https://godbolt.org/z/a55vs8 https://en.cppreference.com/w/cpp/ranges/borrowed_range

If you uncomment everything and specialise everything relevant, you'll see that there are two cases that still don't work.

As I said before: The only reason std::ranges and ::rnages might not compose out of the box is because they rely on mismatching concepts. In this case std::ranges::borrowed_range? vs whatever is the v3 equivalent (viewable_range?) and has nothing to do with the internal implementation of the library types.

If this were the case, then the Library Evolution group would have less motivation to consider a proposal for a mechanism that standardises range adaptor closure objects ASAP, and I wouldn't be writing the paper that is deemed so critical. If you've found a loophole in the wording that means we don't need to immediately spend valuable time designing and discussing this, please point to the specific paragraphs, and I'll write to Library Evolution on your behalf.

[MikeGitb]

[range.adaptor.object]/p1 states that X | some_view is a valid expression if the type of X models std::viewable_range, then X | some_view yields a view. [range.adaptor.object] makes no such guarantees about how R | C and R | C | D work, only that they do. 

What is the difference between X | some_view and R | C? And what is the difference between RC | D and R | C | D, if RC := R | C?

Is there some specific meaning attached to R and C that makes them distinct from X and some_view?

If you've found a loophole in the wording that means we don't need to immediately spend valuable time designing and discussing this, please point to the specific paragraphs, and I'll write to Library Evolution on your behalf.

As I don't understand the problem, I don't know what to look for.

Edit: But just to be clear: Even if I would understand the problem, I'm certainly not qualified to find loopholes in the wording of the standard or determine if they exist. What little knowledge I have about ranges comes from cppreference.com and toying a bit with ranges-v3

[davidhunter22]

The document here seems quite related to this dicussion so https://www.reedbeta.com/blog/ranges-compatible-containers/

[MikeGitb]

Btw.: My guess is that the compile error in @cjdb 's Example is related to the change from an infinite ranges (iota) to a finite ranges (take) and the associated end iterator/sentinel because it only appears in very specific circumstances. Maybe because std::default_sentinel / ranges::default_sentinel is hardcoded as the end iterator type for some cases and suddenly the start and end iterator can't be compared anymore in the mixed cases.