C++21

[ThePhD]

C++17 has stuff we need, but I have no faith that VC++ will get to them before 2021. Here's a list of things we'd need from C++17 that do not exist in either clang++, VC++, or both:

• [ ] constexpr-if
• [ ] fold expressions
• [x] noexcept function types
• [ ] template <auto X> (for c_call and static_trampoline)
• [ ] feature-test macros
• [ ] static reflection (still not in C++17: hopefully by C++21 it will be there and implemented in all compile- hahahaha no it's not)
• [x] std::any
• [x] std::variant
• [x] std::string_view (w, u16, u32)

These will allow us to get pretty great compiler and performance savings out of our code by adding specializations for no-exception function types (already there), fold-expressions to reduce the sheer compiler bloat that comes from unraveling argument packs, and constexpr-if to get rid of tons of redundant functions whose sole purpose is to check a single typedef or a single property of a type.

Gating them behind feature-test macros will allow us to check in a standards-compliant way for when a compiler FINALLY gets it together. I was not able to confirm if VC++ has feature-test macros, but I know it's also missing all of the above. clang++ and g++ might have back-ported the feature to older compilers, but I am still unsure.

[Nava2]

Not suggesting to switch to CMake, but this cmake library works well to do exactly what you want here. Perhaps it could be done via some python scripts?

[ThePhD]

https://github.com/ThePhD/sol2/commit/b811e73b902ff151bc4c7fe8d19c67e530fbf277 noexcept-proofs a very large chunk of sol2's code. In C++17 mode on later compilers, this will be triggered automatically and sol2 will turn off exception handling trampolines exclusively around functions that are marked as noexcept, potentially providing slight speedups to that class of users.

[sagamusix]

I'm not sure if this issue is the right place to discuss, but I guess it's at least related. You mention in the release notes that v2.17.5 will be the final release supporting a whole bunch of compilers that are still supported by their manufacturers and that are still being widely used, because they are the default toolset on many distributions.

I can understand that you want to make use of more modern C++ features, but if you want to drop support for so many compilers, it would be weird to do this while just incrementing the minor or patch number - such a version should definitely be called sol3, possibly allowing sol2 to be developed in parallel. And by "develop" I do not necessarily mean active development of new features, but at least backporting bugfixes from sol3, if applicable. If you are not interested in maintaining the old version, maybe another person can be found to do this.

It seems a bit asymmetric that there is still support for Lua 5.2 and older but only the very latest compilers are supported. If a project has the chance to use the latest available compiler for their platform, I doubt that they would have a reason to be stuck with an old, unsupported version of Lua (only Lua 5.3 currently receives bugfixes).

To add some personal motivation, I would like to always support at least the two most recent versions of Visual Studio with my software, which are currently VS2015 and VS2017. Only supporting a single compiler version can sometimes lead to surprising dead-ends if a compiler bug is found.

[ThePhD]

The latest still works in VS 2015, Clang 3.4, etc: all of the travis tests are still there and still showing as passing. You can always flip through travis tests to know specifically when compiler X is going out the window.

Not even clang supports half of what I listed there, so you literally don't have anything to worry about until -- at the earliest -- 2019. The reason this issue is listed and the reason I want to focus on moving on quickly is because there have been over a dozen users who have complained about things like Binary Size and Compile Times, all things that C++17 can help alleviate. It's in my best interests to have a new version of Sol up and running the minute C++17 support kicks in for a good chunk of the compilers. That will likely mean waiting until 2019 for the next version of Visual Studio.

As a side note, Visual Studio 2017's support for many things is improved, but they're not going to be shipping new compiler features until the next Big Version of Visual Studio. That being said, Visual Studio 2017 support is essentially identical to Visual Studio 2015 support as far as needed-features is concerned, meaning what you want/need out of sol2 won't be going anywhere anytime soon.

When we start making serious changes, it will be sol3. I will likely updated the compatibility page to continue pointing to higher and higher versions sol v2.1x.x that continue to prove that they pass the compilation and runtime tests on older compilers.

[ThePhD]

Finally, regarding Lua support: that's completely orthogonal to compiler features. Lua 5.1+ support is because people like LuaJIT and LuaJIT is feature-locked to 5.1, despite meager compatibility patches they keep adding to their headers that break my own compatibility wrapper and have people filing issues against me for LuaJIT 2.1.0-beta3.

Barely anyone uses 5.2 because there's no pressing reason to: there's no special JIT tied to its version like LuaJIT and it's strictly inferior to 5.3: we test it as a formality and because somebody's not going to update their debian/ubuntu machine and will be using 5.1 / 5.2 because "that's what on their machine" and running a Yet Another Helpdesk/Debug session is not high on my list of things to do.

Anyone who cares / updates gets themselves a freshly built, ANSI-C compatible slice of Lua right off the website, which is 5.3 (or whatever the latest version ends up being when someone reads this) and cannot have an easier build + installation path.

If LuaJIT migrated to 5.3, I'd seriously consider throwing 5.1 out the window because there's a host of optimizations I do not make strictly because the Lua 5.1 C API is lossy/bad in many regards. Obviously, it would once again be considered part of sol3, so nothing for you to worry about.

[sagamusix]

Thanks for the clarifications. It's just that those latest release notes read in a rather alarming way. :)

It's in my best interests to have a new version of Sol up and running the minute C++17 support kicks in for a good chunk of the compilers.

There is absolutely nothing wrong with that, I just hope that someone (not necessarily you) would keep a version for C++14 compilers alive, if any bugfixes would be necessary.

[ThePhD]

Added more clear wording in the docs about this, and edited the release page: http://sol2.readthedocs.io/en/latest/compilation.html#supported-compilers

[sagamusix]

Yes, that clears up things a lot, thanks for that. Oh, and as I haven't said it yet, thanks a lot for maintaining sol2! Over the years I have tried to use pure Lua as well as other bindings but always got demotivated very quickly again (particularly because this was mostly pre-C++11). Now with sol2, I'm getting things done much more quickly and syntactically more easily, which motivates me a lot to incorporate scripting into my application again!

[ThePhD]

@Nava2 Sorry about not saying anything earlier! As a side note about using that library, I don't think I quite need it here. I'm not looking to "mock up" or "fake" compiler support for stuff: if the compiler doesn't support it, I want to drop it completely from sol2 and have it not exist at all. It's a bit of a rough decision but unlike with the compat headers with Lua, C++ is a much bigger monster and there's no 1 definitive source of "patching it over" (see: boost, various stdlib re-implementations, various optional and variant and ... types implemented by the community).

I'd much rather the user consciously create extension points for the things they are missing. Besides, the compiler-specific stuff I need (fold expression, constexpr-if, template <auto X>) unfortunately can't be faked by Macros and patching functions, much as I would praise the gods if they could be.

[Nava2]

@ThePhD no worries. 👍

Thought I'd mention it, it's a pretty useful library! I completely understand the hesitation using "shims" instead of compiler features. Not to mention, some of these block features and can't work with shims as you said. 😞

[ThePhD]

https://github.com/ThePhD/sol2/commit/16fc7d87ff0b9c2fc18268a420e0b058a67060a1

This commit adds support for std::variant, std::string_view and its friends. We do not make a specialization for std::any, because it is essentially an opaque userdata and should simply be transported as such.

As a side note, it would probably be a good idea if we found out any other C++14/17 types people would want to be vocabulary-serialized. I can't think of any more types that need to be put in or routines that need core getters/pushers asides from the ones I just added.

[ThePhD]

Important side note: places in the API that take a std::string should be changed to take a std::string_view under the new defines for C++17. One problem, however, is that do_string, I believe, requires that the input string is a c_str? This might inhibit our ability to use string_view because it's not guaranteed to be a c-style null-terminated string (just a pointer and a count).

A quick glance at the manual and the core of luaL_dostring seems that it (and everything else) is based off lua_load, which takes a reader-type structure. We'd have to write a string_view reader type to get it to work: https://www.lua.org/manual/5.3/manual.html#lua_load

[ThePhD]

The whole framework has been vetted to use string_view where appropriate: https://github.com/ThePhD/sol2/commit/ca685e01c97191d2f5f40d2eae8b7402c4697808

[ThePhD]

Clang 3.5 has a critical bug that's absolutely blocking not C++Latest or C++17 code, but C++11 and 14 code. It's going to be removed from the tests and removed from support. Thankfully, 3.5 is way out of typical repository storage these days and is not exactly the most supported anymore.

[ThePhD]

Latest release heavily proofs all of sol2's code for C++17, and also adapts all necessary functions to use std::string_view: https://github.com/ThePhD/sol2/releases/tag/v2.18.0

[OrfeasZ]

@ThePhD does this break support for older compilers, or will the likes of msvc140 still work?

[ThePhD]

@OrfeasZ I thought I replied to this already, oops. No, no it shouldn't break VS 2015.

[ThePhD]

There is nothing left to do here, until C++20/C++19 comes out.

The issue will be closed and re-opened in the fuuutuuuuuure.