Null pointers for intrinsic types

[fafalone]

Going through the Windows API codebase for tbShellLib, I realized there's a lot more scenarios of optional types followed by non-optional types that are not UDTs. tB currently allows the special case of vbNullPtr to substitute for UDTs to pass a null pointer, I thought it time to make a formal request for tracking purposes to allow it to substitute for ByRef intrinsics too, that way we're not forced into either treating these as mandatory, potentially causing problems for APIs where they're optional and expecting null if you're not using it, or losing strong typing by declaring it 'As Any

I don't think this would cause any backcompat issues, but if it does, should definitely discuss it. Allowing Optional is a possibility as well; I don't know if one or the other would be easier to implement.

[Greedquest]

I think fundamentally VBA is not a language of pointers, allowing optional in these scenarios is definitely the idiomatic approach if it can be made to work.

In other languages, optional parameters followed by mandatory ones necessitates passing the later arguments as named only (arg:= foo). Maybe implicit overloads for optional args could work, with pattern matching on a signature with and without that parameter, to infer which overload the caller was after.

An issue I see is that Optional x in VBA really means default x -the variable is initialised to the default for its data type (with special case for VARIANTARG = Missing not Empty). So perhaps a new language keyword would make this clearer (or good documentation that Optional means something a bit different in the context of a Declare)

[fafalone]

Good point about initialization with Optional; that wouldn't work for this, would break backcompat; there's no way to indicate you didn't mean an initialized but unassigned variable without specifying, make it no longer optional. Would still like to see that, but wouldn't substitute for vbNullPtr.

While pointers are less common in VBA, that ship has long since sailed for VB6. Pointers are ubiquitous. Win32 API, even more omnipresent. Remeber the alternative here: Is the current status quo of changing a documented function from As Long to As Any then passing ByVal 0 improved by unifying how it's done with UDTs and passing vbNullPtr instead? I'd say yes.

[Greedquest]

Yes definitely agree it's a good feature. Strong typing instead of As Any

Would just be great to label the arguments as optional somehow - since that's what they are most similar in VBA land, rather than dealing with null pointers. How might the language enforce not using nullptr for non-optional arguments without it feeling surprising? That's what I'm wondering about. Or avoiding "why can't I put a null pointer in any old variable, why just a function call" confusion?

[fafalone]

I'd imagine it would remain the same as now; it's treated as 0 and has the effect of assigning 0.

It makes sense in that when you use x = y, it's "Let x Equal y", not "Turn x into y"... so x = vbNullPtr, it's not confusing that x = 0, which is what happens now, subject to type conversion for LongPtr as it's the same as CLngPtr(0).

[Greedquest]

Ah so if it's just a constant - what if the API has an optional byref longptr or byref long on 32 bit in. I guess that would pass varptr(vbnullptr) rather than a literal null pointer? This is why I thought null pointer would be a bit special.

Alternative could have vbNullLong As Long, vbNullSingle As Single etc,Ah so if it's just a constant - what if the API has an optional byref longptr or byref long on 32 bit in. I guess that would pass varptr(vbnullptr) rather than a literal null pointer? This is why I thought null pointer would be a bit special.

Alternative could have vbNullLong As Long, vbNullSingle As Single etc, like vbNullString, all just strongly typed null_ptrs

Would be a bit weird because value types are copied on assignment, so your null would become 0 when assigned or passed byval, but would work for byref scenarios like vbNullString, all just strongly typed null_ptrs

Would be a bit weird because value types are copied on assignment, so your null would become 0 when assigned or passed byval, but would work for byref scenarios

[bclothier]

I think we need to back up and look at the big picture.

VB originally was designed to not deal with pointers because the level of indirection it usually involves can be quite distracting when solving high level problems and by not having to think about pointers but rather objects, VB was made much easier.

For this reason, VB did not have a native addressof operator, and it still does not have the indirection operators (e.g. * deference¹ and -> member access² operators in C).

Instead, VB deal with the ByVal and ByRef semantics which may help simplify things just little but has some surprises if one is not careful with what is actually meant by those semantics. Several people don't fully grok that ByVal o As SomeObject doesn't mean a full copy but rather copy the pointer to the object which behaves differently from ByVal x As Long which indeed do copy the contents.

When we are using vbNullPtr, it's much more closer to doing a CallMyStuff ByVal 0. As fafalone said, vbNullPtr allows us to pass it in to a strong-typed parameter whereas in VB, you'd have to use As Any and lose the type safety and additionally remember to specify ByVal 0 to ensure that you are passing the pointer's value, and not the contents to whatever the pointer is pointing at. Thus we are almost back to the C's world.

I think it might be good to pay attention to what .NET team are trying to solve with nullable types in C# 8, and how Rust manages the memory at compile-time. Possibly other ideas, but what those have in common is to try and avoid null reference error, which referred as a billion-dollar mistake. If you've looked at XBasic, you can see what kind of mess they got themselves when they added pointers, giving it a feel of being a child of an unholy union of C and BASIC and inheriting the worst from each parent. I think having a nullable type support is far more valuable without reverting into the pointer-chasing foolishness in C.

One big problem with Rust's type system is that it makes it nigh-impossible to use win32 APIs without going all unsafe, which makes it hard to be able to statically analyze the codepaths and prove at the compile-time that there will not be an unhandled exception. Thus, they have to invest a fair amount of effort to build a Rust idiomatic library to call win32 in the Rust way, so it's kind of questionable how much of an improvement it is over the VB's current situation where we have to do the same thing albeit manually and without any type safety. If tB is going to do better in this, it'll have to be some kind of opt-in mechanism so that the programmers can choose to leverage the additional type safety and compile-time validations without being constrained to write a whole layer of wrapper just to use it.

So for example, to handle the optional parameter that could be either a null pointer or a specific type, we could declare API as following:

Declare PtrSafe Function  Foo Lib "Bar" (A As Long, B As Long?, C As Double) As Boolean

And from this, the compiler is able to reason that the B can only be either a ByVal 0 as represented by the vbNullPtr or a valid Long variable, including any implicit conversion from whatever to a Long data type, allowing us to handle a call of Foo("1", "2", "3").

--

1. In VB, you would use ***Ptr() functions to dereference the pointer address of a given variable, but that's a function, not an operator.
2. From a language design standpoint, I never liked that you had to choose between . and -> for member access, depending on whether you were dealing with a variable or a pointer to that variable and think they made the right call to not include it in VB.