Rock fails to match generics if class doesn't have new functions

[horasal]

This fails:

Foo : abstract class<T>{
    init: func
    acceptThis: func (other: This<T>){
        other acceptOther(this)
    }

    acceptOther: func (other: This<T>){
    }
}

describe("Should match generic class", ||
    bar := Foo <Int> new()
    baz := Foo <Int> new()
    baz acceptThis(bar)
)

rock/mytest.ooc:4:15 error No such function acceptOther(Foo<T>) for `Foo<T>`

        other acceptOther(this)
              ~~~~~~~~~~~      

rock/mytest.ooc:7:5 info Nearest match is `Foo acceptOther(other: Foo<T>)`
..but the arg `this` should be of type `Foo<T>` instead of `Foo<T>`

    acceptOther: func (other: This<T>){
    ~~~~~~~~~~~                        

[FAIL]

This compiles and passes:

Foo : class<T>{
    value: T

    init: func(=value)

    acceptThis: func (other: This<T>){
        other acceptOther(this)
    }

    acceptOther: func (other: This<T>){
        value = other value
    }
}

describe("Should match generic class", ||
    bar := Foo <Int> new(2)
    baz := Foo <Int> new(3)
    baz acceptThis(bar)
    assert(baz value == bar value)
)

[horasal]

Some debug info:

Resolving call other acceptOther(this)
Got tDecl ClassDecl Foo <T>, resolving, meta = ClassDecl FooClass <T>

====> Search other acceptOther(this) in FooClass (which has 4 functions)
  - Got Foo acceptThis(other: Foo<T>)!
  - Got Foo acceptOther(other: Foo<T>)!
  - Got Foo __defaults__!
  - Got static Foo __load__!
Regular arg consumes one.
matchesArg, score is now 256
typeScore for Foo<T> vs Foo<T> == -100000    for call other acceptOther(this) (Foo<T> vs Foo<T>) [0x1159d00 vs 0x1159d00]
Final score = -99487
Considering fDecl Foo acceptOther(other: Foo<T>) for fCall other acceptOther(this), score = -99487

    \o/ Found fDecl for acceptOther, it's Foo acceptOther(other: Foo<T>)
** [refScore = -1, ref = Foo acceptOther(other: Foo<T>)] Got suggestion Foo acceptOther(other: Foo<T>) for other acceptOther(this)
Regular arg consumes one.
matchesArg, score is now 256
typeScore for Foo<T> vs Foo<T> == -100000    for call other acceptOther(this) (Foo<T> vs Foo<T>) [0x1159d00 vs 0x1159d00]
Final score = -99487
** New high score, -99487/Foo acceptOther(other: Foo<T>) wins against -1/Foo acceptOther(other: Foo<T>)

T is resolved and does have the ref Class. However, when calculating score, T (TypeAccess->inner) lose its ref.

[davidhesselbom]

It might not help, but, don't you need an init function in the first example?

[horasal]

@davidhesselbom

It works after adding init... but I believe this is just a coincidence. Because the same error also occured in ooc-kean/collections/Vector.ooc:

collections/Vector.ooc:66:14 error No such function copy(Int, Vector<T>, Int, Int) for `Vector<T>`

        this copy ~to (sourceStart, this, targetStart, capacity)
             ~~~~                                               

collections/Vector.ooc:69:2 info Nearest match is `Vector copy~to(sourceStart: Int, target: Vector<T>, targetStart: Int, capacity: Int = 0)`
..but the arg `this` should be of type `Vector<T>` instead of `Vector<T>`

    copy: func ~to (sourceStart: Int, target: Vector<T>, targetStart: Int, capacity := 0) {
    ~~~~                                                                                   

[FAIL] [287ms, 0ms] collections/VectorTest.ooc (compilation error - exit code: 1)

while Vector does have init functions.

I will search for a better test case....

[davidhesselbom]

Vector is an abstract class, unlike Foo above. Does that matter at all?

[horasal]

You are correct.

I have updated the issue by making class abstract.

refs of typeargs are missing if there doesn't exist a new function. However, even though abstract class may have init functions, no new function is generated at all. (rock/middle/ClassDecl.ooc/line216)

I'm still wondering how the ref of typearg becomes null after it has been suggested with Class.

[fasterthanlime]

I'm still wondering how the ref of typearg becomes null after it has been suggested with Class.

That sounds like something that would be done somewhere in rock "to make sure it's re-resolved again properly". The fix is probably a typeArg clone() call away.

This kind of issue is why I started oc / wanted to make a much cleaner compiler, btw. rock sure may be compiling a lot of things, but it's still a pile of hacks, especially as far as generics are concerned.

[davidhesselbom]

@fasterthanlime I know this isn't strictly on topic, but since you brought up generics...

You know how, when you have, say, an ArrayList<T> and you use its [] operator and it returns the value at the index by memcpying T size bytes, even if T is a cover type (because it doesn't know at compile time what T is), and you have this program that uses lists a lot, and you end up with a program that spends most of its time doing memcpy calls? Is that something you consider a problem and want, plan, or think there's something that can be done to improve? I have no expectations at all, I'm just wondering. Also, no sarcasm, just to be clear! :)

[fasterthanlime]

I have no expectations at all

Now you have the right mindset!

It's definitely a problem, but I don't think it's one that can be solved in rock.

[davidhesselbom]

Alright. I vaguely remember you mentioning something about generics and a thesis of yours where you fixed things that you hadn't had the time to add to rock. Can't seem to find anything about that now, though...

[fasterthanlime]

I vaguely remember [...] specialization

Oh, it's real: https://speakerdeck.com/nddrylliog/generic-specialization-in-ooc and https://github.com/fasterthanlime/rock/commits/specialize

Got me a good grade too, like the original j/ooc compiler :) Let's see where this one goes.

[davidhesselbom]

Ooh, there it is! So what became of it? The presentation made it look like things went well, at least, so why is the problem one you

don't think [is] one that can be solved in rock

?

Again, I have no expectations.

[davidhesselbom]

The reason I'm asking is, well, that application I mentioned where most of the time is spent in memcpy... that's the one I'm making. And while I can imagine ways of implementing lists other than by using ooc generics, I'd prefer not to, so I'd like to know whether specialization is still in its own branch because it won't work after all, or because you simply haven't had the time and neither has anyone else, or because it will break everything ever built in ooc, and not just the SDK, or if there's some other reason that I've missed. If it's just a matter of time constraints, I (and the rest of my team) would be interested in helping out.

[fasterthanlime]

why is specialize unmerged

I mention a few reasons in the deck:

• There's a big difference between "getting one case of specialization to work" and "getting specialization to work on all generic ooc classes ever written". The semester project was about getting one case to work, to show it was possible, but just look at the commits in the branch.. it's not quality code.
• Some abstractions are leaky, for example (cited in the deck), look at the removeAt implementation in ArrayList - it uses memmov directly. The deck suggests a potential solution (range assignments), but, again, doing it right would take much time and effort.

The thing with adding more big things to rock is: I'm not going to do it, and I don't think anybody else should (but I'm not anybody else's parent..). I tried to subtly hint you by linking to cork (here's its original motivation) - a new compiler project of mine.

And I'm pretty sure the reaction of anyone using ooc semi-seriously is "shit, no, Amos has yet ANOTHER pet project on top of music and shit, we'll never get anything fixed in rock now, which is the only semi-useful piece of software he's written" — but please, read the cork doc. Feel my pain. It's a miracle rock ever got to compile what it did, because had we (the authors) been slightly less stubborn, we'd have realized how broken it was years ago.

The thing is, we did realize how broken it was, to some extent. oc was started in 2010, that's just one year after rock. But then I was so busy fixing up rock (mostly because people bitched at me "hey come fix bugs on the real thing instead of having fun on vaporware" — but also because, well, I was using rock to write oc) that I didn't get too far. And it was too ambitious, it was a "fuck everything we've done until now, let's do all the outrageous things" (pluggable backends, coroutines, woo).

The worst thing about rock is not that some features work only halfway, it's not the random crashes, it's not the unreliable maintainer (me) that sometimes uploads broken bootstraps, the worst thing about rock is that complexity within the codebase isn't separated. It could be a complex piece of software that works in various passes, cleanly separated, but it isn't.

To prove my point, let me open VariableAccess.ooc, (should be a simple AST node, right?) and list the ways.. I mean the responsibilities this class has:

• If its parent is a Scope, it warns that there is a "statement with no effect"
• If you're accessing a member, it turns "foo" into "Bar foo" or "bar foo", depending whether it's static or not (in the constructor)
• Returns false to hasSideEffects — which is wrong, btw! A property getter could totally have side effects.
• Oh, yeah, it has a debugCondition final method, just like FunctionCall, or FunctionDecl, because in 2015 we're still using printf debugging to figure out what exactly rock is doing, since its resolve process is so convoluted.
• It has a suggest method, because the access resolving process involves different nodes (Module, Scope, TypeDecl) - all with their own ways to find where variable declarations are hiding. That's good, right, dispatch instead of switch? Doesn't matter, we still have a match in suggest.
  • If we're getting a VariableDecl, and we're a member access (e.g. foo bar instead of just bar), ignore non-member accesses (why can other nodes suggest that in the first place?)
  • Ignores instance members when we're accessing a member of a class
  • If an Addon is suggested, it's in the VariableAccess's suggest method that we point back to the base class (because chances are, the Addon is declared closer to the access, e.g. further up in the same module)
  • Oh yeah we somehow support namespaced imports too — somewhat, sometimes
• isResolved is wrong (see cork's design doc for an explanation) - it checks ALL THE THINGS including: do we have a reference, have we handled func type fuckery yet, do we have a valid type, does it have its own reference, if we do have an expr, does its isResolved method return true? (which, again, is lying).
• I don't even want to get into the resolve method, but..
  • It handles generic boxing/unboxing (including a process we call realtypizing (because fuck words), which basically lets us turn stuff like HashMap<K, V> into HashMap<String, Int>, but also T into Foobar, and so on — oh, fun fact, FunctionCall has a process just like it, except even more convoluted)
  • It handles function types conversion (between, say, Pointer, which would be a simple C function pointer, and Func, which is the Closure struct in disguise). Ah and it does so by creating a StructLiteral and throwing a "Closure" type in there directly, there's no AST node for that, that would be too easy...
  • It has two different algorithms to resolve the access, depending on whether it's a member access or a naked access
  • Oh it has magic to change 42 class to Int, apparently
  • It'll tell you if you forgot to dereference a variable
  • On occasion it'll replace itself with a property getter - except if we're in a property declaration, of course, in that case it somehow marks the property as non-virtual and it's referring to a real member..
• Ah it's also in charge of finding modules you haven't imported, that might contain a type you're referring to
• Oh yeah it also handles builtins, aka replacing __BUILD_DATETIME__ with a string that contains the current date and time, and 4 other built-ins.
• Its isMember method has.. custom logic if it's a namespace access! because import module as mod; mod a print() isn't really a member access, it's just a variable access from elsewhere.

And I've omitted thing for the sake of everyone's sanity. Not trying to shut anyone up, but, c'mon, I've been dealing with that for 6 years, don't I get a third chance now?

[davidhesselbom]

Dude, relax!

I tried to subtly hint you by linking to cork

... and I didn't even look long enough to see that it said "Another attempt at an ooc compiler.", but just assumed it was another language project like Pug. I feel horrible now. Sorry!

But implementing performant immutable data structure is another big project, and since my time is limited, I believe a better compiler can be built.

I don't see how that can possibly be a faster solution, but you would know better than me! If you think you can make a better compiler than rock faster by starting over from scratch, I wholeheartedly support you in doing so. I might even have a shot at a chance of understanding how it works by seeing it evolve.

[fasterthanlime]

Dude, relax!

Haha :) I wasn't angry at all when I wrote that! Just explaining, it's worth it to me too, I have to re-understand all of rock in order to write another piece of software that does the same thing (but differently).

[davidhesselbom]

Assuming you intend to make cork (ooooh, I see it now) completely compatible with existing ooc code, you have a nice suite of tests now that you didn't have when you made rock. That should help :)

[fasterthanlime]

That should help :)

My feeling too. I'm curious if I can get stuff like that working:

do: func (f: Func <T> (Int, Int) -> T) {
    f(1, 2)
}

do(|x, y|
    x + y
)

do(|x, y|
    "#{x}#{y}"
)

(It fails miserably in rock, if you were wondering...)

[vendethiel]

I'm happy to see cork :D. That example would be sweet, what is the reason it fails in rock? Matching of types?

[davidhesselbom]

Shouldn't it be

do: func (f: Func <T> (Int, Int) -> T) -> T {
    f(1, 2)
}

?

[fasterthanlime]

@davidhesselbom Doesn't matter, I'm not using the return value of f - rock fails beforehand.

[vendethiel]

is the still available for the outer function's return type?

[fasterthanlime]

@vendethiel only if it had <T> itself, otherwise it would not be a parameterized function and T would simply be an undefined symbol.