Infinite loop in expanding abstract class with self type

[Meowcolm024]

The following code causes an infinite loop in type checking

abstract class Foo[T]: Bar { fun x: T }
abstract class Bar(val x: Int) extends Foo[Int] { fun f(y) = this.x + y }

fun test(f: Foo['a]) = if f is Foo then f.x

• for an abstract Bar, we have Bar => (#Bar<Object,Foo> & Bar)
• for a non abstract Bar, we have Bar => #Bar<Object,Foo>

[LPTK]

Minimized:

abstract class Foo[T]: Foo[Int]
(f: Foo['a]) => f : Str

Minimized further (but the trace looks different):

abstract class Foo: Foo
(f: Foo) => f : Str

[LPTK]

Another one:

type Foo = Foo & Int
(f: Foo) => f : Str

We have to be more careful when we expand types during normalization...

[LPTK]

Note that this only happens on the "new parser/typer frontend" (https://github.com/hkust-taco/mlscript/pull/187). In the legacy frontend, we were careful to check for cycles in definitions and yielded "Error in class definitions: illegal cycle involving type Foo" for the latter example. We have to adapt these sorts of checks to the new frontend.

[LPTK]

Of course, we can also get unsoundness from such ill-formed types:

type A = A
//│ type A = A

(1 : A)()
//│ nothing
//│ res
//│ Runtime error:
//│   TypeError: 1 is not a function