Gradualizer issues with Gradualizer

[OvermindDL1]

As a test to see how it ran, I ran Gradualizer on Gradualizer itself, there are a few issues, and they do seem to be legitimate:

╰─➤  find ebin/ -name '*.beam' -exec ./gradualizer -pa ebin {} \;                                                               127 ↵
ebin/gradualizer_db.beam: The function get_src_map on line 145 is expected to return #{module() => string()} but it returns #{module() => file:filename()}
ebin/gradualizer_db.beam: The pattern #{K := Types} on line 156 doesn't have the type #{mfa() => [type()]}
ebin/gradualizer_db.beam: The pattern #{K := TypeInfo} on line 271 doesn't have the type #{mfa() => #typeinfo{}}
ebin/gradualizer_db.beam: The pattern #{autoimport := false} on line 289 doesn't have the type opts()
ebin/gradualizer_db.beam: The pattern #{Mod := Filename} on line 308 doesn't have the type #{module() => string()}
ebin/gradualizer_db.beam: The pattern #{Mod := Filename} on line 320 doesn't have the type #{module() => string()}
ebin/gradualizer_db.beam: The variable 'Forms1' on line 343 has type {file:posix() | badarg | system_limit, file:filename()} |
          file:filename() |
          abstract_forms() but is expected to have type gradualizer_file_utils:abstract_forms()
escript: exception error: no match of right hand side value 
                 {{type,0,any,[]},
                  #{},
                  {constraints,#{},#{},
                               {set,0,16,16,8,80,48,
                                    {[],[],[],[],[],[],[],[],[],[],[],[],[],
                                     [],[],[]},
                                    {{[],[],[],[],[],[],[],[],[],[],[],[],[],
                                      [],[],[]}}}}}
  in function  typechecker:type_check_expr/2 (src/typechecker.erl, line 1005)
  in call from typechecker:type_check_expr/2 (src/typechecker.erl, line 906)
  in call from typechecker:type_check_block_in/3 (src/typechecker.erl, line 2023)
  in call from typechecker:type_check_block_in/3 (src/typechecker.erl, line 2024)
  in call from typechecker:check_clause/4 (src/typechecker.erl, line 2202)
  in call from lists:map/2 (lists.erl, line 1239)
  in call from typechecker:check_clauses/4 (src/typechecker.erl, line 2183)
  in call from typechecker:check_clauses_fun/3 (src/typechecker.erl, line 2153)
ebin/gradualizer_file_utils.beam: The tuple on line 28 does not have type parsed_file()
ebin/gradualizer_file_utils.beam: The tuple on line 37 does not have type parsed_file()
ebin/rebar_prv_gradualizer.beam: Undefined remote_type rebar_state:t/0 on line 8
ebin/rebar_prv_gradualizer.beam: Undefined remote_type rebar_state:t/0 on line 22
ebin/rebar_prv_gradualizer.beam: Undefined remote_type rebar_app_info:t/0 on line 32
ebin/rebar_prv_gradualizer.beam: Undefined remote_type rebar_app_info:t/0 on line 38
ebin/rebar_prv_gradualizer.beam: Call to undefined function rebar_dir:src_dirs/2 on line 71
ebin/typechecker.beam: The variable 'Tys' on line 366 has type [abstract_type()] |
          [abstract_type()] |
          [abstract_type()] |
          any |
          af_singleton_integer_type() |
          [(Name :: af_atom()) | af_record_field_type()] |
          [abstract_type()] |
          [af_assoc_type()] |
          any |
          [af_singleton_integer_type()] |
          af_function_type() |
          [{type, anno(), any} | abstract_type()] |
          [] |
          [] |
          [af_singleton_integer_type()] but is expected to have type [type()]
ebin/typechecker.beam: The tuple on line 597 does not have type int_range()
ebin/typechecker.beam: The tuple on line 615 does not have type type()
ebin/typechecker.beam: The tuple on line 781 does not have type any |
          {type_error, type()} |
          {fun_ty, type(), type(), constraints:constraints()} |
          {fun_ty_any_args, type(), constraints:constraints()} |
          {fun_ty_intersection, [type()], constraints:constraints()} |
          {fun_ty_union, [any()], constraints:constraints()}
ebin/typechecker.beam: The pattern {VEnv3,Cs2} on line 2246 doesn't have the type none() | none() | none() | any()
ebin/typelib.beam: The tuple on line 52 does not have type type()
ebin/typelib.beam: The operator '++' on line 97 is expected to have a non-list argument of type module() | file:filename()
ebin/typelib.beam: The tuple on line 129 does not have type type()

And it looks like there is a crash of Gradualizer itself within all that as well.

[josefs]

Thanks @OvermindDL1! We're aware that self-gradualization doesn't work just yet. Indeed, we just recently celebrated that the gradualizer doesn't crash when run on itself.

Since the gradualizer is not what you would call a typical Erlang program, I haven't made it a priority to get self-gradualization to work. I've focused on programs that most people write.

That being said, we certainly want to make this work eventually. So thanks for opening this ticket.

[OvermindDL1]

Thanks @OvermindDL1! We're aware that self-gradualization doesn't work just yet. Indeed, we just recently celebrated that the gradualizer doesn't crash when run on itself.

I figured, it's mostly just for fun as well as it's a useful issue to have as a Target since I didn't see one otherwise. :-)

Since the gradualizer is not what you would call a typical Erlang program, I haven't made it a priority to get self-gradualization to work. I've focused on programs that most people write.

As has been learned by the Elixir's Dialyxir project, any possible type that is representable by the BEAM is indeed used in the wild far more often than you'd think, so even Gradualizer itself is still a 'usual' erlang program. :-)

[Zalastax]

Only partially fixed, right?

[josefs]

@Zalastax, indeed! The ticket was automatically closed by github. Thanks for reopening it.

[zuiderkwast]

It would nice to require that gradualizer doesn't crash when checking itself. Perhaps we can add that to the .travis file?

Sometimes these crashes are fixed but every now and then, we introduce new crashes. Currently, I get this crash:

$ make gradualize 
rebar3 escriptize
===> Verifying dependencies...
===> Compiling gradualizer
===> Building escript...
cp _build/default/bin/gradualizer .
./gradualizer -pa src/ src/*.erl
src/constraints.erl: The record field Cs#constraints.exist_vars on line 21 at column 58 is expected to have type set(Element) but it has type sets:set(string())
src/constraints.erl: The record field C1#constraints.exist_vars on line 50 at column 40 is expected to have type set(Element) but it has type sets:set(string())
src/gradualizer.erl: The variable Forms on line 52 at column 36 is expected to have type [erl_parse:abstract_form()] but it has type [erl_parse:abstract_form() | erl_parse:form_info()]
src/gradualizer.erl: The variable File on line 69 at column 29 is expected to have type file:filename() but it has type cover_compiled | non_existing | preloaded | [char()]
src/gradualizer.erl: The function filename:join on line {87,55} is expected to return atom() | [char()] | [char() | atom() | deep_list()] but it returns file:filename_all()
escript: exception error: no case clause matching false
  in function  typechecker:list_op_arg_types/2 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 2529)
  in call from typechecker:type_check_list_op_in/6 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 2506)
  in call from typechecker:type_check_expr_in/3 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 1962)
  in call from typechecker:check_clause/4 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 3024)
  in call from typechecker:'-check_clauses/4-fun-0-'/4 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 2999)
  in call from lists:foldl/3 (lists.erl, line 1263)
  in call from typechecker:check_clauses/4 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 2997)
  in call from typechecker:do_type_check_expr_in/3 (/home/viktor/repos/gradualizer/_build/default/lib/gradualizer/src/typechecker.erl, line 2179)
make: *** [Makefile:50: gradualize] Error 127

[Zalastax]

@zuiderkwast I can add that to the CI!

[zuiderkwast]

That's great @Zalastax :star: but before we merge that, we should fix the crash too I guess.