teal-language / tl

The compiler for Teal, a typed dialect of Lua
MIT License
2.04k stars 101 forks source link

Fix a case union with typearg in return value is not resolved properly #604

Closed pigpigyyy closed 1 year ago

pigpigyyy commented 1 year ago

Recent Teal commits are breaking an small case in my project. Took some time debugging around to find that when check_args_rets passes most of the args and returns checking, the mark_invalid_typeargs function will still mark typearg in union to be unresolvable.

github-actions[bot] commented 1 year ago

Teal Playground URL: https://604--teal-playground-preview.netlify.app

hishamhm commented 1 year ago

@pigpigyyy Thank you for the PR with the testcase and initial solution! I took a long time investigating this. I'm not 100% sure about my solution either, but I think it is a little more complete. You can compare the difference by running TL_DEBUG=1 ./tl check 604.tl and seeing the types resolved by the intermediate nodes (in particular, with a930e41, the "op @funcall" node resolves to (A | nil), instead of resolving to (T | nil) and being ignored later.

As always, let me know if this causes any other problems! Thank you again!!