`hy.core.language` import in `hy/__init__.py

[brandonwillard]

While working on Hy I've run into problems involving the hy.core.language imports in hy/__init__.py on numerous occasions. These imports are automatic and necessarily trigger Hy compilation—among other things—when loading literally anything from the package.

If one intends to simply import the model objects from hy.models—for example—then hy/core/language.hy is compiled, which seems like a lot of unnecessary work. Furthermore, this behavior obfuscates the true source of errors—where there are any—by triggering package-loading exceptions for functionally independent parts of the package.

It seems like we could replace the hy.core.language imports with their Python equivalents, or simply remove those from __init__.py altogether. Is there something to this scenario that would warrant retention of hy.core.language within hy/__init__.py?

[Kodiologist]

I like the way you're thinking. I've been bitten by this before. It ought to be possible to access pure-Python parts of the hy namespace without invoking the compiler.

There are currently four imports from hy.core.language in __init__.py:

• mangle
• unmangle
• read_str
• read

The first two can just be imported from where they're actually defined, hy.lex, instead of hy.core.language. The latter two could be written in Python instead of Hy.

[brandonwillard]

Exactly!

[brandonwillard]

Maybe put these changes together with https://github.com/hylang/hy/issues/1695 in a reorganizing PR?

[brandonwillard]

While we're on the topic of organization, what do you think about forcing test_bin.py tests to run last and adding a prefix to the .hy test files (e.g. test_*.hy)?
Those test_bin.py tests usually run first and indirectly expose errors more clearly demonstrated by the smaller unit tests that follow. The Hy file renaming should also help with pytest's initial search/collection process, and otherwise avoid issues resulting from temporary files created by editors.

[Kodiologist]

That's a decent idea. I usually run the tests with --ignore tests/test_bin.py anyway because they're so much slower than everything else.

[brandonwillard]

Yet another "decoupling" point: the compiler automatically loads/imports modules for the standard libraries (macros included); it would be nice if the compiler's could run without these. Once again, development can be particularly difficult because of these dependencies.

In other words, would we like to have a lightweight compiler that exclusively translates Hy AST to Python AST and implicitly relies on the current state of its target module for macro and function dependencies? This is possible now that we're using module objects more consistently.

[brandonwillard]

Also, before I forget: HyREPL calls hy_compile on every processed command. When that happens, a HyASTCompiler is constructed and all the stdlib imports are triggered again and again.
We should probably create and reuse a single HyASTCompiler instance for the lifetime of HyREPL, no?

[Kodiologist]

In other words, would we like to have a lightweight compiler that exclusively translates Hy AST to Python AST and implicitly relies on the current state of its target module for macro and function dependencies?

Maybe. I see your argument, but it's hard to guess how things would shake out in practice.