python / cpython

The Python programming language
https://www.python.org/
Other
61.34k stars 29.56k forks source link

C API Functions Bypass __import__ Override #88036

Open 0d08f93e-5d9c-463b-91e5-ff3243266059 opened 3 years ago

0d08f93e-5d9c-463b-91e5-ff3243266059 commented 3 years ago
BPO 43870
Nosy @insertinterestingnamehere

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', '3.9'] title = 'C API Functions Bypass __import__ Override' updated_at = user = 'https://github.com/insertinterestingnamehere' ``` bugs.python.org fields: ```python activity = actor = 'Ian.H' assignee = 'none' closed = False closed_date = None closer = None components = [] creation = creator = 'Ian.H' dependencies = [] files = [] hgrepos = [] issue_num = 43870 keywords = [] message_count = 1.0 messages = ['391220'] nosy_count = 1.0 nosy_names = ['Ian.H'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue43870' versions = ['Python 3.9'] ```

0d08f93e-5d9c-463b-91e5-ff3243266059 commented 3 years ago

Some of the import-related C API functions are documented as bypassing an override to builtins.__import. This appears to be the case, but the documentation is incomplete in this regard. For example, PyImport_ImportModule is implemented by calling PyImportImport which does respect an override to builtins.\_import, but PyImport_ImportModule doesn't mention respecting an override. On the other hand some routines (like PyImport_ImportModuleLevelObject) do not respect an override to the builtin import.

Is this something that people are open to having fixed? I've been working on an academic project downstream that involved some overrides to the __import machinery (I haven't figured out a way to do this with just import hooks) and having some modules skip going through our override threw us for a bad debugging loop. The easiest long-term fix from our perspective is to patch the various PyImport routines to always respect an __import override. This technically is a backwards compatibility break, but I'm unsure if anyone is actually relying on the fact that specific C API functions bypass builtins.__import__ entirely. It seems more likely that the current behavior will cause bugs downstream like it did for us.