Use `importlib` as the import mode for `pytest`

[ehogan]

Description

• Closes #3670

• Link to documentation:

• • *

Before you get started

• [x] ☝ Create an issue to discuss what you are going to do

Checklist

It is the responsibility of the author to make sure the pull request is ready to review. The icons indicate whether the item will be subject to the 🛠 Technical or 🧪 Scientific review.

• [x] 🛠 This pull request has a descriptive title
• [x] 🛠 Code is written according to the code quality guidelines
• [x] 🛠 Tests run successfully
• [x] 🛠 All checks below this pull request were successful

---

To help with the number of pull requests:

• 🙏 We kindly ask you to review two other open pull requests in this repository

[bouweandela]

Are you sure all the imports work like this? I'm a bit concerned about the missing __init__.py files in many directories.

[bouweandela]

On the other hand, the tests are passing, so it's probably fine. What's your opinion? Add __init__.py files to be on the safe side or leave it as is?

[ehogan]

On the other hand, the tests are passing, so it's probably fine. What's your opinion? Add __init__.py files to be on the safe side or leave it as is?

As I understand it, __init__.py files shouldn't be needed when running pytest when using the importlib mode.

I have just run the recipe_ensclus.yml and recipe_iht_toa.yml recipes at the MO (because I know we have the data for these recipes!) using this branch and they both succeed, so these changes do work when executed 💪

I would prefer not to add lots of __init__.py files, if we can help it, from a maintenance perspective 😊

[ehogan]

Package-level imports are very good and very welcome, Emma, cheers! Note that Bouwe and me are doing that in #3646 since we added an __all__ member in the init, plus, this is the way to import anyway. I would, however, support Bouwe's idea to add init files - they don't need maintenance, since they be empty 🍺

Apologies for not replying to this sooner! I did some quick research into __init__.py modules and discovered that in Python 3.3 a new type of package was introduced (alongside regular packages); namespace packages, see Python documentation: Modules: Packages. Namespace packages have no __init__.py module. PEP 420 provides information about implicit namespace packages.

I haven't yet figured out the advantages and disadvantages for each approach / which is prefered / whether they can be used together, but it might be good to understand this before deciding what approach to take 👍

[bouweandela]

I think it will be simplest if we make ESMValTool a single package. At the moment, due to missing __init__.py files, various subdirectories are interpreted as separate (namespace) packages. But we can address that later, as long as it works for now.