search

fast-aircraft-design / FAST-OAD

FAST-OAD: An open source framework for rapid Overall Aircraft Design
GNU General Public License v3.0
48 stars 25 forks source link

Relative import while using new library as plugin #366

Closed areysset closed 1 year ago

areysset commented 3 years ago

Describe the bug Some unitary tests exhib top_level_collect errors due to plugin behavior and relative imports. By letting an unused import in the unitary test file (un-commented) : from fastoad.io import VariableIO, permits to avoid the error but the reason behind is not clear to me. We would like a clear explanation about what can be done about import: relative/absolute....

To Reproduce Dowload the relative branch from FASTGA and launch unitary tests on aerodynamics models.

Expected behavior With or without the unused import line commented, the code should work fine.

Error message

test_beechcraft_76.py:None (test_beechcraft_76.py)
ImportError while importing test module 'D:\a.reysset\Documents\Github_new\FAST-GA\src\fastga\models\aerodynamics\unitary_tests\test_beechcraft_76.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\_pytest\python.py:511: in _importtestmodule
    mod = self.fspath.pyimport(ensuresyspath=importmode)
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\py\_path\local.py:704: in pyimport
    __import__(modname)
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\_pytest\assertion\rewrite.py:152: in exec_module
    exec(co, module.__dict__)
test_beechcraft_76.py:32: in <module>
    from ..components.cd0 import Cd0
..\components\__init__.py:14: in <module>
    from .compute_cl_extreme import ComputeExtremeCL
..\components\compute_cl_extreme.py:22: in <module>
    from fastga.models.aerodynamics.external.xfoil.xfoil_polar import XfoilPolar
..\external\xfoil\xfoil_polar.py:35: in <module>
    from fastoad._utils.resource_management.copy import copy_resource
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\__init__.py:28: in <module>
    fastoad.module_management._plugins.load_plugins()
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_plugins.py:37: in load_plugins
    BundleLoader().explore_folder(module_name, is_package=True)
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_bundle_loader.py:93: in explore_folder
    bundles, failed = self._install_python_package(folder_path)
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_bundle_loader.py:357: in _install_python_package
    sub_bundles, sub_failed = self._install_python_package(".".join([package_name, item]))
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_bundle_loader.py:357: in _install_python_package
    sub_bundles, sub_failed = self._install_python_package(".".join([package_name, item]))
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_bundle_loader.py:357: in _install_python_package
    sub_bundles, sub_failed = self._install_python_package(".".join([package_name, item]))
C:\Users\a.reysset\AppData\Local\pypoetry\Cache\virtualenvs\fast-oad-ga-wlz2mNe--py3.7\lib\site-packages\fastoad\module_management\_bundle_loader.py:346: in _install_python_package
    package_contents = list(contents(package_name))
C:\Users\a.reysset\AppData\Local\Programs\Python\Python37\lib\importlib\resources.py:248: in contents
    package = _get_package(package)
C:\Users\a.reysset\AppData\Local\Programs\Python\Python37\lib\importlib\resources.py:47: in _get_package
    module = import_module(package)
C:\Users\a.reysset\AppData\Local\Programs\Python\Python37\lib\importlib\__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
..\aerodynamics.py:18: in <module>
    from .aerodynamics_high_speed import AerodynamicsHighSpeed
..\aerodynamics_high_speed.py:25: in <module>
    from .components import ComputeMachInterpolation
E   ImportError: cannot import name 'ComputeMachInterpolation' from 'fastga.models.aerodynamics.components' (D:\a.reysset\Documents\Github_new\FAST-GA\src\fastga\models\aerodynamics\components\__init__.py)

Environment

Additional context Add any other context about the problem here.

areysset commented 3 years ago

Importing whatever method or class from fastoad (ex: from fastoad.module_management import constants) even if not used leads to a working test with no error.

christophe-david commented 3 years ago

When launching the test, I get the same error message with some more information (maybe because I use Python 3.8?): there may be a circular import.

E   ImportError: cannot import name 'ComputeMachInterpolation' from partially initialized module 'fastga.models.aerodynamics.components' (most likely due to a circular import) (D:\cdavid\PycharmP
rojects\FAST-GA\src\fastga\models\aerodynamics\components\__init__.py)

I guess the problem is that FAST-OAD loads plugins (and by doing so, does the Python imports) as soon as it is loaded. And because of that, the "time" when you import the fastoad package matters. If you do this import before everything else, all your code is scanned by Python at this time and all goes fine. But if you do this import "later", there goes the cycle. More specifically, your test imports your aerodynamics package, and while it follows all the import chain, it comes to an import of fastoad, which triggers the loading of plugins, which triggers an import of all your code... including the aerodynamics package.

A fix in FAST-OAD maybe to delay the loading of plugins to ensure it is done only when needed. It may be a good thing also because it should speed up some operations (it is annoying to wait for the loading process when doing just fastoad -h in CLI).

In the meantime, the workaround I can suggest is to ensure fastoad is loaded before any test. It can be done by adding the pytest "pre-processing" file conftest.py at the root of your project with this simple content:

import fastoad
christophe-david commented 2 years ago

PR #409 may fix this problem, since it actually delays the loading of plugins. To be confirmed...

christophe-david commented 2 years ago

As anticipated, PR #409 fixes this issue (tests done on plugin_error branch ).

florentLutz commented 2 years ago

With the update of the dependencies of FAST-GA to FAST-OAD-core, some unit tests can't run because some group that are tested are using submodels which cannot be loaded. The solution is to add an instance of the FASTOADLoader inside the conftest.py

christophe-david commented 1 year ago

This problem is only with unit tests, with an existing solution as described above. Closing the issue.