airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
15.31k stars 3.95k forks source link

[source-monday] Incompatible with latest `airbyte-cdk` versions #41961

Open tbrlpld opened 1 month ago

tbrlpld commented 1 month ago

Connector Name

source-monday

Connector Version

2.1.3

What step the error happened?

Configuring a new connector

Relevant information

If you update the CDK version to a later version than the one from the poetry.lock file, you get errors because of the use of Cursor. (This might happen if you build install source-monday directly from Pypi, because the dependency in pyproject.toml is not locked down properly.)

E.g. update dependency:

% poetry update airbyte-cdk

Try to run unittest

% poetry run pytest unit_tests/test_components.py

Then you get

E   ImportError: cannot import name 'Cursor' from 'airbyte_cdk.sources.declarative.incremental' (/Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/incremental/__init__.py)

I would contribute, but I am not experienced enough with Airbyte to refactor this. I guess I could contribute the workaround of adding an upper bound the the dependency definition in pyproject.toml.

Relevant log output

% poetry run pytest unit_tests/test_components.py
=================================================================== test session starts ====================================================================
platform darwin -- Python 3.9.18, pytest-6.2.5, py-1.11.0, pluggy-1.4.0 -- /Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/bin/python
cachedir: .pytest_cache
rootdir: /Users/tibox/1-projects/monday-com-operating-app-integration/airbyte, configfile: pytest.ini
plugins: mock-3.14.0, requests-mock-1.12.1
collected 0 items / 1 error                                                                                                                                

========================================================================== ERRORS ==========================================================================
_______________________________ ERROR collecting airbyte-integrations/connectors/source-monday/unit_tests/test_components.py _______________________________
ImportError while importing test module '/Users/tibox/1-projects/monday-com-operating-app-integration/airbyte/airbyte-integrations/connectors/source-monday/unit_tests/test_components.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/lib/python3.9/site-packages/_pytest/python.py:578: in _importtestmodule
    mod = import_path(self.fspath, mode=importmode)
/Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/lib/python3.9/site-packages/_pytest/pathlib.py:524: in import_path
    importlib.import_module(module_name)
/Users/tibox/.pyenv/versions/3.9.18/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1030: in _gcd_import
    ???
<frozen importlib._bootstrap>:1007: in _find_and_load
    ???
<frozen importlib._bootstrap>:986: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:680: in _load_unlocked
    ???
/Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/lib/python3.9/site-packages/_pytest/assertion/rewrite.py:170: in exec_module
    exec(co, module.__dict__)
unit_tests/test_components.py:14: in <module>
    from source_monday.components import IncrementalSingleSlice, IncrementalSubstreamSlicer
source_monday/components.py:10: in <module>
    from airbyte_cdk.sources.declarative.incremental import Cursor
E   ImportError: cannot import name 'Cursor' from 'airbyte_cdk.sources.declarative.incremental' (/Users/tibox/Library/Caches/pypoetry/virtualenvs/source-monday-CDtQCRH6-py3.9/lib/python3.9/site-packages/airbyte_cdk/sources/declarative/incremental/__init__.py)
================================================================= short test summary info ==================================================================
ERROR unit_tests/test_components.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================== 1 error in 1.23s =====================================================================

Contribute

marcosmarxm commented 1 month ago

@tbrlpld, feel free to me Slack if you need help with the contribution