johntdyer
commented
7 months ago Ran into this in #16, and I tried your fix but it didnt seem to make much of a difference
▓ ~/Pr/pixbyt on main !1 ?1 ▓▒░ docker compose build
[+] Building 24.9s (17/22) docker:desktop-linux
=> [pixbyt internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.58kB 0.0s
=> [pixbyt internal] load metadata for docker.io/meltano/meltano:latest 0.3s
=> [pixbyt internal] load .dockerignore 0.0s
=> => transferring context: 110B 0.0s
=> [pixbyt base 1/12] FROM docker.io/meltano/meltano:latest@sha256:1729d935e12010461f19319bd6635 0.0s
=> [pixbyt internal] load build context 0.0s
=> => transferring context: 3.66kB 0.0s
=> CACHED [pixbyt base 2/12] WORKDIR /project 0.0s
=> CACHED [pixbyt base 3/12] RUN apt-get update 0.0s
=> CACHED [pixbyt base 4/12] RUN apt-get install -y curl 0.0s
=> CACHED [pixbyt base 5/12] RUN echo 0.27.2 0.0s
=> CACHED [pixbyt base 6/12] RUN curl -L -o pixlet.tar.gz https://github.com/tidbyt/pixlet/relea 0.0s
=> CACHED [pixbyt base 7/12] RUN tar -xvf pixlet.tar.gz 0.0s
=> CACHED [pixbyt base 8/12] RUN chmod +x ./pixlet 0.0s
=> CACHED [pixbyt base 9/12] RUN mv pixlet /usr/local/bin/pixlet 0.0s
=> CACHED [pixbyt base 10/12] COPY ./meltano.yml . 0.0s
=> CACHED [pixbyt base 11/12] COPY ./plugins/utilities/ ./plugins/utilities/ 0.0s
=> [pixbyt base 12/12] COPY ./plugins/plugins.meltano.yml ./plugins/plugins.meltano.yml 0.0s
=> ERROR [pixbyt prod-preinstall 1/1] RUN meltano --log-level=debug install 24.5s
------
> [pixbyt prod-preinstall 1/1] RUN meltano --log-level=debug install:
1.264 2024-02-06T15:22:03.013759Z [debug ] meltano 3.3.1, Linux
1.266 2024-02-06T15:22:03.016993Z [debug ] /etc/timezone found, contents:
1.266 Etc/UTC
1.266
1.267 2024-02-06T15:22:03.017489Z [debug ] /etc/localtime found
1.268 2024-02-06T15:22:03.018880Z [debug ] 2 found:
1.268 {'/etc/timezone': 'Etc/UTC', '/etc/localtime is a symlink to': 'Etc/UTC'}
1.270 2024-02-06T15:22:03.021143Z [debug ] Creating DB engine for project at '/project' with DB URI 'sqlite://project/.meltano/meltano.db'
1.425 2024-02-06T15:22:03.175766Z [debug ] Found plugin parent parent=airflow plugin=airflow source=<DefinitionSource.LOCKFILE: 4>
1.425 Installing 4 plugins...
1.428 Installing extractor 'tap-pixlet'...
1.458 2024-02-06T15:22:03.208806Z [debug ] Packages for 'extractors/tap-pixlet' have changed so performing a clean install.
1.459 2024-02-06T15:22:03.209675Z [debug ] Removed old virtual environment for 'extractors/tap-pixlet'
1.459 2024-02-06T15:22:03.209799Z [debug ] Creating virtual environment for 'extractors/tap-pixlet'
1.468 Installing loader 'target-tidbyt'...
1.484 2024-02-06T15:22:03.234734Z [debug ] Variable '$TIDBYT_TOKEN' is not set in the provided env dictionary.
1.485 2024-02-06T15:22:03.235427Z [debug ] Variable '$TIDBYT_DEVICE_ID' is not set in the provided env dictionary.
1.485 2024-02-06T15:22:03.236062Z [debug ] Variable '$TIDBYT_TOKEN' is not set in the provided env dictionary.
1.486 2024-02-06T15:22:03.236568Z [debug ] Variable '$TIDBYT_DEVICE_ID' is not set in the provided env dictionary.
1.488 2024-02-06T15:22:03.238575Z [debug ] Packages for 'loaders/target-tidbyt' have changed so performing a clean install.
1.488 2024-02-06T15:22:03.238805Z [debug ] Removed old virtual environment for 'loaders/target-tidbyt'
1.488 2024-02-06T15:22:03.238957Z [debug ] Creating virtual environment for 'loaders/target-tidbyt'
1.509 Installing loader 'target-webp'...
1.528 2024-02-06T15:22:03.278793Z [debug ] Packages for 'loaders/target-webp' have changed so performing a clean install.
1.529 2024-02-06T15:22:03.279771Z [debug ] Removed old virtual environment for 'loaders/target-webp'
1.530 2024-02-06T15:22:03.280051Z [debug ] Creating virtual environment for 'loaders/target-webp'
1.536 Installing utility 'airflow'...
1.576 2024-02-06T15:22:03.326661Z [debug ] Packages for 'utilities/airflow' have changed so performing a clean install.
1.577 2024-02-06T15:22:03.327244Z [debug ] Removed old virtual environment for 'utilities/airflow'
1.577 2024-02-06T15:22:03.327374Z [debug ] Creating virtual environment for 'utilities/airflow'
2.505 2024-02-06T15:22:04.255025Z [debug ] Upgrading pip for 'extractors/tap-pixlet'
2.505 2024-02-06T15:22:04.255550Z [debug ] Upgrading with args '--upgrade pip' in existing virtual environment for 'extractors/tap-pixlet'
2.530 2024-02-06T15:22:04.280599Z [debug ] Upgrading pip for 'loaders/target-webp'
2.530 2024-02-06T15:22:04.280821Z [debug ] Upgrading with args '--upgrade pip' in existing virtual environment for 'loaders/target-webp'
2.542 2024-02-06T15:22:04.292215Z [debug ] Upgrading pip for 'loaders/target-tidbyt'
2.542 2024-02-06T15:22:04.292454Z [debug ] Upgrading with args '--upgrade pip' in existing virtual environment for 'loaders/target-tidbyt'
2.567 2024-02-06T15:22:04.317798Z [debug ] Upgrading pip for 'utilities/airflow'
2.568 2024-02-06T15:22:04.318796Z [debug ] Upgrading with args '--upgrade pip' in existing virtual environment for 'utilities/airflow'
6.820 2024-02-06T15:22:08.569507Z [debug ] Installing with args 'git+https://github.com/DouweM/target-webp.git' into virtual environment for 'loaders/target-webp'
6.899 2024-02-06T15:22:08.648245Z [debug ] Installing with args 'git+https://github.com/DouweM/tap-pixlet.git' into virtual environment for 'extractors/tap-pixlet'
6.939 2024-02-06T15:22:08.689701Z [debug ] Installing with args 'git+https://github.com/meltano/airflow-ext.git@811287a6c2e7dc74498c26ef5f830e9df78dc976 apache-airflow==2.7.0 psycopg2-binary --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.3.4/constraints-no-providers-3.10.txt' into virtual environment for 'utilities/airflow'
7.193 2024-02-06T15:22:08.943683Z [debug ] Installing with args 'git+https://github.com/DouweM/target-tidbyt.git' into virtual environment for 'loaders/target-tidbyt'
12.80 2024-02-06T15:22:14.548413Z [info ] Logged pip install output to /project/.meltano/logs/pip/utilities/airflow/pip.log
12.80 Utility 'airflow' could not be installed: Failed to install plugin 'airflow'.
12.80 Collecting git+https://github.com/meltano/airflow-ext.git@811287a6c2e7dc74498c26ef5f830e9df78dc976
12.80 Cloning https://github.com/meltano/airflow-ext.git (to revision 811287a6c2e7dc74498c26ef5f830e9df78dc976) to /tmp/pip-req-build-fl0zd8lh
12.80 Running command git clone --filter=blob:none --quiet https://github.com/meltano/airflow-ext.git /tmp/pip-req-build-fl0zd8lh
12.80 Running command git rev-parse -q --verify 'sha^811287a6c2e7dc74498c26ef5f830e9df78dc976'
12.80 Running command git fetch -q https://github.com/meltano/airflow-ext.git 811287a6c2e7dc74498c26ef5f830e9df78dc976
12.80 Running command git checkout -q 811287a6c2e7dc74498c26ef5f830e9df78dc976
12.80 Resolved https://github.com/meltano/airflow-ext.git to commit 811287a6c2e7dc74498c26ef5f830e9df78dc976
12.80 Installing build dependencies: started
12.80 Installing build dependencies: finished with status 'done'
12.80 Getting requirements to build wheel: started
12.80 Getting requirements to build wheel: finished with status 'done'
12.80 Preparing metadata (pyproject.toml): started
12.80 Preparing metadata (pyproject.toml): finished with status 'done'
12.80 ERROR: Cannot install apache-airflow==2.7.0 because these package versions have conflicting dependencies.
12.80
12.80 The conflict is caused by:
12.80 The user requested apache-airflow==2.7.0
12.80 The user requested (constraint) apache-airflow==2.3.4
12.80
12.80 To fix this you could try to:
12.80 1. loosen the range of package versions you've specified
12.80 2. remove package versions to allow pip attempt to solve the dependency conflict
12.80
12.80 ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
12.80
22.62 Installed loader 'target-webp'
22.86 Installed extractor 'tap-pixlet'
23.32 Installed loader 'target-tidbyt'
23.32 Installed 3/4 plugins
24.00 2024-02-06T15:22:25.075773Z [debug ] Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
24.00 join our friendly Slack community.
24.00
24.00 Failed to install plugin(s)
24.00 ╭───────────────────── Traceback (most recent call last) ──────────────────────╮
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/__init__.py:102 in _run_cli │
24.00 │ │
24.00 │ 99 │ """ │
24.00 │ 100 │ try: │
24.00 │ 101 │ │ try: # noqa: WPS225, WPS505 │
24.00 │ ❱ 102 │ │ │ cli(obj={"project": None}) │
24.00 │ 103 │ │ except ProjectReadonly as err: │
24.00 │ 104 │ │ │ raise CliError( │
24.00 │ 105 │ │ │ │ f"The requested action could not be completed: {err}", │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/core.py:1157 in __call__ │
24.00 │ │
24.00 │ 1154 │ │
24.00 │ 1155 │ def __call__(self, *args: t.Any, **kwargs: t.Any) -> t.Any: │
24.00 │ 1156 │ │ """Alias for :meth:`main`.""" │
24.00 │ ❱ 1157 │ │ return self.main(*args, **kwargs) │
24.00 │ 1158 │
24.00 │ 1159 │
24.00 │ 1160 class Command(BaseCommand): │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/cli.py:46 in main │
24.00 │ │
24.00 │ 43 │ │ │ args: Positional arguments for the Click group. │
24.00 │ 44 │ │ │ kwargs: Keyword arguments for the Click group. │
24.00 │ 45 │ │ """ │
24.00 │ ❱ 46 │ │ return super().main(*args, windows_expand_args=False, **kwargs │
24.00 │ 47 │
24.00 │ 48 │
24.00 │ 49 @click.group( │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/core.py:1078 in main │
24.00 │ │
24.00 │ 1075 │ │ try: │
24.00 │ 1076 │ │ │ try: │
24.00 │ 1077 │ │ │ │ with self.make_context(prog_name, args, **extra) as c │
24.00 │ ❱ 1078 │ │ │ │ │ rv = self.invoke(ctx) │
24.00 │ 1079 │ │ │ │ │ if not standalone_mode: │
24.00 │ 1080 │ │ │ │ │ │ return rv │
24.00 │ 1081 │ │ │ │ │ # it's not safe to `ctx.exit(rv)` here! │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/utils.py:641 in invoke │
24.00 │ │
24.00 │ 638 │ │ enact_environment_behavior(self.environment_behavior, ctx) │
24.00 │ 639 │ │ if ctx.obj.get("tracker"): │
24.00 │ 640 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_cont │
24.00 │ ❱ 641 │ │ super().invoke(ctx) │
24.00 │ 642 │
24.00 │ 643 │
24.00 │ 644 class InstrumentedDefaultGroup(InstrumentedGroupMixin, DefaultGroup, D │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/core.py:1688 in invoke │
24.00 │ │
24.00 │ 1685 │ │ │ │ super().invoke(ctx) │
24.00 │ 1686 │ │ │ │ sub_ctx = cmd.make_context(cmd_name, args, parent=ctx │
24.00 │ 1687 │ │ │ │ with sub_ctx: │
24.00 │ ❱ 1688 │ │ │ │ │ return _process_result(sub_ctx.command.invoke(sub │
24.00 │ 1689 │ │ │
24.00 │ 1690 │ │ # In chain mode we create the contexts step by step, but afte │
24.00 │ 1691 │ │ # base command has been invoked. Because at that point we do │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/utils.py:690 in invoke │
24.00 │ │
24.00 │ 687 │ │ if ctx.obj.get("tracker"): │
24.00 │ 688 │ │ │ ctx.obj["tracker"].add_contexts(CliContext.from_click_cont │
24.00 │ 689 │ │ │ ctx.obj["tracker"].track_command_event(CliEvent.started) │
24.00 │ ❱ 690 │ │ super().invoke(ctx) │
24.00 │ 691 │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/core.py:1434 in invoke │
24.00 │ │
24.00 │ 1431 │ │ │ echo(style(message, fg="red"), err=True) │
24.00 │ 1432 │ │ │
24.00 │ 1433 │ │ if self.callback is not None: │
24.00 │ ❱ 1434 │ │ │ return ctx.invoke(self.callback, **ctx.params) │
24.00 │ 1435 │ │
24.00 │ 1436 │ def shell_complete(self, ctx: Context, incomplete: str) -> t.List │
24.00 │ 1437 │ │ """Return a list of completions for the incomplete value. Loo │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/core.py:783 in invoke │
24.00 │ │
24.00 │ 780 │ │ │
24.00 │ 781 │ │ with augment_usage_errors(__self): │
24.00 │ 782 │ │ │ with ctx: │
24.00 │ ❱ 783 │ │ │ │ return __callback(*args, **kwargs) │
24.00 │ 784 │ │
24.00 │ 785 │ def forward( │
24.00 │ 786 │ │ __self, __cmd: "Command", *args: t.Any, **kwargs: t.Any # no │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/click/decorators.py:33 in new_func │
24.00 │ │
24.00 │ 30 │ """ │
24.00 │ 31 │ │
24.00 │ 32 │ def new_func(*args: "P.args", **kwargs: "P.kwargs") -> "R": │
24.00 │ ❱ 33 │ │ return f(get_current_context(), *args, **kwargs) │
24.00 │ 34 │ │
24.00 │ 35 │ return update_wrapper(new_func, f) │
24.00 │ 36 │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/params.py:27 in decorate │
24.00 │ │
24.00 │ 24 │ │ if database_uri: │
24.00 │ 25 │ │ │ ProjectSettingsService.config_override["database_uri"] = da │
24.00 │ 26 │ │ │
24.00 │ ❱ 27 │ │ return func(*args, **kwargs) │
24.00 │ 28 │ │
24.00 │ 29 │ return functools.update_wrapper(decorate, func) │
24.00 │ 30 │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/params.py:75 in decorate │
24.00 │ │
24.00 │ 72 │ │ │ │ except MigrationError as err: │
24.00 │ 73 │ │ │ │ │ raise CliError(str(err)) from err │
24.00 │ 74 │ │ │ │
24.00 │ ❱ 75 │ │ │ func(project, *args, **kwargs) │
24.00 │ 76 │ │ │
24.00 │ 77 │ │ return functools.update_wrapper(decorate, func) │
24.00 │ 78 │
24.00 │ │
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/install.py:110 in install │
24.00 │ │
24.00 │ 107 │ ) │
24.00 │ 108 │ if not success: │
24.00 │ 109 │ │ tracker.track_command_event(CliEvent.failed) │
24.00 │ ❱ 110 │ │ raise CliError("Failed to install plugin(s)") │
24.00 │ 111 │ tracker.track_command_event(CliEvent.completed) │
24.00 │ 112 │
24.00 │ 113 │
24.00 ╰──────────────────────────────────────────────────────────────────────────────╯
24.00 CliError: Failed to install plugin(s)
24.00
24.00 The above exception was the direct cause of the following exception:
24.00
24.00 ╭───────────────────── Traceback (most recent call last) ──────────────────────╮
24.00 │ /venv/lib/python3.9/site-packages/meltano/cli/__init__.py:112 in _run_cli │
24.00 │ │
24.00 │ 109 │ │ except MeltanoError as err: │
24.00 │ 110 │ │ │ handle_meltano_error(err) │
24.00 │ 111 │ │ except Exception as err: │
24.00 │ ❱ 112 │ │ │ raise CliError(f"{troubleshooting_message}\n{err}") from e │
24.00 │ 113 │ except CliError as cli_error: │
24.00 │ 114 │ │ cli_error.print() │
24.00 │ 115 │ │ sys.exit(1) │
24.00 ╰──────────────────────────────────────────────────────────────────────────────╯
24.00 CliError: Need help fixing this problem? Visit http://melta.no/ for
24.00 troubleshooting steps, or to
24.00 join our friendly Slack community.
24.00
24.00 Failed to install plugin(s)
24.00
24.00 Need help fixing this problem? Visit http://melta.no/ for troubleshooting steps, or to
24.00 join our friendly Slack community.
24.00
24.00 Failed to install plugin(s)
------
failed to solve: process "/bin/sh -c meltano --log-level=debug install" did not complete successfully: exit code: 1
░▒▓ ~/Pr/pixbyt on main !1 ?1 ▓▒░
DouweM
bboatwright
The airflow-ext package is specified to be
main, which is a branch that has changed sinceplugins.meltano.ymlwas authored. When I build today, I get constraints errors:To Reproduce:
ERROR: Cannot install airflow-ext and apache-airflow==2.7.0 because these package versions have conflicting dependencies.