search

airbytehq / airbyte

The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
https://airbyte.com
Other
14.72k stars 3.78k forks source link

airbyte-ci fails to run #39466

Open kikoncuo opened 2 weeks ago

kikoncuo commented 2 weeks ago

Topic

airbyte-ci crashes when running any command because it uses itself and makes itself busy

Relevant information

I did a clean install of airbyte-ci using the docs from: https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md Running on a clean Ubuntu 20.04 on Hetzner using ssh

When running any command or trying to compile anything I get error OSError: [Errno 26] Text file busy: '/home/ubuntu/.local/bin/airbyte-ci'

Here is the full error stack:

ubuntu@Airbyte:~/airbyte$ airbyte-ci update
[17:03:38] INFO     root: Setting working directory to /home/ubuntu/airbyte                                                                                           ensure_repo_root.py:58[17:03:39] INFO     root: Updating to the latest version of airbyte-ci...                                                                                                     commands.py:22Linux based system detected.
Downloading from https://connectors.airbyte.com/files/airbyte-ci/releases/ubuntu/latest/airbyte-ci
Traceback (most recent call last):
  File "airbyte_ci.py", line 217, in <module>
  File "asyncclick/core.py", line 1205, in __call__
  File "anyio/_core/_eventloop.py", line 68, in run
  File "anyio/_backends/_asyncio.py", line 204, in run
  File "asyncio/runners.py", line 44, in run
  File "asyncio/base_events.py", line 649, in run_until_complete
  File "anyio/_backends/_asyncio.py", line 199, in wrapper
  File "asyncclick/core.py", line 1208, in _main
  File "asyncclick/core.py", line 1120, in main
  File "asyncclick/core.py", line 1739, in invoke
  File "asyncclick/core.py", line 1485, in invoke
  File "asyncclick/core.py", line 824, in invoke
  File "pipelines/airbyte_ci/update/commands.py", line 23, in update
    install_airbyte_ci_binary(version)
  File "pipelines/external_scripts/airbyte_ci_install.py", line 109, in main
    shutil.copy(tmp_file, destination_path)
  File "shutil.py", line 417, in copy
  File "shutil.py", line 256, in copyfile
OSError: [Errno 26] Text file busy: '/home/ubuntu/.local/bin/airbyte-ci'
[104899] Failed to execute script 'airbyte_ci' due to unhandled exception!
kikoncuo commented 2 weeks ago

No other service is using that binary, a temp solution I found is to clone the folder that contains the executable and run the clone from the root directory

marcosmarxm commented 2 weeks ago

@alafanechere do you know if airbyte-ci runs smoothly in Linux?

kikoncuo commented 2 weeks ago

I tried this on a Hetzner instance running Ubuntu 20.04

alafanechere commented 6 days ago

airbyte-ci should run smoothly on linux. @kikoncuo do you still have an error if you use airbyte-ci --disable-update-check --disable-auto-update?

Here's a pointer on how we install airbyte-ci in our Github action CI, which is running ubuntu, it might help: https://github.com/airbytehq/airbyte/master/main/.github/actions/install-airbyte-ci/action.yml#L45