search

dbt-labs / hubcap

This app adds modules to the hubsite at hub.getdbt.com
13 stars 100 forks source link

Malformed directory structure should be handled gracefully instead of crashing hubcap #312

Open joellabes opened 6 months ago

joellabes commented 6 months ago

In https://github.com/dbt-labs/hubcap/pull/289, https://github.com/BfdCampos/source_db can't be added to the hub because it kept crashing hubcap.

I ran it locally and got this error: 

2024-03-20 15:56:00 INFO     checking out branch bump-BfdCampos-source_db-1710902409 in the hub repo
2024-03-20 15:56:00 INFO     writing index.json to /Users/joel/Documents/GitHub/hubcap/target/hub.getdbt.com-test/data/packages/BfdCampos/source_db/index.json
2024-03-20 15:56:00 INFO     
Traceback (most recent call last):
  File "/Users/joel/Documents/GitHub/hubcap/hubcap.py", line 75, in <module>
    new_branches = package.commit_version_updates_to_hub(
  File "/Users/joel/Documents/GitHub/hubcap/hubcap/package.py", line 130, in commit_version_updates_to_hub
    branch_name, org_name, package_name = task.run(hub_dir_path, pr_strategy)
  File "/Users/joel/Documents/GitHub/hubcap/hubcap/records.py", line 121, in run
    require_dbt_version = package.parse_require_dbt_version(Path(os.getcwd()))
  File "/Users/joel/Documents/GitHub/hubcap/hubcap/package.py", line 34, in parse_require_dbt_version
    with open(repo_dir / Path("dbt_project.yml"), "r") as stream:
FileNotFoundError: [Errno 2] No such file or directory: '/Users/joel/Documents/GitHub/hubcap/target/BfdCampos_source_db/dbt_project.yml'

It turns out that the 0.1.0 release had the package inside of a subdirectory so dbt_project.yml couldn't be found.

Instead of crashing, this should be handled gracefully (it caused an outage where no other releases could be detected by anyone else).

dbeatty10 commented 6 months ago

Seems similar to https://github.com/dbt-labs/hubcap/issues/269 but distinct from it.