search

dbt-labs / dbt-core

dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
https://getdbt.com
Apache License 2.0
9.52k stars 1.58k forks source link

[CT-2727] [Bug] dbt docs serve edits the manifest.json #7922

Closed Nisond closed 1 year ago

Nisond commented 1 year ago

Is this a new bug in dbt-core?

Current Behavior

When I run dbt docs serve --port 8000 it edits the manifest.json, the index.html, and the partial_parse.msgpack in the target folder.

I have verified that this edit happens by looking at the last modified timestamp on the file as well as by manually making edits to the json which then are reverted upon running docs serve.

Expected Behavior

dbt docs serve --port 8000 to not edit any files in the target folder

Steps To Reproduce

Unclear if other are able to replicate this. I'm using dbt-core=1.5.0 and dbt-bigquery=1.5.0 on linux.

These are steps I can take to replicate

  1. opened a terminal
  2. activate the virtual environment I use for dbt, containing dbt-core and dbt-bigquery
  3. used cd to navigate to the dbt project folder
  4. ran ‘dbt docs serve --port 8000’

I have a single terminal open on the local machine and am not hosting documentation. I have tried restarting the machine first.

Relevant log output

N/A

Environment

- OS: Debian 4.19.0-24-cloud-amd64
- Python: 3.7.12
- dbt:1.5.0

Which database adapter are you using with dbt?

bigquery

Additional Context

See here for original issue and discussion.

dbeatty10 commented 1 year ago

Thank you for opening this issue @Nisond !

I read through the discussion in Discourse that you linked -- thank you for including that helpful context. 🧠

It sounds like your concrete goal in this instance is to do post-processing of manifest.json and add hyperlinks to model descriptions. While trying to accomplish that, it sounds like you were surprised that dbt docs serve doesn't just use the manifest.json as-is.

Follow-up questions

Could you give an example of a model description that you'd be adding a hyperlink to?

Also, what would be the barriers to adding those hyperlinks to the descriptions directly like below?

Before

models/_models.yml

models:
  - name: legendary_creatures
    description: This description doesn't have any hyperlinks

After

models/_models.yml

models:
  - name: legendary_creatures
    description: >
      [`Legendary_creatures`](https://en.wikipedia.org/wiki/Category:Legendary_creatures) is a category on Wikipedia.
      [Deadpool](https://en.wikipedia.org/wiki/Deadpool) was listed in the top-10 as-of 2023-06-22 ([ref](https://en.wikipedia.org/wiki/Wikipedia:Popular_pages#Legendary_Creatures)).
joellabes commented 1 year ago

I think this might be resolved by https://github.com/dbt-labs/dbt-core/pull/7554?

dbeatty10 commented 1 year ago

@Nisond we believe this was resolved by https://github.com/dbt-labs/dbt-core/pull/7554 and available in dbt-core 1.5.1 and onwards.

Could you give that a shot and let us know if you run into any problems?