Multiple Projects, Unable to Locate Files

[hc1wwilger]

I have a singular dbt repo with multiple projects within. I have set the DBT_PROJECT_DIR and DBT_PROFILE_DIR environment variables both in terminal and in the dbt plugin settings. The issue is that the venv is not in the same directory as the project. We run a single venv in the root of the repo, with each project in a folder/subfolder structure also in the root. Is there a way to make this work?

When I try to compile SQL I am met with:

  Database Error in model ingest_result_flattened (models/ingest/ingest_result_flattened.sql)
    [Errno 2] No such file or directory: '*****'
Exception ignored in: <function Pool.__del__ at 0x106ae2700>
Traceback (most recent call last):
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/pool.py", line 268, in __del__
    self._change_notifier.put(None)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/queues.py", line 378, in put
    self._writer.send_bytes(obj)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 205, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 416, in _send_bytes
    self._send(header + buf)
  File "/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/multiprocessing/connection.py", line 373, in _send
    n = write(self._handle, buf)
OSError: [Errno 9] Bad file descriptor

[ramonvermeulen]

Hi @hc1wwilger,

Thanks for reporting this issue. I've never tested the plugin with multiple dbt projects sharing one venv, when I have time I will see if there is anything I can do to make it work. Would be nice to receive a little bit more context about your directory structure so that I might be able to reproduce the issue. For example do all these dbt projecsts share the same profiles.yml?

I assume you have something like:

venv: ~/projects/my-repository/.venv profiles: ~/projects/my-repository/profiles.yml dbt-project-a: ~/projects/my-repository/dbt-project-a dbt-project-b: ~/projects/my-repository/dbt-project-b dbt-project-c: ~/projects/my-repository/dbt-project-c

And then you open your IDE with ~/projects/my-repository as "project" folder, and select ~/projects/my-repository/.venv as active venv?

Some things you can try to solve the issue for the time being (please let me know if one of these works):

• Simply open the IDE directly with one of the dbt project directories as project root, then manually select the venv from the parent directory
• Ensure the dbt project root and target directory settings are set to the dbt project instead of the parent directory

Also could you try to copy the command that is ran by the plugin from the console window (the line that is logged with >>> <command>, take the command and see what happens if it runs manually in your terminal (try both from parent directory as dbt project directory, trying to figure out if it is a working dir related issue).