Turbo Error Trying to Read MariaDB Directory

[hatemjaber]

Verify canary release

• [X] I verified that the issue exists in the latest Turborepo canary release.

Link to code that reproduces this issue

pnpm install turbo -g

What package manager are you using / does the bug impact?

pnpm

What operating system are you using?

Linux

Which canary version will you have in your reproduction?

1.10.17+

Describe the Bug

I decided to update to the latest version of turbo on my laptop running the latest versions of node, npm, pnpm... I have a script that starts up docker containers for mariadb, redis, redisinsight, adminer, and manticoresearch along with running the turbo run dev:WHATEVER command to start a specific stack. This has been working fine for a long time running turbo@1.10.16 with no issues; everything starts as expected. The latest version throws a permissions denied error trying to read my .docker/mariadb/db-name/db_data directory. I'm not sure why it's even trying to read that directory since it never did before running version 1.10.16. I tried every version from 1.10.17 to the latest canary and they all experience the same issue throwing the permission denied error for the mariadb directory.

I don 't know why it's trying to read the directory that I use to persist the database data. I switched to using docker volumes and the issue went away but I prefer to have my data persisted to the host in development. I switched back to 1.10.16 for now since it works but thought it was worth bringing to your attention.

Expected Behavior

I expect to run pnpm admin and the script that runs docker, turbo, etc... should just run without getting a permission denied error. I expect for turbo to only worry about the directories that actually contain apps or packages. I scanned the documentation for a config that I can tell turbo to ignore certain directories but couldn't find that.

To Reproduce

You would need a monorepo with a directory to hold the mysql/mariadb database and use docker to start up. Here is the script that I use to run the entire project:

#!/bin/bash

# Initialize a variable to determine whether to capture the PID or not
capture_pid=false

# Parse command-line options for start/stop and PID capture
action="start"  # Default action is to start the application

while getopts ":ps" opt; do
  case $opt in
    p)
      capture_pid=true  # Set capture_pid to true if -p flag is present
      ;;
    s)
      action="stop"  # Set action to stop if -s flag is present
      ;;
    \?)
      echo "Invalid option: -$OPTARG" >&2
      exit 1
      ;;
  esac
done
shift $((OPTIND-1))  # Shift off the options and optional --.

# Function to run the dev server
run_dev() {
  echo "Starting the dev server..."
  if [ "$capture_pid" = true ]; then
    turbo run dev:${FILE_NAME} &
    echo $! > turbo.pid
    echo "Captured PID in turbo.pid"
  else
    turbo run dev:${FILE_NAME}
  fi
}

# Function to clean up containers, networks, and volumes
cleanup() {
  echo "Cleaning up containers, networks, and volumes..."
  docker-compose -f .docker/docker-compose.yml down
  if [ -f turbo.pid ]; then
    kill $(cat turbo.pid) && rm turbo.pid
    echo "Killed turbo process and removed PID file."
  fi
}

# Function to start the application
start_app() {
  # Check for file name argument
  if [ -z "$1" ]; then
    echo "Please provide an argument for the file name."
    exit 1
  fi

  FILE_NAME=$1

  # Add domains to hosts file
  source ./.scripts/functions.sh
  echo "Adding domains to hosts file..."
  add_domain_to_hosts dashboard.cfapp.local
  add_domain_to_hosts account.cfapp.local
  sleep 1

  # Start the containers in the foreground
  docker-compose -f .docker/docker-compose.yml up -d

  # Run the dev server after the docker-compose up command completes
  run_dev
}

# Trap the SIGINT signal (Ctrl+C) only when starting the application
trap cleanup SIGINT

# Main script logic
case $action in
  start)
    start_app $1
    ;;
  stop)
    cleanup
    ;;
  *)
    echo "Invalid action. Use -p for PID capture and -s for stopping the application."
    ;;
esac

# Note: The cleanup function will be called automatically when Ctrl+C is pressed.

Additional context

I tested from 1.10.17 to the recent canary version and face the same issue. The last stable version that allows me to run my stack is 1.10.16

[chris-olszewski]

I'm guessing that we're trying to read that file as an input for a task. It sounds like there was an unintentional behavior change with what files we consider for task inputs. Could you include your turbo.json?

[hatemjaber]

@chris-olszewski here it is. it's been the same since starting out with turbo

{
  "$schema": "https://turbo.build/schema.json",
  "globalDependencies": [
    "**/.env.*local"
  ],
  "pipeline": {
    "build": {
      "dependsOn": [
        "^build"
      ],
      "outputs": [
        "build/**",
        "dist/**",
        ".next/**",
        ".svelte-kit/**",
        "package/**"
      ]
    },
    "dev": {
      "cache": false,
      "dependsOn": [
        "^build"
      ]
    },
    "dev:admin": {
      "cache": false,
      "dependsOn": [
        "^build"
      ]
    },
    "dev:account": {
      "cache": false,
      "dependsOn": [
        "^build"
      ]
    },
    "logs:dev": {
      "cache": false,
      "dependsOn": [
        "^build"
      ]
    },
    "clean": {
      "cache": false
    }
  }
}

[chukwumaijem]

I'm having the same issue. I was on "turbo": "1.10.16" and everything worked well. Just updated the project to "turbo": "^1.11.3" and started getting the error.

Turbo is trying to read postgres docker volume in the project.

ERROR  run failed: IO Error: failed to match directory tree: failed to read file at `Some(".../docker/db/data")`: Permission denied (os error 13)

[hatemjaber]

I'm having the same issue. I was on "turbo": "1.10.16" and everything worked well. Just updated the project to "turbo": "^1.11.3" and started getting the error.

Turbo is trying to read postgres docker volume in the project.

ERROR  run failed: IO Error: failed to match directory tree: failed to read file at `Some(".../docker/db/data")`: Permission denied (os error 13)

I tested all the versions from 1.10.17 to canary and they all seem to be doing the same thing. I even played around with permissions etc... but that didn't help or do anything to fix the problem. Over the weekend I tested this on a Mac M2 and it seems to be the same issue.

[chukwumaijem]

There's a way to get around this. You can set the permission on the compose file. Essentially making the logged in user the owner of the volumn. This worked for me.

version: '3.9'

services:
  database:
    image: postgres:14-alpine
    user: ${USER_ID}:${GROUP_ID}
    ports:
      - 5432:5432
    volumes:
      - ./db/data:/var/lib/postgresql/data
    env_file:
      - ./db/.env

The important line here is user: ${USER_ID}:${GROUP_ID}. Then you can start docker with USER_ID=$(id -u) GROUP_ID=$(id -g) docker compose up

I also have this script to help me run my app.

    "start:compose": "cd docker && USER_ID=$(id -u) GROUP_ID=$(id -g) docker compose up",

@hatemjaber

[hatemjaber]

I went down that path at some point but opted to stay on 1.10.16 until they offer a fix for it. I'd rather turbo do what it is designed for and not interfere with the rest of the project. I would like to be on the latest version and hope they come up with a solution for this. Thank you @chukwumaijem for sharing that config!

[chris-olszewski]

Hi all, just got a chance to dig into this and https://github.com/vercel/turbo/pull/6957 should fix it. Pre 1.11.0 we were ignoring any permission issues and this gets us back to that behavior.

[hatemjaber]

@chris-olszewski apologies in advance for asking, but will this be fixed in all future versions as well?

[hatemjaber]

@chris-olszewski I tried the latest version 1.11.3 and it worked on my desktop running an AMD processor (not sure if that helps) and on Macbook Pro M2 (colima to run docker everything installed with homebrew...). On my other system where the problem originally started, I cannot run anything other than 1.10.16 for whatever reason. I will remove docker, node, etc... and start with a clean setup to see if that helps. That system has an 11th gen intel processor if that makes a difference.

It's puzzling that it works on the other two systems and doesn't work on the Intel machine. I know you are moving the code base from Go to Rust, I don't think any of that has to do with system specifics or hardware. If you have any suggestions please share, and thank you for taking the time to look into this!

[hatemjaber]

@chris-olszewski I know you marked this closed but I'm still experiencing this issue on a laptop that I can only run 1.10.16. I updated to the latest LTS node and npm. I installed pnpm fresh. I installed turbo the latest version and still getting the error on the directory that it should not be reading. Is there something cached somewhere that I might have missed? I'm running it and it's fine, but I would like to know the reason why I can't upgrade to the latest on this machine. Is there anything you can suggest for me to try to get this resolved?

[hatemjaber]

@chris-olszewski for whatever reason, the system that was working fine with the latest version of 1.11.3 is not working and I resorted to 1.10.16 which is the last working version. Can you please offer some guidance on this or have someone look into it??? If this is not something your team is interested in fixing at the time please let me know so I can figure out another solution. I am worried that the version that is currently working might stop working at some point and I need to be able to figure out what to do in a situation like that.

[hatemjaber]

@Zertsov I noticed you pushed the fix for the reported issue. can you please look into why this is happening? It all of a sudden stopped working and I could never get it to work on my other system. I have no clue why it just stopped, I didn't make any changes to this system or update/upgrade turbo. I was on 1.11.3 and now back to 1.10.16 which is the only version that I know works that is recent.

[hatemjaber]

@Zertsov and @chris-olszewski sorry for spamming this issue... I just checked and see that there is a new version. this version 1.12.0 seems to be working fine. what are the chances that it will break like 1.11.3 and is there anything that I can do to prevent this from happening? do we know the source of the issue? not sure if it helps, but I'm running the latest linux kernel on my system and the latest docker/docker-compose. Thank you and my apologies for spamming the chat.