search

microsoft / azure-pipelines-tasks

Tasks for Azure Pipelines
https://aka.ms/tfbuild
MIT License
3.49k stars 2.61k forks source link

The Bash@3 task ignores the `target: container` step setting. #16190

Closed gprossliner closed 1 year ago

gprossliner commented 2 years ago

Required Information

Question, Bug, or Feature?
Type: Bug

Enter Task Name: Bash@3,

Environment

Issue Description

This is an example of the pipeline, showing the issue:

resources:
  containers:
  - container: sqlserver
    image: mcr.microsoft.com/mssql/server:2019-latest
    env:
      SA_PASSWORD: supersecure1!
      ACCEPT_EULA: Y
    ports:
    - 1433/tcp
jobs:
- job: Tests
  services:
    sqlserver: sqlserver
  steps:
  - task: Bash@3
    target: sqlserver    ## here i specify the container target
    inputs:
      targetType: inline
      script: |

        for i in {0..10..1}; do
            /opt/mssql-tools/bin/sqlcmd -U sa -Psupersecure1! -Q"SELECT @@SERVERNAME"
            if [ $? -eq 0 ]; then
                exit 0
            fi
            sleep 1
        done

        echo Giving up after 10 retries, but it can work ...

The Bash@3 task doesn't execute in the target: sqlserver container, but in the host. It is not ignored completely. If I pass in an invalid name like sqlserver_error, an error is returned (A container resource with name sqlserver_error could not be found. The container resource does not exist...)

Checking the logs, there is no indication that the task actually tries to execute anything in the container. I expected the script to be execute as docker exec ID ....

As you see within the logs, the error /opt/mssql-tools/bin/sqlcmd: No such file or directory occurs. There is no sqlcmd on the agent.

Task logs

##[debug]Evaluating condition for step: 'Wait from sqlserver to accept connections'
##[debug]Evaluating: SucceededNode()
##[debug]Evaluating SucceededNode:
##[debug]=> True
##[debug]Result: True
Starting: Wait from sqlserver to accept connections
==============================================================================
Task         : Bash
Description  : Run a Bash script on macOS, Linux, or Windows
Version      : 3.163.3
Author       : Microsoft Corporation
Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/bash
==============================================================================
##[debug]agent.TempDirectory=/azp/agent/_work/_temp
##[debug]loading inputs and endpoints
##[debug]loading INPUT_TARGETTYPE
##[debug]loading INPUT_FILEPATH
##[debug]loading INPUT_SCRIPT
##[debug]loading INPUT_WORKINGDIRECTORY
##[debug]loading INPUT_FAILONSTDERR
##[debug]loading INPUT_NOPROFILE
##[debug]loading INPUT_NORC
##[debug]loading ENDPOINT_AUTH_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_SCHEME_SYSTEMVSSCONNECTION
##[debug]loading ENDPOINT_AUTH_PARAMETER_SYSTEMVSSCONNECTION_ACCESSTOKEN
##[debug]loading SECRET_SYSTEM_ACCESSTOKEN
##[debug]loaded 11
##[debug]Agent.ProxyUrl=undefined
##[debug]Agent.CAInfo=undefined
##[debug]Agent.ClientCert=undefined
##[debug]Agent.SkipCertValidation=undefined
##[debug]noProfile=true
##[debug]noRc=true
##[debug]check path : /azp/agent/_work/_tasks/Bash_6c731c3c-3c68-459a-a5c9-bde6e6595b5b/3.163.3/task.json
##[debug]adding resource file: /azp/agent/_work/_tasks/Bash_6c731c3c-3c68-459a-a5c9-bde6e6595b5b/3.163.3/task.json
##[debug]system.culture=en-US
##[debug]failOnStderr=false
##[debug]workingDirectory=/azp/agent/_work/2/s
##[debug]check path : /azp/agent/_work/2/s
##[debug]targetType=inline
##[debug]script=for i in {0..10..1}; do
    /opt/mssql-tools/bin/sqlcmd -U sa -Psupersecure1! -Q"SELECT @@SERVERNAME"
    if [ $? -eq 0 ]; then
        exit 0
    fi
    sleep 1
done

echo Giving up after 10 retries, but it can work ...
Generating script.
##[debug]which 'bash'
##[debug]found: '/bin/bash'
##[debug]Agent.Version=2.170.1
##[debug]agent.tempDirectory=/azp/agent/_work/_temp
##[debug]check path : /azp/agent/_work/_temp
========================== Starting Command Output ===========================
##[debug]which '/bin/bash'
##[debug]found: '/bin/bash'
##[debug]/bin/bash arg: --noprofile
##[debug]/bin/bash arg: --norc
##[debug]/bin/bash arg: /azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh
##[debug]exec tool: /bin/bash
##[debug]arguments:
##[debug]   --noprofile
##[debug]   --norc
##[debug]   /azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh
/bin/bash --noprofile --norc /azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
/azp/agent/_work/_temp/0ff0d0a0-acc1-44e1-b399-4fb361dff64b.sh: line 2: /opt/mssql-tools/bin/sqlcmd: No such file or directory
Giving up after 10 retries, but it can work ...

##[debug]Exit code 0 received from tool '/bin/bash'
##[debug]STDIO streams have closed for tool '/bin/bash'
##[debug]task result: Succeeded
##[debug]Processed: ##vso[task.complete result=Succeeded;done=true;]
Finishing: Wait from sqlserver to accept connections

Can you tell what's wrong here? Or am I doing something wrong?

Thank you!

Kozlov-Igor commented 2 years ago

Hi @gprossliner thanks for reporting! We are working on more prioritized issues at the moment, but will get back to this one soon.

gprossliner commented 2 years ago

Hi @Kozlov-Igor. thank you for the answer.

This is the workaround:

  - script: |
        CID=$(echo "$AGENT_CONTAINERMAPPING" | jq -r ".sqlserver.id")

        for i in {0..10..1}; do
            docker exec $CID /opt/mssql-tools/bin/sqlcmd -U sa -Psupersecure1! -Q"SELECT @@SERVERNAME"
            if [ $? -eq 0 ]; then
                exit 0
            fi
            sleep 1
        done

But it would be nice if this could be fixed. Because it's documented to work this way. Please also let me know if you need some help (tests, PR, ...)

vmapetr commented 2 years ago

Hi @gprossliner!

Sorry for the delayed answer. This is not a problem with a task Bash itself, but a combination of issues with insufficient user permissions on the MsSQL container and usage of services alongside containers.

Using services in your template will initialize required containers in non-interactive mode, and further steps will be handled by the host machine instead of the target container. If you want to pass the task directly to the target container you should remove the additional services definition from your template. Please, notice that in this case there will be an issue with the MsSQL container initialization job, due to the specific of the azure agent - to execute tasks directly on the container, the default container user should have root permissions, however on the MsSQL image default user is mssql. You can use options: --user 0:0 in the container definition to avoid this.

Example of the working template:

resources:
  containers:
  - container: sqlserver
    image: mcr.microsoft.com/mssql/server:2019-latest
    options: --user 0:0 
    env:
      SA_PASSWORD: supersecure1!
      ACCEPT_EULA: Y
    ports:
    - 1433/tcp

jobs:
- job: Tests
  steps:
  - task: Bash@3
    target: sqlserver
    inputs:
      targetType: inline
      script: |
        for i in {0..10..1}; do
            /opt/mssql-tools/bin/sqlcmd -U sa -Psupersecure1! -Q"SELECT @@SERVERNAME"
            if [ $? -eq 0 ]; then
                exit 0
            fi
            sleep 1
        done

        echo Giving up after 10 retries, but it can work ...
gprossliner commented 2 years ago

Hi @vmapetr!

Sorry for the delayed response. Because I had the workaround in place, I have not checked the issue again.

Using services in your template will initialize required containers in non-interactive mode, ... you should remove the additional services definition from your template.

If I have no service, would it still be possible to execute queries against the SQL Server? Currently I use it like:

variables:
  connectionstring: UID=sa;SERVER=localhost,$(agent.services.sqlserver.ports.1433);PWD=supersecure1!;

- task: DotNetCoreCLI@2
  displayName: Run Tests
  inputs:
    command: 'test'
    projects: '**/*.Test.csproj'
    arguments: -c Release -e TESTS_CONNECTION_STRING="$(connectionstring)"

Without a service, how can I access the port like agent.services.sqlserver.ports.1433?

github-actions[bot] commented 1 year ago

This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days