Packing does not work with `poetry`

paulschroeder-tomtom commented 10 months ago

We use poetry to manage our dependencies. When we use in our workflows like that:

    - name: Upload Function
      uses: Azure/functions-action@v1
      with:
        app-name: ${{ inputs.function_app_name }}
        package: ${{ inputs.function_directory }}
        publish-profile: ${{ steps.pubprofile.outputs.publish_profile_xml }}
        scm-do-build-during-deployment: true
        enable-oryx-build: true

we encounter the following error:

...
Removing existing manifest file
Running pip install poetry...
Running poetry install...
Done in 0 sec(s).
/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e1997a166d9/build.sh: line 302: poetry: command not found
"2023-10-23 [13](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:14):34:00"|WARNING|/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e1997a[16](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:17)6d9/build.sh: line 302: poetry: command not found | Exit code: {pythonBuildExitCode} | Please review message | More information: https://aka.ms/troubleshoot-python
....

I also found this two issue:

https://github.com/microsoft/Oryx/issues/1370 but the suggested fix of setting the PRE_BUILD_COMMAND env.var. did not work.
https://github.com/microsoft/Oryx/issues/1769 which seems to be stale without any suggested fix.

Full log:

Run Azure/functions-action@v1
Successfully parsed SCM credential from publish-profile format.
Using SCM credential for authentication, GitHub Action will not perform resource validation.
(node:1823) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Successfully acquired app settings from function app (with SCM credential)!
Will archive orchestration-api into /home/runner/work/_temp/temp_web_package_6374599572604525.zip as function app content
Will use Kudu https://<scmsite>/api/zipdeploy to deploy since publish-profile is detected.
Setting SCM_DO_BUILD_DURING_DEPLOYMENT in Kudu container to true
Update using context.kuduService.updateAppSettingViaKudu
Response with status code 204
App setting SCM_DO_BUILD_DURING_DEPLOYMENT propagated to Kudu container
Setting ENABLE_ORYX_BUILD in Kudu container to true
Update using context.kuduService.updateAppSettingViaKudu
Response with status code 204
App setting ENABLE_ORYX_BUILD propagated to Kudu container
Package deployment using ZIP Deploy initiated.
Updating submodules.
Preparing deployment for commit id '0999517d-a'.
PreDeployment: context.CleanOutputPath False
PreDeployment: context.OutputPath /home/site/wwwroot
Repository path is /tmp/zipdeploy/extracted
Running oryx build...
Command: oryx build /tmp/zipdeploy/extracted -o /tmp/build/expressbuild --platform python --platform-version 3.10.4 -i /tmp/8dbd3ccab83cb35 -p packagedir=.python_packages/lib/site-packages
Operation performed by Microsoft Oryx, https://github.com/Microsoft/Oryx
You can report issues at https://github.com/Microsoft/Oryx/issues

Oryx Version: 0.2.20230508.1, Commit: 7fe2bf39b357dd68572b438a85ca50b5ecfb4592, ReleaseTagName: 20230508.1

Build Operation ID: bf103ce1e3005bdd
Repository Commit : 0999517d-a5cd-4ea4-94bf-7fc2f3f48e91
OS Type           : bullseye
Image Type        : githubactions

Detecting platforms...
Detected following platforms:
  python: 3.10.4
Version '3.10.4' of platform 'python' is not installed. Generating script to install it...

Using intermediate directory '/tmp/8dbd3ccab83cb35'.

Copying files to the intermediate directory...
Done in 0 sec(s).

Source directory     : /tmp/8dbd3ccab83cb35
Destination directory: /tmp/build/expressbuild

Downloading and extracting 'python' version '3.10.4' to '/tmp/oryx/platforms/python/3.10.4'...
Detected image debian flavor: bullseye.
Downloaded in 2 sec(s).
Verifying checksum...
Extracting contents...
performing sha512 checksum for: python...
Done in 12 sec(s).

image detector file exists, platform is python..
OS detector file exists, OS is bullseye..
Python Version: /tmp/oryx/platforms/python/3.10.4/bin/python3.10
Creating directory for command manifest file if it does not exist
/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e1997a166d9/build.sh: /tmp/oryx/platforms/python/3.10.4/bin/pip: /opt/python/3.10.4/bin/python3.10: bad interpreter: No such file or directory
Removing existing manifest file
Running pip install poetry...
Running poetry install...
Done in 0 sec(s).
/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e1997a166d9/build.sh: line 302: poetry: command not found
"2023-10-23 [13](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:14):34:00"|WARNING|/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e1997a[16](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:17)6d9/build.sh: line 302: poetry: command not found | Exit code: {pythonBuildExitCode} | Please review message | More information: https://aka.ms/troubleshoot-python
/tmp/BuildScriptGenerator/c8cd985a6073411b9c027e[19](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:20)97a166d9/build.sh: /tmp/oryx/platforms/python/3.10.4/bin/pip: /opt/python/3.10.4/bin/python3.10: bad interpreter: No such file or directory\n/opt/Kudu/Scripts/starter.sh oryx build /tmp/zipdeploy/extracted -o /tmp/build/expressbuild --platform python --platform-version 3.10.4 -i /tmp/8dbd3ccab83cb35 -p packagedir=.python_packages/lib/site-packages

Generating summary of Oryx build
Deployment Log file does not exist in /tmp/oryx-build.log
The logfile at /tmp/oryx-build.log is empty. Unable to fetch the summary of build
Deployment Failed. deployer = GITHUB_ZIP_DEPLOY_FUNCTIONS_V1 deploymentPath = Functions App ZipDeploy. Extract zip. Remote build.
Error: Failed to deploy web package to App Service.
Error: Execution Exception (state: PublishContent) (step: Invocation)
Error:   When request Azure resource at PublishContent, zipDeploy : Failed to use /home/runner/work/_temp/temp_web_package_63745995726045[25](https://github.com/.../actions/runs/6613992160/job/17963065166?pr=143#step:7:26).zip as ZipDeploy content
Error:     Package deployment using ZIP Deploy failed. Refer logs for more details.
Error: Deployment Failed!

github-actions[bot] commented 10 months ago

This issue is idle because it has been open for 14 days with no activity.

paulschroeder-tomtom commented 10 months ago

Bump...

github-actions[bot] commented 9 months ago

This issue is idle because it has been open for 14 days with no activity.

GodVenn commented 8 months ago

@paulschroeder-tomtom Hi, I just made a workflow deploying a python flask app as an azure function, using poetry. Here is an excerpt, hope it helps!

env:
  AZURE_FUNCTIONAPP_NAME: 'my_name'   # set this to your function app name on Azure
  AZURE_FUNCTIONAPP_PACKAGE_PATH: '.'       # set this to the path to your function app project, defaults to the repository root
  PYTHON_VERSION: '3.8'                     # set this to the python version to use (e.g. '3.6', '3.7', '3.8')
  POETRY_VERSION: '1.7.1'                     # set this to the python version to use (e.g. '3.6', '3.7', '3.8')

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
    - name: 'Checkout GitHub Action'
      uses: actions/checkout@v3

    - name: Setup Python ${{ inputs.PYTHON_VERSION }} Environment
      uses: actions/setup-python@v4
      with:
        python-version: ${{ inputs.PYTHON_VERSION }}

    - name: 'Set up poetry'
      uses: abatilo/actions-poetry@v2
      with:
        poetry-version: ${{ inputs.POETRY_VERSION }}

    - name: 'Resolve Project Dependencies Using Poetry'
      shell: bash
      run: |
        pushd './${{ inputs.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
        poetry export --format requirements.txt --output requirements.txt
        pip install -r requirements.txt --target=".python_packages/lib/site-packages"
        popd

    - name: Log into Azure
      uses: azure/login@v1
      with:
        client-id: ${{ secrets.AZURE_CLIENT_ID }}
        tenant-id: ${{ secrets.AZURE_TENANT_ID }}
        subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

    - name: 'Run Azure Functions Action'
      uses: Azure/functions-action@v1
      id: fa
      with:
        app-name: ${{ inputs.AZURE_FUNCTIONAPP_NAME }}
        package: ${{ inputs.AZURE_FUNCTIONAPP_PACKAGE_PATH }}
        scm-do-build-during-deployment: false
        enable-oryx-build: false
        respect-funcignore: true

github-actions[bot] commented 7 months ago

This issue is idle because it has been open for 14 days with no activity.

paulschroeder-tomtom commented 4 months ago

@GodVenn thank! But since MS was not helpful in that regard (honestly and sadly no surprise :disappointed:) we settled on the following workaround:

jobs:
  func:
    runs-on: ubuntu-latest
    steps:
    - name: Checkout repository
      uses: actions/checkout@v4
      with:
        ref: ${{ inputs.commit_sha }}

    - name: Install dependencies and login to Azure CLI
      uses: ./.github/actions/prepare-runner
      with:
        azure-tenant-id: ${{ vars.AZURE_TENANT_ID }}
        azure-subscription-id: ${{ vars.AZURE_SUBSCRIPTION_ID }}
        azure-client-id: ${{ vars.AZURE_CLIENT_ID }}

# interesting stuff starts here:

    - name: Install poetry
      uses: abatilo/actions-poetry@v2
# optionally select a version:
      #      with:
      #        poetry-version: # default=latest

    - name: Make Oryx happy with requirements.txt
      working-directory: ${{ inputs.function_directory }}
      run: |
        poetry export --without=test --without-hashes --format=requirements.txt > requirements.txt

    - name: Upload Function
      uses: Azure/functions-action@v1
      with:
        app-name: ${{ inputs.function_app_name }}
        package: ${{ inputs.function_directory }}
        scm-do-build-during-deployment: true
        enable-oryx-build: true

I seems like the Oryx builder which the Azure/functions-action@v1 is using does the pip install itself. I complains if its not finding a requirements.txt, which gets fixed by exporting with poetry. Feel free to mix and match our solutions.

GodVenn commented 4 months ago

@paulschroeder-tomtom Yeah, that's essentially the same workaround we have!

GPetrites commented 1 month ago

Very strange because App Services, which also uses Oryx, works correctly with poetry

Azure / functions-action

Packing does not work with `poetry` #206