New templating engine not fully compatible with cookiecutter

nadworny commented 2 years ago

Expected Behavior

After migrating to the new scaffolder.backstage.io/v1beta3, the templating behaves the same as before migration.

Actual Behavior

Templating fails.

I have the following catalog-info.yaml:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: {{ cookiecutter.project_slug.replace('-','') }}
  description: {{ cookiecutter.project_slug.replace('-',' ').title()}} microservice

First problem

title() seems to break the template generation with the following error:

2022-05-18T12:09:30.884Z info: Writing file {{cookiecutter.project_slug}}/catalog-info.yaml to template output path with mode 33188. {"timestamp":"2022-05-18T12:09:30.891Z"}[22](https://backstage.devsit.xxx.com/create/tasks/3c96eb43-ae64-4c17-aab1-a94b1837cf51#line-22)
2022-05-18T12:09:30.884Z Error: (unknown path) [Line 4, Column 51][23](https://backstage.devsit.xxx.com/create/tasks/3c96eb43-ae64-4c17-aab1-a94b1837cf51#line-23)  Error: Unable to call `the return value of (cookiecutter["project_slug"]["replace"])["title"]`, which is undefined or falsey

Second problem

Replace doesn't behave the same as with cookiecutter. It just replaces the first occurrence instead of all. It's strange because if I look at the replace function in nunjucks, it also replaces all occurrences by default. Assuming we do {{ cookiecutter.project_slug.replace('-','') }} with project_slug: this-is-a-test Before beta3 produces thisisatest Current beta3 produces thisis-a-test

Steps to Reproduce

Use the following template.yaml definition:


apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
name: templateapipython
title: Create Python API 
description: Create a project...
spec:
owner: test-team
type: service

parameters:
- title: Fill in needed information
  required:
    - name
  properties:
    name:
      title: Name
      type: string
      description: Unique name of the component with "-" 
      ui:autofocus: true
      ui:options:
        rows: 5

steps:
- id: fetch_base
  name: Fetch Base
  action: fetch:template
  input:
    url: '.'
    copyWithoutRender: ['template/main.jinja2', 'template.yaml']
    cookiecutterCompat: true
    values:
      module_name: ${{ parameters.name }}
      project_slug: ${{ parameters.name }} 

output:
links:
  - title: Repository
    url: ${{ steps.publish.output.remoteUrl }}
  - title: Open in catalog
    icon: catalog
    entityRef: ${{ steps.register.output.entityRef }}

2. Take the above catalog-info.yaml as a template
3. Create a new template using backstage with a name "this-is-a-test"

## Context
Compatibility with cookiecutter is a must for us as we have a lot of templates. Migration might be very difficult.
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context (e.g. links to configuration settings, -->
<!--- stack trace or log data) helps us come up with a solution that is most useful in the real world -->

## Your Environment
- Browser Information: Microsoft Edge Version 101.0.1210.39 (Official build) (arm64)

- Output of `yarn backstage-cli info`: <!--- Paste into the code block below  -->

```text
OS:   Darwin 21.4.0 - darwin/arm64
node: v16.15.0
yarn: 1.22.17
cli:  0.17.0 (installed)

Dependencies:
  @backstage/app-defaults               1.0.1
  @backstage/backend-common             0.13.2
  @backstage/backend-tasks              0.3.0
  @backstage/catalog-client             1.0.1
  @backstage/catalog-model              1.0.1
  @backstage/cli-common                 0.1.8
  @backstage/cli                        0.17.0
  @backstage/config-loader              1.1.0
  @backstage/config                     1.0.0
  @backstage/core-app-api               1.0.1
  @backstage/core-components            0.9.3
  @backstage/core-plugin-api            1.0.1
  @backstage/errors                     1.0.0
  @backstage/integration-react          1.0.1
  @backstage/integration                1.1.0
  @backstage/plugin-api-docs            0.8.4
  @backstage/plugin-app-backend         0.3.31
  @backstage/plugin-auth-backend        0.13.0
  @backstage/plugin-auth-node           0.2.0
  @backstage/plugin-catalog-backend     1.1.1
  @backstage/plugin-catalog-common      1.0.1
  @backstage/plugin-catalog-graph       0.2.16
  @backstage/plugin-catalog-import      0.8.7
  @backstage/plugin-catalog-react       1.0.1
  @backstage/plugin-catalog             1.1.0
  @backstage/plugin-github-actions      0.5.4
  @backstage/plugin-org                 0.5.4
  @backstage/plugin-permission-common   0.6.0
  @backstage/plugin-permission-node     0.6.0
  @backstage/plugin-permission-react    0.4.0
  @backstage/plugin-proxy-backend       0.2.25
  @backstage/plugin-scaffolder-backend  1.1.0
  @backstage/plugin-scaffolder-common   1.0.1
  @backstage/plugin-scaffolder          1.1.0
  @backstage/plugin-search-backend-node 0.6.0
  @backstage/plugin-search-backend      0.5.1
  @backstage/plugin-search-common       0.3.3
  @backstage/plugin-search-react        0.1.0
  @backstage/plugin-search              0.8.0
  @backstage/plugin-tech-radar          0.5.11
  @backstage/plugin-techdocs-backend    1.1.0
  @backstage/plugin-techdocs-node       1.1.0
  @backstage/plugin-techdocs-react      0.1.0
  @backstage/plugin-techdocs            1.1.0
  @backstage/plugin-user-settings       0.4.3
  @backstage/release-manifests          0.0.2
  @backstage/search-common              0.3.3
  @backstage/test-utils                 1.0.1
  @backstage/theme                      0.2.15
  @backstage/types                      1.0.0
  @backstage/version-bridge             1.0.1

jhaals commented 2 years ago

Hi there!

You're right that the the templates is not fully compatible with cookiecutter, that's because the default template engine now uses nunjucks for templating which has the benefit of not requiring installation of cookiecutter. You can still use cookiecutter if you are unable to work around the differences in the two templating engines by using scaffolder-backend-module-cookiecutter, there's also some additional help and context in the migration guide. I hope that helps!

nadworny commented 2 years ago

Hi @jhaals Thanks for your input. I didn't find anything specific to the cookiecutter scaffolder except of setting cookiecutterCompat: true. Did I miss something? Would you be so kind and tell me how can I enable the scaffolder-backend-module-cookiecutter?

Rugvip commented 2 years ago

@nadworny The scaffolder-backend-module-cookiecutter is a separate module that you install if you want full cookiecutter support. You'll need to either have cookiecutter installed inside the scaffolder instance, or it needs to have access to a docker daemon. The setup instructions are documented in the README.

Will close this as it's working as intended. Let us know if we missed anything.

backstage / backstage