Use copier instead of cookiecutter

GenevieveBuckley commented 2 months ago

This PR switches the plugin template repo to using copier instead of cookiecutter

The biggest advantage of using copier is that users can run copier update to check for any changes in the template.

This will be very useful for ongoing code maintenance (eg: updating versions of dependencies, any changes required to keep the test_and_deploy github action working smoothly, etc.)

But it also has some smaller advantages, like being able to conditionally generate files, depending on the user's answers to the prompt questions (cookiecutter does not do this, were were previously using a hack in the hooks/post_gen.py file to delete the unnecessary files)

Steps required (ideally all steps should be close to simultaneous):

[ ] Merge this PR
[ ] rename repository napari-plugin-template instead of cookiecutter-napari-plugin (and remove the other test napari-plugin-template repo)
[ ] Merge corresponding documentation PR, replacing references to cookiecutter with copier: https://github.com/napari/docs/pull/427

Previously:

jni commented 2 months ago

Thanks for reopening @GenevieveBuckley! 🚀

Tests are failing currently 🤔 Also when I pull this down to try it out, I got an error:

$ copier copy --trust cookiecutter-napari-plugin algospatial
No git tags found in template; using HEAD as ref
🎤 The name of your plugin
   algospatial
🎤 Display name for your plugin
   AlgoSpatial
🎤 Plugin module name
   algospatial
🎤 Short description of what your plugin does
   Spatial algorithms in napari
🎤 Developer name
   Juan Nunez-Iglesias
🎤 Email address
   juan.nunez-iglesias@monash.edu
🎤 Github user or organisation name
   jni
🎤 Github repository URL
   provide later
🎤 Include reader plugin?
   Yes
🎤 Include writer plugin?
   No
🎤 Include sample data plugin?
   No
🎤 Include widget plugin?
   Yes
🎤 Use git tags for versioning?
   Yes
🎤 Install pre-commit? (Code formatting checks)
   No
🎤 Install dependabot? (Automatic security updates of dependency versions)
   No
🎤 Which licence do you want your plugin code to have?
   Mozilla Public License 2.0

Copying from template version 0.0.0.post109.dev0+134ce29
    create  .
    create  .napari-hub
    create  .napari-hub/DESCRIPTION.md
    create  .napari-hub/config-yml
    create  .gitignore
    create  .github
    create  .github/workflows
    create  .github/workflows/test_and_deploy
    create  src
    create  src/algospatial
    create  src/algospatial/_tests
    create  src/algospatial/_tests/__init__.py

followed by this crash:

Traceback (most recent call last):
  File "/Users/jni/micromamba/envs/copier/bin/copier", line 10, in <module>
    sys.exit(CopierApp.run())
             ^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/plumbum/cli/application.py", line 638, in run
    inst, retcode = subapp.run(argv, exit=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/plumbum/cli/application.py", line 633, in run
    retcode = inst.main(*tailargs)
              ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 280, in main
    return _handle_exceptions(inner)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 70, in _handle_exceptions
    method()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 271, in inner
    with self._worker(
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 217, in __exit__
    raise value
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/cli.py", line 278, in inner
    worker.run_copy()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 769, in run_copy
    self._render_folder(src_abspath)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 659, in _render_folder
    self._render_folder(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 661, in _render_folder
    self._render_file(file)
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/copier/main.py", line 582, in _render_file
    new_content = tpl.render(**self._render_context()).encode()
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/environment.py", line 1304, in render
    self.environment.handle_exception()
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/environment.py", line 939, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/private/var/folders/z7/mv1jznhx59q5vc3bp09gj69m0000gn/T/copier.vcs.clone.y8mvpar1/template/src/{{module_name}}/_tests/{% if include_widget_plugin %}test_widget.py{% endif %}.jinja", line 3, in top-level template code
  File "/Users/jni/micromamba/envs/copier/lib/python3.12/site-packages/jinja2/sandbox.py", line 327, in getattr
    value = getattr(obj, attribute)
            ^^^^^^^^^^^^^^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'cookiecutter' is undefined

maybe something missing from carrying the PR over? (Since there shouldn't be any reference to cookiecutter left...?)

Also, the plugin name is a default rather than a placeholder — I think it should also be a placeholder?

GenevieveBuckley commented 2 months ago

Also, the plugin name is a default rather than a placeholder — I think it should also be a placeholder?

I'm starting to feel like placeholders are most suitable for optional variables. Otherwise, if someone just hits enter the whole way through the prompts, that will give us empty prompt values and can crash the operation. Opinions?

You must have a non-empty value for three questions: plugin_name, display_name, and module_name. If any of those are left blank, because of how we've set the template repository up, you can't create a new repo from it. There's a returned non-zero exit status 1 error message.

Details (click to expand)

```python-traceback 🎤 The name of your plugin napari-foobar 🎤 Display name for your plugin 🎤 Plugin module name napari_foobar 🎤 Short description of what your plugin does A simple plugin to use FooBar segmentation within napari 🎤 Developer name Napari Developer 🎤 Email address yourname@example.com 🎤 Github user or organisation name githubuser 🎤 Github repository URL provide later 🎤 Include reader plugin? Yes 🎤 Include writer plugin? Yes 🎤 Include sample data plugin? Yes 🎤 Include widget plugin? Yes 🎤 Use git tags for versioning? No 🎤 Install pre-commit? (Code formatting checks) No 🎤 Install dependabot? (Automatic security updates of dependency versions) No 🎤 Which licence do you want your plugin code to have? BSD-3 Copying from template version 0.0.0.post113.dev0+3c0a5eb create . create .napari-hub create .napari-hub/DESCRIPTION.md create .napari-hub/config-yml create LICENSE create .gitignore create .github create .github/workflows create .github/workflows/test_and_deploy create src create src/napari_foobar create src/napari_foobar/_writer.py create src/napari_foobar/_tests create src/napari_foobar/_tests/test_sample_data.py create src/napari_foobar/_tests/test_writer.py create src/napari_foobar/_tests/__init__.py create src/napari_foobar/_tests/test_widget.py create src/napari_foobar/_tests/test_reader.py create src/napari_foobar/_sample_data.py create src/napari_foobar/napari.yaml create src/napari_foobar/__init__.py create src/napari_foobar/_reader.py create src/napari_foobar/_widget.py create src/README.md create src/pyproject.toml create src/MANIFEST.in create src/tox.ini > Running task 1 of 1: ['/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/python3.11', '/private/var/folders/82/phlx_f6j5518qtvpm4x0_0fm0000gq/T/copier.vcs.clone.i6_fg585/_tasks.py', '--plugin_name=napari-foobar', '--module_name=napari_foobar', '--project_directory=plugin-test4', '--install_precommit=False', '--github_repository_url=provide later', '--github_username_or_organization='] ERROR:pre_gen_project:b'\xf0\x9f\x85\x87 Invalid! 4 validation errors for PluginManifest\ndisplay_name\n none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> display_name\n none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> display_name\n none is not an allowed value (type=type_error.none.not_allowed)\ncontributions -> sample_data -> 0 -> uri\n field required (type=value_error.missing)' Traceback (most recent call last): File "/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/copier", line 10, in sys.exit(CopierApp.run()) ^^^^^^^^^^^^^^^ File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/plumbum/cli/application.py", line 638, in run inst, retcode = subapp.run(argv, exit=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/plumbum/cli/application.py", line 633, in run retcode = inst.main(*tailargs) ^^^^^^^^^^^^^^^^^^^^ File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 280, in main return _handle_exceptions(inner) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 70, in _handle_exceptions method() File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 271, in inner with self._worker( File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 217, in __exit__ raise value File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/cli.py", line 278, in inner worker.run_copy() File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 774, in run_copy self._execute_tasks(self.template.tasks) File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/site-packages/copier/main.py", line 296, in _execute_tasks subprocess.run(task_cmd, shell=use_shell, check=True, env=local.env) File "/Users/genevieb/mambaforge/envs/plugin-template-dev/lib/python3.11/subprocess.py", line 571, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/Users/genevieb/mambaforge/envs/plugin-template-dev/bin/python3.11', '/private/var/folders/82/phlx_f6j5518qtvpm4x0_0fm0000gq/T/copier.vcs.clone.i6_fg585/_tasks.py', '--plugin_name=napari-foobar', '--module_name=napari_foobar', '--project_directory=plugin-test4', '--install_precommit=False', '--github_repository_url=provide later', '--github_username_or_organization=']' returned non-zero exit status 1. ```

jni commented 2 months ago

I'm fine with an error. I personally find it jarring to have to erase text to type what I want. It would be better if copier would give you the option of setting something as not-empty so that the error could be caught early, but I think that's a thing for copier to worry about, not us. (Maybe the feature exists, or maybe we can make an issue in the repo.)

jni commented 2 months ago

https://copier.readthedocs.io/en/stable/reference/user_data/#copier.user_data.Question

Looks like you can put in a "validator":

Jinja template with which to validate the user input. This template will be rendered with the combined answers as variables; it should render nothing if the value is valid, and an error message to show to the user otherwise.

I'm not familiar enough with template language here but I presume we can implement exactly the check above...

GenevieveBuckley commented 2 months ago

Oh good. There's one more check that needs to happen - a valid email address also appears to be required (you can generate a new repo from the template without it, but then tox will fail on that new repo)

jni commented 2 months ago

Here's an example of using validators:

https://copier.readthedocs.io/en/stable/configuring/#advanced-prompt-formatting

project_name:
    type: str # Any value will be treated raw as a string
    help: An awesome project needs an awesome name. Tell me yours.
    default: paradox-specifier
    validator: >-
        {% if not (project_name | regex_search('^[a-z][a-z0-9\-]+$')) %}
        project_name must start with a letter, followed one or more letters, digits or dashes all lowercase.
        {% endif %}

GenevieveBuckley commented 2 months ago

Ooh, maybe we can also use the validators to check module_name first the PEP specs. If we can get rid of _tasks.py, then we don't need to use the copier --trust command anymore, which would be nice.

jni commented 2 months ago

The above validator seems to do just that, so I think that's a yes! And indeed it would be amazing!

GenevieveBuckley commented 2 months ago

Two points:

Sadly we can't get rid of _tasks.py and --trust. It does more than just plugin and module name validation (also git init, optional pre-commit install, and validating the manifest with npe2). I'd forgotten about those extra tasks in my excitement.
I'm choosing not to validate the input email address, because then users cannot leave it blank. That seems a bit privacy violating to me. The jinja regex engine will not match an empty string, no matter how I write that in the regex, and I can't figure out if there's an 'allow_empty' kwarg I can somehow pass it through copier.

jni commented 1 month ago

Closing in favour of napari/napari-plugin-template#7

napari / cookiecutter-napari-plugin

Use copier instead of cookiecutter #190