Regex error when installing pydantic as dependency

[OldGrumpyViking]

Tested on python 3.12.1 & hatch 1.9.1 & hatch-pip-compile 1.9.0.

Following pip-compile settings in pyproject.toml:

[project]
dependencies = ["pydantic"]

[tool.hatch.envs.default]
python = "3.8"
features = []
type = "pip-compile"
pip-compile-constraint = "default"         # keep locks between default & others consistent
lock-filename = "locks/{env_name}.lock"
pip-compile-hashes = true
pip-compile-verbose = true
pip-compile-args = ["--no-emit-index-url"]

[tool.hatch.envs.dev]
features = ["dev"]

Then running hatch run dev:pip -V i get the following traceback:

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch\cli\__init__.py:221 in     │
│ main                                                                                             │
│                                                                                                  │
│   218                                                                                            │
│   219 def main():  # no cov                                                                      │
│   220 │   try:                                                                                   │
│ ❱ 221 │   │   return hatch(prog_name='hatch', windows_expand_args=False)                         │
│   222 │   except Exception:  # noqa: BLE001                                                      │
│   223 │   │   from rich.console import Console                                                   │
│   224                                                                                            │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:1157 in __call__   │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:1078 in main       │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:1688 in invoke     │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:1434 in invoke     │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:783 in invoke      │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\decorators.py:45 in        │
│ new_func                                                                                         │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\decorators.py:33 in        │
│ new_func                                                                                         │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch\cli\run\__init__.py:79 in  │
│ run                                                                                              │
│                                                                                                  │
│   76 │   elif not env_name:                                                                      │
│   77 │   │   env_name = 'system'                                                                 │
│   78 │                                                                                           │
│ ❱ 79 │   ctx.invoke(                                                                             │
│   80 │   │   run_command,                                                                        │
│   81 │   │   args=[command, *args],                                                              │
│   82 │   │   env_names=[env_name],                                                               │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\core.py:783 in invoke      │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\click\decorators.py:45 in        │
│ new_func                                                                                         │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch\cli\env\run.py:180 in run  │
│                                                                                                  │
│   177 │   │   │   if env_name == 'system':                                                       │
│   178 │   │   │   │   environment.exists = lambda: True                                          │
│   179 │   │   │                                                                                  │
│ ❱ 180 │   │   │   app.prepare_environment(environment)                                           │
│   181 │   │   │   app.run_shell_commands(                                                        │
│   182 │   │   │   │   environment,                                                               │
│   183 │   │   │   │   [environment.join_command_args(args)],                                     │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch\cli\application.py:107 in  │
│ prepare_environment                                                                              │
│                                                                                                  │
│   104 │   │   │   │   │   with self.status('Running post-installation commands'):                │
│   105 │   │   │   │   │   │   self.run_shell_commands(environment, environment.post_install_co   │
│   106 │   │                                                                                      │
│ ❱ 107 │   │   new_dep_hash = environment.dependency_hash()                                       │
│   108 │   │   current_dep_hash = self.env_metadata.dependency_hash(environment)                  │
│   109 │   │   if new_dep_hash != current_dep_hash:                                               │
│   110 │   │   │   with self.status('Checking dependencies'):                                     │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch_pip_compile\plugin.py:97   │
│ in dependency_hash                                                                               │
│                                                                                                  │
│    94 │   │   """                                                                                │
│    95 │   │   Get the dependency hash                                                            │
│    96 │   │   """                                                                                │
│ ❱  97 │   │   self.run_pip_compile()                                                             │
│    98 │   │   hatch_hash = super().dependency_hash()                                             │
│    99 │   │   if not self.dependencies:                                                          │
│   100 │   │   │   return hatch_hash                                                              │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch_pip_compile\plugin.py:129  │
│ in run_pip_compile                                                                               │
│                                                                                                  │
│   126 │   │   │   │   │   _ = self.piptools_lock.compare_python_versions(                        │
│   127 │   │   │   │   │   │   verbose=self.config.get("pip-compile-verbose", None)               │
│   128 │   │   │   │   │   )                                                                      │
│ ❱ 129 │   │   │   │   self.pip_compile_cli()                                                     │
│   130 │                                                                                          │
│   131 │   def pip_compile_cli(self) -> None:                                                     │
│   132 │   │   """                                                                                │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch_pip_compile\plugin.py:180  │
│ in pip_compile_cli                                                                               │
│                                                                                                  │
│   177 │   │   │   │   shutil.copy(self.piptools_lock_file, output_file)                          │
│   178 │   │   │   self.piptools_lock_file.parent.mkdir(exist_ok=True, parents=True)              │
│   179 │   │   │   self.plugin_check_command(cmd)                                                 │
│ ❱ 180 │   │   │   self.piptools_lock.process_lock(lockfile=output_file)                          │
│   181 │   │   │   shutil.move(output_file, self.piptools_lock_file)                              │
│   182 │   │   self.lockfile_up_to_date = True                                                    │
│   183                                                                                            │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\site-packages\hatch_pip_compile\lock.py:60 in  │
│ process_lock                                                                                     │
│                                                                                                  │
│    57 │   │   │   constraints_path = self.constraints_file.relative_to(self.project_root)        │
│    58 │   │   │   constraints_line = f"# [constraints] {constraints_path} (SHA256: {constraint   │
│    59 │   │   │   joined_dependencies = "\n".join([constraints_line, "#", joined_dependencies]   │
│ ❱  60 │   │   │   cleaned_input_file = re.sub(                                                   │
│    61 │   │   │   │   r"-c \S*",                                                                 │
│    62 │   │   │   │   f"-c {constraints_path}",                                                  │
│    63 │   │   │   │   cleaned_input_file,                                                        │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\re\__init__.py:186 in sub                      │
│                                                                                                  │
│   183 │   if a string, backslash escapes in it are processed.  If it is                          │
│   184 │   a callable, it's passed the Match object and must return                               │
│   185 │   a replacement string to be used."""                                                    │
│ ❱ 186 │   return _compile(pattern, flags).sub(repl, string, count)                               │
│   187                                                                                            │
│   188 def subn(pattern, repl, string, count=0, flags=0):                                         │
│   189 │   """Return a 2-tuple containing (new_string, number).                                   │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\re\__init__.py:334 in _compile_template        │
│                                                                                                  │
│   331 @functools.lru_cache(_MAXCACHE)                                                            │
│   332 def _compile_template(pattern, repl):                                                      │
│   333 │   # internal: compile replacement pattern                                                │
│ ❱ 334 │   return _sre.template(pattern, _parser.parse_template(repl, pattern))                   │
│   335                                                                                            │
│   336 # register myself for pickling                                                             │
│   337                                                                                            │
│                                                                                                  │
│ C:\Users\user\.pyenv\pyenv-win\versions\3.12.1\Lib\re\_parser.py:1075 in parse_template           │
│                                                                                                  │
│   1072 │   │   │   │   │   this = chr(ESCAPES[this][1])                                          │
│   1073 │   │   │   │   except KeyError:                                                          │
│   1074 │   │   │   │   │   if c in ASCIILETTERS:                                                 │
│ ❱ 1075 │   │   │   │   │   │   raise s.error('bad escape %s' % this, len(this)) from None        │
│   1076 │   │   │   │   lappend(this)                                                             │
│   1077 │   │   else:                                                                             │
│   1078 │   │   │   lappend(this)                                                                 │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
error: bad escape \d at position 8

The same command works in the default environment hatch run pip -V

[oprypin]

Aha this is a fun one 🙂

Indeed this is the problematic part

f"-c {constraints_path}",

The replacement string of a regex is a special syntax with its own escapes, here it's saying probably that for "-c locks\default.lock" \d is not a valid escape.

Actually 2 things to do here:

• not use special replacement strings, it can instead be a replacement with a lambda that returns the same string

• the path should have normal slashes even on Windows

[juftin]

Oh, it's a RegEx + Windows problem 🥴

Thank you so much for reporting this, I'll get it resolved today. I'll also see what I can do to add a Windows testing environment that would've caught this.

[oprypin]

There's a regex problem and a Windows problem. The linked pull request is still not fully safe from the regex problem 😅

[juftin]

:tada: This issue has been resolved in version 1.9.1 :tada:

The release is available on GitHub release

Your semantic-release bot :package::rocket:

[OldGrumpyViking]

@juftin & @oprypin thank you very much for the promt fix :) This package is just what i was missing 👍🏻