search

Aider-AI / aider

aider is AI pair programming in your terminal
https://aider.chat/
Apache License 2.0
22.17k stars 2.06k forks source link

SyntaxWarning: "invalid escape sequence '\.'" coming from `pypandoc` dependency #1722

Open Quidge opened 1 month ago

Quidge commented 1 month ago

Issue

$ aider
/opt/homebrew/Cellar/aider/0.57.1/libexec/lib/python3.12/site-packages/pypandoc/pandoc_download.py:61: SyntaxWarning: invalid escape sequence '\.'
  regex = re.compile(r"/jgm/pandoc/releases/download/.*(?:"+processor_architecture+"|x86|mac).*\.(?:msi|deb|pkg)")
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Aider v0.57.1
Main model: claude-3-5-sonnet-20240620 with diff edit format, prompt cache, infinite output
Weak model: claude-3-haiku-20240307
Git repo: .git with 33 files
Repo-map: using 1024 tokens, files refresh
Use /help <question> for help, run "aider --help" to see cmd line args
─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
CONVENTIONS.md
>

This appears to be an issue on Python 3.12 for the pypandoc library which aider uses. There's a patch in pypandoc where they fix this but it isn't released yet. aider's requirements.txt is currently set to use pypandoc==1.13 which still has the syntax warning. I think Aider will need to do something like pypandoc>=1.14 once pypandoc releases to get rid of that warning.

Version and model info

Aider v0.57.1 (installed with playright as well) Python 3.12

fry69 commented 1 month ago

Thank you for filing this issue.

At first glance this looks like you installed aider in an environment with many other Python modules and aider's dependencies got mixed up. Can you please try to install aider in a separate Python environment? Either with venv or pipx?

This document may be helpful -> https://aider.chat/docs/troubleshooting/imports.html

savchenko commented 1 month ago

I have noticed the same error in a different project, completely unrelated to aider. The issue is in the pypandoc.

Ref. https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1994136.html

ngharrington commented 1 month ago

The introduction of pydub also seems to have introduced more of these warnings

aider-sandbox$ uv venv --python 3.12 --no-project --seed venv312
aider-sandbox$ uv python find
/home/neal/workspace/aider-sandbox/venv312/bin/python3
neal@neal-xps:aider-sandbox$ uv pip install --no-cache aider-chat 
Resolved 84 packages in 1.41s
   Built pyperclip==1.9.0
Prepared 84 packages in 4.02s
Installed 84 packages in 48ms
 + aider-chat==0.59.1
 + aiohappyeyeballs==2.4.0
 + aiohttp==3.10.5
 + aiosignal==1.3.1
 + annotated-types==0.7.0
 + anyio==4.6.0
 + attrs==24.2.0
 + backoff==2.2.1
 + beautifulsoup4==4.12.3
 + certifi==2024.8.30
 + cffi==1.17.1
 + charset-normalizer==3.3.2
 + click==8.1.7
 + configargparse==1.7
 + diff-match-patch==20230430
 + diskcache==5.6.3
 + distro==1.9.0
 + filelock==3.16.1
 + flake8==7.1.1
 + frozenlist==1.4.1
 + fsspec==2024.9.0
 + gitdb==4.0.11
 + gitpython==3.1.43
 + grep-ast==0.3.3
 + h11==0.14.0
 + httpcore==1.0.5
 + httpx==0.27.2
 + huggingface-hub==0.25.0
 + idna==3.10
 + importlib-metadata==7.2.1
 + importlib-resources==6.4.5
 + jinja2==3.1.4
 + jiter==0.5.0
 + json5==0.9.25
 + jsonschema==4.23.0
 + jsonschema-specifications==2023.12.1
 + litellm==1.47.0
 + markdown-it-py==3.0.0
 + markupsafe==2.1.5
 + mccabe==0.7.0
 + mdurl==0.1.2
 + multidict==6.1.0
 + networkx==3.2.1
 + numpy==1.26.4
 + openai==1.47.0
 + packaging==24.1
 + pathspec==0.12.1
 + pexpect==4.9.0
 + pillow==10.4.0
 + prompt-toolkit==3.0.47
 + psutil==6.0.0
 + ptyprocess==0.7.0
 + pycodestyle==2.12.1
 + pycparser==2.22
 + pydantic==2.9.2
 + pydantic-core==2.23.4
 + pydub==0.25.1
 + pyflakes==3.2.0
 + pygments==2.18.0
 + pypandoc==1.13
 + pyperclip==1.9.0
 + python-dotenv==1.0.1
 + pyyaml==6.0.2
 + referencing==0.35.1
 + regex==2024.9.11
 + requests==2.32.3
 + rich==13.8.1
 + rpds-py==0.20.0
 + scipy==1.13.1
 + smmap==5.0.1
 + sniffio==1.3.1
 + sounddevice==0.5.0
 + soundfile==0.12.1
 + soupsieve==2.6
 + tiktoken==0.7.0
 + tokenizers==0.19.1
 + tqdm==4.66.5
 + tree-sitter==0.21.3
 + tree-sitter-languages==1.10.2
 + typing-extensions==4.12.2
 + urllib3==2.2.3
 + wcwidth==0.2.13
 + yarl==1.11.1
 + zipp==3.20.2

aider-sandbox$ aider
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pydub/utils.py:300: SyntaxWarning: invalid escape sequence '\('
  m = re.match('([su]([0-9]{1,2})p?) \(([0-9]{1,2}) bit\)$', token)
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pydub/utils.py:301: SyntaxWarning: invalid escape sequence '\('
  m2 = re.match('([su]([0-9]{1,2})p?)( \(default\))?$', token)
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pydub/utils.py:310: SyntaxWarning: invalid escape sequence '\('
  elif re.match('(flt)p?( \(default\))?$', token):
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pydub/utils.py:314: SyntaxWarning: invalid escape sequence '\('
  elif re.match('(dbl)p?( \(default\))?$', token):
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pypandoc/pandoc_download.py:61: SyntaxWarning: invalid escape sequence '\.'
  regex = re.compile(r"/jgm/pandoc/releases/download/.*(?:"+processor_architecture+"|x86|mac).*\.(?:msi|deb|pkg)")

Notably when you run for the second time using the same python env, the pycache may cause the warnings to be suppressed. You can get around that by running the following after creating a fresh environment to shutoff bytecode. 

python -B $(which aider)

Notably, the master branch of the pydub repo actually works fine

Install aider-chat in a fresh environment then update pydub:

neal@neal-xps:aider-sandbox$ uv venv --python 3.12 --no-project --seed venv312
Using CPython 3.12.6
Creating virtual environment with seed packages at: venv312
 + pip==24.2
Activate with: source venv312/bin/activate
neal@neal-xps:aider-sandbox$ source venv312/bin/activate
(venv312) neal@neal-xps:aider-sandbox$ uv pip install aider-chat
...

neal@neal-xps:aider-sandbox$ uv pip install --upgrade --no-cache git+https://github.com/ngharrington/pydub.git
 Updated https://github.com/ngharrington/pydub.git (996cec4)
Resolved 1 package in 2.42s
   Built pydub @ git+https://github.com/ngharrington/pydub.git@996cec42e9621701edb83354232b2c0ca0121560
Prepared 1 package in 620ms
Uninstalled 1 package in 0.33ms
Installed 1 package in 0.67ms
 - pydub==0.25.1
 + pydub==0.25.1 (from git+https://github.com/ngharrington/pydub.git@996cec42e9621701edb83354232b2c0ca0121560)
(venv312) neal@neal-xps:aider-sandbox$ python -B $(which aider) 
/home/neal/workspace/aider-sandbox/venv312/lib/python3.12/site-packages/pypandoc/pandoc_download.py:61: SyntaxWarning: invalid escape sequence '\.'
  regex = re.compile(r"/jgm/pandoc/releases/download/.*(?:"+processor_architecture+"|x86|mac).*\.(?:msi|deb|pkg)")
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Aider v0.59.1
Main model: gpt-4o-2024-08-06 with diff edit format
Weak model: gpt-4o-mini
Git repo: .git with 2 files
Repo-map: using 1024 tokens, auto refresh
VSCode terminal detected, pretty output has been disabled.
Use /help <question> for help, run "aider --help" to see cmd line args
───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
>

As you see only the pandoc warning remains in this case.

ngharrington commented 1 month ago

I've added an issue here

https://github.com/jiaaro/pydub/issues/814

fry69 commented 1 month ago

If possible, do not install or update other Python modules in the same Python environment.

Please use venv or pipx to isolate aider as much as possible.

See also -> https://aider.chat/docs/troubleshooting/imports.html

github-actions[bot] commented 3 weeks ago

This issue has been labelled stale because it has been open for 2 weeks with no activity. Remove stale label or add a comment to keep this issue open. Otherwise, it will be closed in 7 days.

paul-gauthier commented 3 weeks ago

I can not reproduce this with a clean install in a fresh virtual environment with python 3.12.6. It seems this warning may be caused by installing aider with the wrong dependencies or in an unsupported way.

@Quidge appears to be using an aider installed with Homebrew. That is known to install the wrong dependencies.

uv isn't currently a supported way to install aider.

Can anyone reproduce this problem with one of the supported install methods?

$ python --version
Python 3.12.6

$ pip install aider-chat
Collecting aider-chat
  Downloading aider_chat-0.60.0-py3-none-any.whl.metadata (19 kB)
...

$ aider
───────────────────────────────────────────────────────────────────────────────────────────────
No git repo found, create one to track aider's changes (recommended)? (Y)es/(N)o [Yes]:
Git repository created in /Users/gauthier/tmp/issue1722
Aider v0.56.0
Main model: claude-3-5-sonnet-20240620 with diff edit format, infinite output
Weak model: claude-3-haiku-20240307
Git repo: .git with 0 files
Repo-map: using 1024 tokens, auto refresh
Use /help <question> for help, run "aider --help" to see cmd line args
Quidge commented 3 weeks ago

Hi! Yes, I would have very likely installed aider via Homebrew and I manage my projects with uv. I didn't realize there was a pipx advised installation option (I will likely use uvx cause I like the uv tool in the future, but I'll be sure to doublecheck with a pipx installation before raising another issue).

And sorry for the silence -- I'm dead center in the hurricane Helene affected area in the US and got knocked out of any personal project work for a month or so (lost internet for two weeks starting a day after posting, heh). Didn't mean to post here and ghost.

I still don't have time to retry where I was at with a pipx installation and verify if there are errors, and it looks like there won't be with ya'll being unable to replicate and my installation approach having known issues.

This can be closed if wanted. Thank you again for the patience. I absolutely love this project and am looking forward to continuing to use it. It's one of the few GH projects that I have new releases set to email me; I read all of the release notes!

paul-gauthier commented 3 weeks ago

FYI, I just bumped the aider dependencies to pypandoc==1.14.

The change is available in the main branch. You can get it by installing the latest version from github:

aider --install-main-branch

# or...

python -m pip install --upgrade git+https://github.com/paul-gauthier/aider.git

If you have a chance to try it, let me know if it works better for you.

gchamon commented 3 weeks ago

@paul-gauthier after updating system I started having this issue, but I can confirm that installing aider from the main branch fixes the issue:

uv tool install git+https://github.com/paul-gauthier/aider.git

I also use uv, it's orders of magnitude faster than pipx.

EDIT: while it works without the warnings and errors, it's unusable as it's hanging. I'll try later with pipx.

gchamon commented 2 weeks ago

Couldn't install aider-chat on arch, neither through pipx nor uvx. It just hangs there. I'm using docker to run aider:

docker build -t aider-chat - <<EOF                                                    
FROM python:3.12
WORKDIR /app
RUN pip install aider-chat
ENTRYPOINT ["aider"]
EOF

docker run --env ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY -it --rm -v $PWD:/app aider-chat
paul-gauthier commented 2 weeks ago

You can run aider directly from an existing docker image:

https://aider.chat/docs/install/docker.html

gchamon commented 2 weeks ago

Yeah that's what's going to work for me. Thanks!

gchamon commented 2 weeks ago

One thing, I see from the docs that the recommended way to map volume to docker in order to have aider access it is by -v $(pwd):/app which won't work for git subfolders. Maybe -v $(git rev-parse --show-toplevel):/app better resembles what aider does when executed in subfolders. Should I open an issue or a pull request with this change?

Here is my full alias:

alias aider='docker run \
  -it --rm \
  --user $(id -u):$(id -g) \
  --env ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY \
  -v $(git rev-parse --show-toplevel):/app \
  -v $HOME/.config/aider:/.config/aider:ro \
  -v $HOME/.aider:/.aider \
  -v $HOME/.gitconfig:/.gitconfig \
  aider-chat --config /.config/aider/config.yaml --no-auto-commits '

Putting -v $HOME/.gitconfig:/.gitconfig also fixes any problem with git config for repos

paul-gauthier commented 2 weeks ago

There are a lot of ways users may wish to map portions of their filesystem into the docker image. I'm not sure it's appropriate for the docs to try and recommend/anticipate all possible needs and variations.

gchamon commented 2 weeks ago

As a maintainer I can understand the sentiment. Documentation needs to be kept up to date and all information is a liability.

It's just that dockerized aider behaves differently than natively installed and that could be confusing, but if you think this will put too much burden on the docs it's alright