refactor: upgrade repo

[jeanbmar]

The repo is currently broken, this MR fixes it:

• Upgrade all dependencies
• Upgrade project structure to comply with Poetry expectations
• Manage all dependencies through Poetry and get rid of requirements.txt
• Manage gpt-migrate execution through Poetry as well using poetry run gpt-migrate
• Use relative path for internal file reading
• Add a maxtokens options to allow models using only 4096 tokens
• Prevent gpt-migrate from crashing when model response content contains Python None
• Prevent gpt-migrate from crashing when the project to convert contains nested directories
• Use openai models by default

[jeanbmar]

Fixing:

• 50

• 48

• 46

• 45

• 43 (use openai as default)

• 42

• 41

[donfn]

Your PR seems to work, thanks a lot! Although I'm not quite familiar with poetry, I had to mv ./src/gpt_migrate/ ./gpt_migrate, because it wouldn't recognize gpt-migrate as an internal package.

[natea]

Yeah, I encountered the same issue when I tried to run main.py:

% python main.py --model gpt-4-32k --targetlang nodejs
Traceback (most recent call last):
  File "/Users/nateaune/Documents/code/gpt-migrate-fixed/src/gpt_migrate/main.py", line 7, in <module>
    from gpt_migrate.ai import AI
ModuleNotFoundError: No module named 'gpt_migrate'

This occurred after poetry complained about gpt_migrate not being a package:

% poetry install
Installing dependencies from lock file

Package operations: 13 installs, 31 updates, 0 removals

  - Downgrading typing-extensions (4.12.2 -> 4.10.0)
  - Installing annotated-types (0.6.0)
  - Downgrading certifi (2024.8.30 -> 2024.2.2)
  - Downgrading charset-normalizer (3.4.0 -> 3.3.2)
  - Downgrading exceptiongroup (1.2.2 -> 1.2.0)
  - Downgrading idna (3.10 -> 3.6)
  - Downgrading pydantic-core (2.27.0 -> 2.16.3)
  - Downgrading urllib3 (2.2.3 -> 2.2.1)
  - Downgrading filelock (3.16.1 -> 3.13.1)
  - Downgrading anyio (4.6.2.post1 -> 4.3.0)
  - Downgrading frozenlist (1.5.0 -> 1.4.1)
  - Downgrading fsspec (2024.10.0 -> 2024.2.0)
  - Downgrading httpcore (1.0.7 -> 1.0.4)
  - Installing jsonpointer (2.4)
  - Downgrading multidict (6.1.0 -> 6.0.5)
  - Installing orjson (3.9.15)
  - Downgrading packaging (24.2 -> 23.2)
  - Updating pydantic (1.10.8 -> 2.6.3)
  - Downgrading pyyaml (6.0.2 -> 6.0.1)
  - Downgrading requests (2.32.3 -> 2.31.0)
  - Downgrading tqdm (4.67.0 -> 4.66.2)
  - Downgrading attrs (24.2.0 -> 23.2.0)
  - Downgrading httpx (0.27.2 -> 0.27.0)
  - Downgrading huggingface-hub (0.26.2 -> 0.21.3)
  - Installing jsonpatch (1.33)
  - Updating langsmith (0.0.92 -> 0.1.10)
  - Installing markupsafe (2.1.5)
  - Installing regex (2023.12.25)
  - Downgrading tenacity (8.5.0 -> 8.2.3)
  - Downgrading yarl (1.17.1 -> 1.9.4)
  - Installing zipp (3.17.0)
  - Downgrading aiohttp (3.11.2 -> 3.9.3)
  - Installing importlib-metadata (7.0.1)
  - Installing jinja2 (3.1.3)
  - Installing langchain-core (0.1.27)
  - Updating openai (0.27.10 -> 1.13.3)
  - Downgrading termcolor (2.5.0 -> 2.4.0)
  - Installing tiktoken (0.6.0)
  - Downgrading tokenizers (0.20.3 -> 0.15.2)
  - Installing langchain-openai (0.0.8)
  - Updating litellm (0.1.213 -> 1.28.0)
  - Installing ruff (0.2.2)
  - Downgrading typer (0.9.4 -> 0.9.0)
  - Updating yaspin (2.5.0 -> 3.0.1)

Installing the current project: gpt-migrate (0.1.0)
gpt_migrate is not a package.