Determine and document the _minimum_ Python language version supported by FawltyDeps

[jherland]

Even though FawltyDeps now needs Python >=v3.8 to run (after https://github.com/tweag/FawltyDeps/pull/459 is merged), we pride ourselves on FawltyDeps being able to analyze projects that are stuck on older Python versions.

However, I don't think we've ever clearly stated what is the minimum Python version that we support analyzing. AFAICS these are the aspects of a project that is likely to determine what minimum version we support:

1. Python syntax: The syntax of the Python language changes from one version to the next, and in order to be able to extract imports (and dependencies from setup.py files), we need to be able to parse the Python syntax used in the project being analyzed.
2. Old dependency declaration formats: If older projects use different ways of declaring dependencies, we need to support these formats in order to support projects using these formats.
3. Obsolete Python environments: If older projects use different ways of setting up Python environments and installing dependencies, we would have to understand these, unless we want to rely on custom mappings (or the identity mapping).

Here are the bounds that I'm aware of:

• AFAIK, we have never intended to support Python 2 projects. The Python 2 syntax is sufficiently different, and its use is so thoroughly discouraged and unsupported by now, that I don't see any good reason to change this.
• I'm not aware of any old dependency declaration formats that we don't support. I believe the use of requirements.txt files and/or setup.py files was already established by the time the Python 2 -> 3 transition took place.
• The same goes for Python environments:
  • Virtualenvs existed as a concept before the Python 2 -> 3 transition (although it relied on the 3rd-party virtualenv tool, as the stdlib venv module was not added until Python 3.3).
  • Conda was also a thing in the Python 2 times, but we've recently added support for this.
  • AFAICS this leaves installation of dependencies at the system or user level (e.g. pip install ... or pip install --user ... outside any virtualenv).
  • We support all of these, and I'm not aware of any variations on these in the early Python 3 era that should not already work.

That leaves, I believe, Python syntax as the main determining factor for what minimum version we support. Since we use the ast module to parse Python code, this question essentially becomes: For a given version of the ast parser that FawltyDeps is running on, what is the oldest version of Python syntax that is supported by this parser?

Here is what different versions of ast.parse() has to say about supported Python versions:

• v3.13:

  Setting feature_version to a tuple (major, minor) will result in a “best-effort” attempt to parse using that Python version’s grammar. For example, setting feature_version=(3, 9) will attempt to disallow parsing of match statements. Currently major must equal to 3. The lowest supported version is (3, 7) (and this may increase in future Python versions); the highest is sys.version_info[0:2]. “Best-effort” attempt means there is no guarantee that the parse (or success of the parse) is the same as when run on the Python version corresponding to feature_version.

• v3.12

  ... The lowest supported version is (3, 4) ...

• v3.11 (same as v3.12)

• v3.10

  Also, setting feature_version to a tuple (major, minor) will attempt to parse using that Python version’s grammar. Currently major must equal to 3. For example, setting feature_version=(3, 4) will allow the use of async and await as variable names. The lowest supported version is (3, 4); the highest is sys.version_info[0:2].

• v3.9 (same as v3.10)

• v3.8 (same as v3.9)

• (for completeness, the feature_version flag was added to ast.parse() in Python v3.8, so we would not have any way to control this if we kept support for running on Python v3.7)

AFAICS, when the feature_version flag is not set, it defaults to the current version, i.e. the one FawltyDeps is running on. We're currently not setting this flag, which means that any older project that we attempt to analyize will fail if it uses syntax that is not forward-compatible to the version FawltyDeps is running on.

FWIW, the feature_version flag to ast.parse() was added in Python v3.8, so I don't believe dropping support for running FD on Python v3.7 (see #459) has changed what older Python projects we might support.

---

Taking a step back, I suspect this is not a huge problem in practice, as Python tries very hard not to break old code when new syntax is introduced. The biggest problem - I assume - is the introduction of new keywords (e.g. async, await, match) that might have been used as e.g. variable names in older code.

Still, if we want to tackle this, we need a suite of tests with "old" Python source code samples that is incompatible with various "new" Python versions. We also need some way for projects to configure FawltyDeps as to which feature_version they need to use.

In summary, I see three main ways we can approach this:

1. Document the current behavior: The Python syntax supported by FawltyDeps must be compatible with the Python version that FawltyDeps is running on. Meaning that if you're analyzing an old project, you should be running FawltyDeps on Python v3.8, in order to minimize the chance of encountering incompatible syntax.
2. Add a settings flag to specify the Python version used by the project. This will be passed as feature_version to ast.parse(), and requires that FawltyDeps is running on a version that supports the given version number. That is, if your project is using Python version v3.4-v3.6, you must run FawltyDeps on Python <v3.13; if your project is using Python version <v3.4, you're out of luck.
3. Use a 3rd-party Python parser like e.g. parso that supports parsing a wider range of Python version (even Python v2!).