pypa / readme_renderer

Safely render long_description/README files in Warehouse
Apache License 2.0
158 stars 88 forks source link

Retire `readme_renderer` / introduce breaking changes into its architecture. #269

Closed KOLANICH closed 1 year ago

KOLANICH commented 1 year ago

IMHO readme_renderer architecture has flaws. This library tries to be an abstraction layer (something high-level, wrapping other libs under convenient and similar interfaces), but it doesn't do the job of an abstraction layer well.

  1. An integrator has a goal to have the data converted, not to reinvent a wheel and do the conversion by hand himself. This lib admits it by wrapping other lower level libraries into functions having similar interfaces, but it doesn't go the full way. An integrator is still plagued by low-level details like deciding himself which submodule he has to import. For example very often format of a file is not known when the software is written, and has to be determined in runtime. This library architecture will cause the integrator to use either dynamic import (importlib.import_module, __import__), or an own dispatching table. It's awful integrator experince, because he has to reinvent the wheel doing the job other libs have to do. For example detecting file format by extension shouldn't be a job of the integrator.

  2. It has no redundancy and is not tolerant to absence of lower level libs. There are usually multiple libs providing the same functionality. By utilizing the libs already installed on end user's machine tools become "almost-zero-dependency" in the sense that installatiin of a lib requires no additional packages be installed.

  3. This lib has no autorouting. This severily limits usefullness of this lib. If one wants to convert for example a readme of a remote GitHub repo (this was my use case), that happenned to be in RST, to the fancy ANSI text displayable in CLI (my use case), one has to do rst -[readme_renderer]-> html -[markdownify]-> gfm -[mdv]-> gfm_ansi conversion. If pypandoc is present and current readme_renderer is not, the path should be rst -[pypandoc]-> gfm -[mdv]-> gfm_ansi.

So I have created a lib https://github.com/KOLANICH-libs/LMRender.py . It is not yet stable, not covered with tests, not documented and currently I haven't even fixed all the issues I know in it, but I guess it can become the replacement to readme_renderer once.

  1. It is designed to be smol.

  2. It is designed to allow a user to specify what he wants to get, not how exactly to do it.

  3. It is designed to be moderately tweakable. An integrator can avoid using the unneeded backends, rebalance the priorities of the existing ones, and add own backends by just constructing an own instance of Renderer. A more careful tweaking of backend priorities can be done by subclassing the Renderer and redefining the method building the adjacency matrix.

  4. Since is designed to require no mandatory dependencies, it is also designed to help projects integrating that lib to provide useful hints to end users about what packages they should install to get the needed functionality. (Currently this feature was broken when I was refactoring my lib).

  5. It provides a function (SupportedFormat.fromFileName) to guess which source format is stored within a file by its extension.

So basically the pipeline for doing the conversion the following:

from LMRender import SupportedFormat as SF, DEFAULT_RENDERER

srcFileName: PurePath = getTheSourceFileName()  # integrator's
src: str = getTheSource(srcFileName)  # integrator's
targetFormat: SF = getTargetFormat()  # integrator's
fmt: SF = SF.fromFileName(srcFileName)
depsToInstall: typing.Iterable[str] = DEFAULT_RENDERER.isSourceUnavailable(fmt)
printMessageToAUserIfNeeded(depsToInstall)  # integrator's
rendered: str = DEFAULT_RENDERER.render(src, fmt, targetFormat)
displayRenderedToUser(rendered)  # integrator's

No low-level issues are touched in this code.

miketheman commented 1 year ago

Hello! Thanks for your comments - I'm not entirely certain that this is the right place to advertise your new library.

The beautiful part about the open source ecosystem is that you can absolutely produce your library and see if folks like it!

In the meantime, we're likely to continue to update and maintain this library is it's currently structured, so that we don't break any external user experiences who trust the current behavior of this library.