Improvements to metadata handling

miyuchina / mistletoe

A fast, extensible and spec-compliant Markdown parser in pure Python.

MIT License

791 stars 110 forks source link

Improvements to metadata handling #172

Closed anderskaplan closed 1 year ago

anderskaplan commented 1 year ago

This is an attempt to make the two places in the code base where metadata is handled -- the ASTRenderer and Token.repr -- work in a similar way, and most importantly: to display only selected attributes on the tokens (as specified in repr_attributes), instead of displaying everything it can find. The PR also includes a small bug fix for Token.repr.

This PR was split out from #169.

In more detail:

Updated the AST renderer to display the "official" attributes on the tokens (as specified in repr_attributes), instead of grabbing all attributes it can find.
Added footnotes to the repr_attributes on the Document token. This is the only attribute expected by the ASTRenderer test suite which wasn't already listed in repr_attributes.
Bug fix for Token.repr: only instance attributes should be included in repr, not class attributes. The class attributes are temporary and used during parsing. Affects headings.

anderskaplan commented 1 year ago

I should mention that this PR is a prereq for the Markdown renderer, too, because the Markdown renderer needs additional information stored on the tokens, and that information breaks the unit tests for the AST renderer and repr if you just add it straight away. It could be an option to add it to the expected output of the AST renderer and repr, of course, but I don't think it belongs there.

@pbodnar I know you're busy, but it would be great if you could find the time to review this PR anyway. It's a fairly small one.

anderskaplan commented 1 year ago

Also, do you think you could split the commit into 2, please? IMHO it would be more practical to have the fix of __repr__() method isolated in a separate commit in history (a separate PR is probably not needed though).

sure, I'll do that.

coveralls commented 1 year ago

Coverage: 89.129% (+0.07%) from 89.063% when pulling 9faff3c29760c3e72fd335dfa9a681838d9b158c on anderskaplan:repr-attributes into 6dc83f1e5c22d25f144804f2806afba9414c1579 on miyuchina:master.

pbodnar commented 1 year ago

@anderskaplan, all good now - merged - nothing except of available time hopefully blocking the implementation of the Markdown renderer. ;)

Note that I have marked this as a breaking-change, because we restrict the output of AST renderer now - but hopefully there will not be too many people relying on the previous "unrestricted" output...