Open t3rn0 opened 1 year ago
anyway i can support with this ticket? i stumbled upon your lib today and it does exactly what i need but this bug is causing me small issues :/
example:
def __init__(self, **kwargs: Any):
"""Initialize a MarkdownTextSplitter."""
separators = [
# First, try to split along class definitions
"\nclass ",
"\ndef ",
"\n\tdef ",
# Now split by the normal type of lines
"\n\n",
"\n",
" ",
"",
]
super().__init__(separators=separators, **kwargs)
becomes this:
def __init__(self, **kwargs: Any):
"""Initialize a MarkdownTextSplitter."""
# First, try to split along class definitions
# Now split by the normal type of lines
separators = ['\nclass ', '\ndef ', '\n\tdef ', '\n\n', '\n', ' ', '']
super().__init__(separators=separators, **kwargs)
Thanks for noticing that. The example you gave is not relevant to the issue, so I opened another one https://github.com/t3rn0/ast-comments/issues/13 I don't have enough time to look at it right now but I'll try to fix it next month
Any progress here? Not urgent for me as this lib already does everything I need but I just wanted to add a gentle nudge to continue developing it as it's an amazing tool for anyone who delved a bit too deep into AST.
Big thanks for making this effort and I hope this library gets a lot more popular!
Hello. Thank you for your interest in the library.
The original problem here was resolved a few commits ago. Almost...
There is one element left: comments for match
blocks.
Now this issue serves as a kind reminder to me that there is still work to be done. 😁
Currently we don't identify inlined comments in code, all comments are just regular (not inlined) (https://github.com/t3rn0/ast-comments#notes). Difference between them is lost after the code is parsed into the tree. We should try distinguish inlined and regular comments when unparsing the tree back to the code.
It'd be great to cover following cases: