ponder-lab / GitHub-Issue-Classifier

Python script to mine for GitHub issues + comments and classify them.
MIT License
6 stars 0 forks source link

fix: proper CODE tokenization #33

Closed y3pio closed 3 years ago

y3pio commented 3 years ago

Fixes: #32

Fixing proper code tokenization to address unexpected undefined line error in multi line code block being processed. We were not properly tokenizing CODE and certain multi line code comments were getting processed line by line, causing the error seen in #32 to be thrown.