cecehoush / spiderbyte-website

0 stars 0 forks source link

Model: NLP Tokenization of titles and descriptions #33

Closed vlvcDev closed 1 month ago

vlvcDev commented 1 month ago

We need to use NLP to tokenize titles and descriptions of challenges for our future similarity score system. This means words like 'heavily' will turn into 'heavy' so that it's easier to compare their similarity. We should be able to use spaCy for this. After this step, we can push challenge data into a vector DB.