vi3k6i5 / flashtext

Extract Keywords from sentence or Replace keywords in sentences.
MIT License
5.57k stars 599 forks source link

add_keywords_from_list when every element in a list has extra information #46

Open suchee opened 6 years ago

suchee commented 6 years ago

is there a way to add keywords with extra information from a list efficiently? I am currently looping and adding each of the keyword with the "add_keyword", but loading the data seems to be taking a non trivial amount of time.

vi3k6i5 commented 6 years ago

Can you give a small sample example, how the data will be like?

PS: there are many methods to load including from dictionary, file, list.. I am assuming you have gone through them and they are not satisfying your use case.

suchee commented 6 years ago

for instance, in your documentation I can add a key word with extra information with :

kp.add_keyword('Delhi', ('Location', 'Delhi'))

while for a list : keyword_processor.add_keywords_from_list(["Delhi", "Bombay"])

What I want is: a way where I can use add_keywords_from_list but where each element also has the category information. I hope it is clear? Currently what I do is run through my list and do something like kp.add_keyword('Delhi', ('Location', 'Delhi')) individually. I just wonder if it can be done more efficiently?

Perhaps: keyword_processor.add_keywords_from_list([('Delhi', ('Location', 'Delhi')), ('Bombay', ('Location', 'Bombay'))])

vi3k6i5 commented 6 years ago

Got it.

So even if you pass the whole thing in a bulk, like a dictionary or a list of lists, it will still be iterated internally. So it will take the same amount of time.

How many terms do you have ?

I am just curious how much time is going in initialisation?