search

vi3k6i5 / flashtext

Extract Keywords from sentence or Replace keywords in sentences.
MIT License
5.57k stars 599 forks source link

Replace with Nothing #74

Closed nh111 closed 5 years ago

nh111 commented 5 years ago

Hi, This is very useful. for below case do we have any option? I want to replace one particular word with 'Nothing'/null/blank. currently if i give space it is working.

below is not working, (I just want to replace new delhi with empty):

keyword_processor.add_keyword('New Delhi', '') new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.') new_sentence

below is working with Space:

keyword_processor.add_keyword('New Delhi', ' ') new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.') new_sentence

silvererudite commented 5 years ago

Hi, I would like to work on this issue but the example above is a bit unclear to me. Can anyone please clarify the comment above or brief me how should I possibly approach the problem?

nh111 commented 5 years ago

Hi, above comment, you are not able to understand? or the approach you want me to explain. Just confirm,

I also made change to below line in my actual question, see if you are able to understand keyword_processor.add_keyword('New Delhi', '')

silvererudite commented 5 years ago

Okay..now I understand the problem here. Do you have any possible suggestions as to how to approach this problem?

vi3k6i5 commented 5 years ago

As a workaround you can try:

keyword_processor.add_keyword('New Delhi', 'EMPTY') new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.').replace('EMPTY', '')

This is a hack though and not a proper solution.

On Thu, Feb 7, 2019 at 10:51 AM silvererudite notifications@github.com wrote:

Okay..now I understand the problem here. Do you have any possible suggestions as to how to approach this problem?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/vi3k6i5/flashtext/issues/74#issuecomment-461292015, or mute the thread https://github.com/notifications/unsubscribe-auth/AC-NwoIUsNK7-PE9EPpQ7AzI8X6sEYVjks5vK7fCgaJpZM4Z677r .

--

Vikash

nh111 commented 5 years ago

As a workaround you can try: keyword_processor.add_keyword('New Delhi', 'EMPTY') new_sentence = keyword_processor.replace_keywords('I love Big Apple and new delhi.').replace('EMPTY', '') This is a hack though and not a proper solution. … On Thu, Feb 7, 2019 at 10:51 AM silvererudite @.***> wrote: Okay..now I understand the problem here. Do you have any possible suggestions as to how to approach this problem? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#74 (comment)>, or mute the thread https://github.com/notifications/unsubscribe-auth/AC-NwoIUsNK7-PE9EPpQ7AzI8X6sEYVjks5vK7fCgaJpZM4Z677r . -- Vikash

Thank you very much for your solution and reply

brlala commented 5 years ago

Need to remove breaking words such as is/are before feeding it to the NLP. This is still not working without the "hack".

Would love to be able to replace with Nothing in the use case above. "can I have my lunch" -> "have lunch"

wangpeipei90 commented 5 years ago

@brlala The solution might be mapping all these breaking words into the same value and using string replace and strip function.

keyword_processor = KeywordProcessor() keyword_processor.add_keyword('is','EMPTY') keyword_processor.add_keyword('are','EMPTY') keyword_processor.add_keyword('can','EMPTY') keyword_processor.add_keyword('I','EMPTY')

new_sentence = keyword_processor.replace_keywords('can I have lunch') new_sentence.replace('__EMPTY','').strip()