search

nateraw / Lda2vec-Tensorflow

Tensorflow 1.5 implementation of Chris Moody's Lda2vec, adapted from @meereeum
MIT License
107 stars 40 forks source link

Dependency on sense2vec #21

Closed svirgilgood closed 5 years ago

svirgilgood commented 5 years ago

I couldn't get sense2vec to build on my system. I haven't dug into the code to see what exactly it is contributing, but how important is it to the process of lda2vec? After the changes I made to the imports list, I commented out the sense2vec nlp appends, and I was able to get the code to import.

What exactly is the function of sense2vec? (I have barely started working on the lda2vec code, I'm not even close to digging into sense2vec!) Is this an important dependency? Or is it something we could get the code from something else?

isubasinghe commented 5 years ago

Yeah I have a similar problem, install sense2vec broke everything on my system. I have been trying to get lda2vec working for the past 2 days.

This implementation seems the most promising, but even this is broken.

@nateraw Do you think you could dockerize your environment please? That would very very helpful, if you could.

nateraw commented 5 years ago

Hey guys, Sense2Vec is not required to use this. So, if you can get everything else to install without Sense2vec, just make sure you don't pass the parameter merge=True when calling to the preprocessing helper. Sense2Vec is from Spacy, and it is for merging noun phrases. So, if you are okay with not having noun phrase merging, Sense2Vec is not required. If you want to do noun phrase merging still, I suggest preprocessing the data yourself. Hope this helps.

nateraw commented 5 years ago

Also...you may have to just delete the import call to sense2vec in the lda2vec.py file. Other than that, you should be all good

svirgilgood commented 5 years ago

I forked the code and commented or deleted all the references to the merge functions of sense2vec here: https://github.com/e2dubba/Lda2vec-Tensorflow

Thanks for the reply!

nateraw commented 5 years ago

Awesome! Looks great. Just a note - sense2vec is a pain to install as it is, but is apparently impossible to install on Mojave due to some weird C library changes. So, if you're on Mojave, that might be why you're running into problems