search

piskvorky / gensim

Topic Modelling for Humans
https://radimrehurek.com/gensim
GNU Lesser General Public License v2.1
15.55k stars 4.37k forks source link

Distributed LDA Connection refused - Dispatcher host is always 127.0.0.1 #2710

Open arshad115 opened 4 years ago

arshad115 commented 4 years ago

Problem description

I am trying to run the distributed LDA on multiple machines. Following this tutorial: https://radimrehurek.com/gensim/models/lda_worker.html

Setup is working but once I run the distributed lda I get the following error: error

Dispatcher worker recognizes that there are lda-workers available but they cannot connect to the dispatcher.

gensim

The dispatcher host is always 127.0.0.1

I have also tried setting the host ip on the dispatcher but no luck

python -m gensim.models.lda_dispatcher --host HOST_IP &

Nameserver is recognized on other machines and shows the correct ip.

borice commented 3 years ago

Have you found a way to set the lda_dispatcher host to anything other than 127.0.0.1 ?

borice commented 3 years ago

Nevermind - right after I wrote the above I found the solution. I had to start the name server using the --host argument. Once I did that, the Pyro4 listing showed the dispatcher with the correct host ... and everything worked.

python -m Pyro4.naming --host 10.128.3.195 --port 9090

(I don't know if it would've made a difference, but I ran the dispatcher and nameserver on the same host)