search

sashafrey / topicmod

This project had been moved to https://github.com/bigartm/bigartm
Other
0 stars 0 forks source link

Add option in perplexity calculation to use collection unigram model when p(w|d)=0 #69

Closed sashafrey closed 10 years ago

sashafrey commented 10 years ago

After https://github.com/sashafrey/topicmod/commit/be50848f4e41956366c351ab28c013667257447d perplexity calculation uses document unigram model. This is an "optimistic" estimate. An option to use collection unigram model will give a "pesimistic" estimate, which is better.

sashafrey commented 10 years ago

Fixed by https://github.com/sashafrey/topicmod/commit/189c9b4f808e313a37e6e5e7826c0f37850fef7c