doc-topic distr.

[mhbodell]

Outputen sparad av "save_doc_theta_estimate = true" har fel dimensioner och uutputen visar inte heller proportioner utan counts.

Detta står i README.txt-filen:

Save the a file with document topic theta estimates (will not include zeros)

Unlike Phi means which are sampled with thinning, theta means is just a simple

average of the topic counts in the last iteration divided by the number of

tokens in the document thus there is not theta_burnin or theta_thinning

save_doc_theta_estimate = true doc_topic_theta_filename = doc_topic_theta.csv

Har en model med 200 ämnen men doc_theta_means filen har 400 kolumner och antal dokument som rader? Varför är antalet kolumner dubbla antalet ämnen i modellen?

Config-file:

configs = Spalias no_runs = 1

[Spalias] title = PCPLDA description = 200 topics with alpha 0.2 and extended priorlist dataset = data/fb_politics_news.txt scheme = spalias_priors seed = 1904 topics = 200 alpha = 0.2 beta = 0.01 iterations = 1500 rare_threshold = 0 batches = 4 topic_batches = 4 topic_interval = 500 start_diagnostic = 200 debug = 0

log_type_topic_density = true

log_document_density = true log_phi_density = true phi_mean_filename = phi-mean.csv phi_mean_burnin = 20 phi_mean_thin = 5 stoplist = nsc-test/PartiallyCollapsedLDA-8.4.0/stoplist-empty.txt save_vocabulary = true vocabulary_filename = lda_vocab.txt topic_prior_filename = wfw/bash/priors/k200_v7.txt keep_connecting_punctuation = true log_topic_indicators = true save_sampler = false save_doc_theta_estimate = true doc_topic_theta_filename = doc_topic_theta.csv save_phi_mean = true

Jag bifogar en bild av delar av outputen så du ser hur den ser ut.

[rebeckahw]

The problem seems to stem from WriteASCIIDoubleMatrix. Decimal numbers are written with commas both as decimal separators and column separators. This adds an extra column for each printed value and every other column gets the value 0.

[lejon]

Yes, I noticed this bug also, and have a fix in 9.2.0, for parts of the problem, but will have to double check if this is also solved with that fix...

[lejon]

9.2.0 should solve this problem

[rebeckahw]

The test for WriteASCIIDoubleMatrix now passes, but the problem unfortunately remains for me. It could maybe? be caused by the method formatDouble in LDAUtils.java:

        String formatString = "%." + noDigits + "f";
        return String.format(formatString, d);

since String.format() depends on defaultLocale (which for me is SE)

[lejon]

Yes, it is due to locale and it is a bit of a mess now unfortunately, the combination of Locale and possibility of selecting separator makes it complicated... I'll have a look and see if I can re-design to a better solution.