Can't plot a comparison word cloud on grouped DFM with TF-IDF weighting

[tomwagstaff-opml]

Describe the bug

I have a DFM which I want to group using one of my docvars, and then create a comparative word cloud. This works fine for simple counts. However, when I apply TF-IDF weighting after grouping, plotting a word cloud with comparative = TRUE gives the following error:

Error: will not weight a dfm already term-weighted as 'count'; use force = TRUE to override

(Plotting a normal word cloud with comparative = FALSE works fine.)

This is the same error message reported in this issue and seems to raise the same philosophical issues. I think what I'm doing makes perfect sense - doing TF-IDF treating each of my broad groups as an individual document. Regardless of that, the technical fix applied to textstat_frequency in that case doesn't seem to have carried over to textplot_wordcloud.

Reproducible code

my_dfm <- my_corpus %>%
  tokens() %>%
  dfm() %>%
  dfm_group(groups = my_docvar) %>%
  dfm_tfidf()

my_dfm %>%
  textplot_wordcloud(comparison = TRUE)

Expected behavior

I wanted to see a word cloud, split by group, with font size reflecting the TF-IDF score of the words.

System information

R version 4.2.3 (2023-03-15 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19045)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8   
[3] LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.utf8    

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods   base     

other attached packages:
 [1] quanteda.textplots_0.94.3 quanteda_3.3.1            rsample_1.1.1            
 [4] readtext_0.90             lubridate_1.9.2           forcats_1.0.0            
 [7] stringr_1.5.0             dplyr_1.1.2               purrr_1.0.2              
[10] readr_2.1.4               tidyr_1.3.0               tibble_3.2.1             
[13] ggplot2_3.4.3             tidyverse_2.0.0          

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.11         RColorBrewer_1.1-3  pillar_1.9.0        compiler_4.2.3     
 [5] stopwords_2.3       tools_4.2.3         digest_0.6.33       lattice_0.20-45    
 [9] lifecycle_1.0.3     gtable_0.3.4        timechange_0.2.0    pkgconfig_2.0.3    
[13] rlang_1.1.1         fastmatch_1.1-4     Matrix_1.5-3        cli_3.6.1          
[17] rstudioapi_0.15.0   parallel_4.2.3      xml2_1.3.5          furrr_0.3.1        
[21] withr_2.5.0         httr_1.4.7          globals_0.16.2      generics_0.1.3     
[25] vctrs_0.6.3         hms_1.1.3           grid_4.2.3          tidyselect_1.2.0   
[29] glue_1.6.2          data.table_1.14.8   listenv_0.9.0       R6_2.5.1           
[33] parallelly_1.36.0   fansi_1.0.4         tzdb_0.4.0          magrittr_2.0.3     
[37] ISOcodes_2022.09.29 codetools_0.2-19    scales_1.2.1        future_1.33.0      
[41] colorspace_2.1-0    renv_0.17.3         utf8_1.2.3          stringi_1.7.12     
[45] RcppParallel_5.1.7  munsell_0.5.0 

Additional info

[kbenoit]

Here's a workaround, which removes the weighted flag. Note that you have many zeroed features if you are applying this to the grouped dfm, since the inverse document frequency weight will zero the features that occur in both groups. (Did you want to weight before applying dfm_group()?)

library("quanteda")
#> Package version: 3.3.1
#> Unicode version: 14.0
#> ICU version: 71.1
#> Parallel computing: 12 of 12 threads used.
#> See https://quanteda.io for tutorials and examples.
library("quanteda.textplots")

my_dfm <- data_corpus_inaugural[54:59] %>%
    tokens() %>%
    dfm() %>%
    dfm_group(groups = Party) %>%
    dfm_tfidf()

# drops the "weighted" flag
my_dfm <- as.dfm(as.matrix(my_dfm))
my_dfm
#> Document-feature matrix of: 2 documents, 2,320 features (65.56% sparse) and 0 docvars.
#>             features
#> docs         president clinton , distinguished guests and my fellow citizens
#>   Democratic         0 0       0             0      0   0  0      0        0
#>   Republican         0 1.20412 0             0      0   0  0      0        0
#>             features
#> docs         the
#>   Democratic   0
#>   Republican   0
#> [ reached max_nfeat ... 2,310 more features ]

# necessary to remove zero-weighted featurs
my_dfm <- dfm_trim(my_dfm)

textplot_wordcloud(my_dfm, comparison = TRUE, min_count = 0)

^{Created on 2023-08-23 with reprex v2.0.2}

[tomwagstaff-opml]

Wow, thank you so much for the lightning fast response @kbenoit! I'm very happy with this workaround, and the results look sensible to me.

On your question about the order of operations - I think this way round makes sense to me - I'm basically treating each group as one big document and then running TF-IDF weighting on that (I agreed with the comments you were making in the other issue about doing the operations the other way around not making a whole lot of sense, or at least not having a clear interpretation).

I did notice a lot of zero weighted terms coming through - I don't fully understand why the IDF should wipe them out completely, I would have thought it would have just down-weighted features that occur in multiple groups. If you're able to explain why that happens that would be great, but you've already been a huge help - thanks again!

[kbenoit]

It's because you are multiplying by the log of inverse document frequency, which for words that occur in both groups, would be log(2/2) = 0. tf-idf will always zero out features that occur across all documents, regardless of how imbalanced are those features.

See p485 of https://kenbenoit.net/pdfs/CURINI_FRANZESE_Ch26.pdf where I wrote more about this "feature" of tf-idf.

[tomwagstaff-opml]

Gotcha! One of those things that's obvious once it's explained...

Since, all of this is by way of exploratory analysis, and the final product (a topic model) will proceed from the individual documents anyway, I think I'll just bowl on and treat this as a feature rather than a bug of TF-IDF, since I'm going to be keeping an eye on the unadjusted counts as well.

Thanks again for all of your help, I would have been a bit lost without it!