search

jeff1evesque / ist-687

Syracuse IST687 final project with Jesse Warren (team member)
2 stars 0 forks source link

Further remove unicode entries from 'Article' #62

Closed jeff1evesque closed 6 years ago

jeff1evesque commented 6 years ago

Some of our entries from our Article column were nonstandard characters, not representing the actual characters native to the corresponding language. This may be a limitation of windows character set. However, we need to properly determine these corresponding entries, to determine whether they should be removed, or retained in the dataframe.

jeff1evesque commented 6 years ago

We were able to manually create csv files from each segmented dataframe representation:

write.table(
  average_top10.m,
  file = 'top10.csv',
  append = FALSE,
  quote = TRUE,
  sep = ' ',
  eol = '\n',
  na = 'NA',
  dec = '.',
  row.names = TRUE,
  col.names = TRUE,
  qmethod = c('escape', 'double'),
  fileEncoding = ''
)

However, we needed to open the corresponding file within notepad. Otherwise the UTF8 representation, will not be in the native character set of the corresponding language:

"Article" "Language" "Access" "variable" "value"
"1" "Заглавная_страница" "ru" "all-access" "2015.07" 39195116
"2" "Заглавная_страница" "ru" "desktop" "2015.07" 18317353
"3" "Заглавная_страница" "ru" "mobile-web" "2015.07" 19462802
"4" "メインページ" "ja" "all-access" "2015.07" 12007556
"5" "メインページ" "ja" "desktop" "2015.07" 7835865
"6" "Служебная:Поиск" "ru" "all-access" "2015.07" 5075981
"7" "Служебная:Поиск" "ru" "desktop" "2015.07" 4798252
"8" "Donald_Trump" "en" "all-access" "2015.07" 2125559
"9" "XHamster" "en" "all-access" "2015.07" 92189
"10" "特別:検索" "ja" "all-access" "2015.07" 3562492
"11" "Заглавная_страница" "ru" "all-access" "2015.08" 43928436
"12" "Заглавная_страница" "ru" "desktop" "2015.08" 22420437
"13" "Заглавная_страница" "ru" "mobile-web" "2015.08" 20292928
"14" "メインページ" "ja" "all-access" "2015.08" 11848960
"15" "メインページ" "ja" "desktop" "2015.08" 7623353
"16" "Служебная:Поиск" "ru" "all-access" "2015.08" 5687670
"17" "Служебная:Поиск" "ru" "desktop" "2015.08" 5428867
"18" "Donald_Trump" "en" "all-access" "2015.08" 2896610
"19" "XHamster" "en" "all-access" "2015.08" 77452
"20" "特別:検索" "ja" "all-access" "2015.08" 4235434
"21" "Заглавная_страница" "ru" "all-access" "2015.09" 43842621
"22" "Заглавная_страница" "ru" "desktop" "2015.09" 21951052
"23" "Заглавная_страница" "ru" "mobile-web" "2015.09" 20914206
"24" "メインページ" "ja" "all-access" "2015.09" 11697789
"25" "メインページ" "ja" "desktop" "2015.09" 7637508
"26" "Служебная:Поиск" "ru" "all-access" "2015.09" 6458467
"27" "Служебная:Поиск" "ru" "desktop" "2015.09" 6162428
"28" "Donald_Trump" "en" "all-access" "2015.09" 2031231
"29" "XHamster" "en" "all-access" "2015.09" 82311
"30" "特別:検索" "ja" "all-access" "2015.09" 3771139
"31" "Заглавная_страница" "ru" "all-access" "2015.10" 47095921
"32" "Заглавная_страница" "ru" "desktop" "2015.10" 24657339
"33" "Заглавная_страница" "ru" "mobile-web" "2015.10" 21415510
"34" "メインページ" "ja" "all-access" "2015.10" 12051516
"35" "メインページ" "ja" "desktop" "2015.10" 7803228
"36" "Служебная:Поиск" "ru" "all-access" "2015.10" 6965280
"37" "Служебная:Поиск" "ru" "desktop" "2015.10" 6658177
"38" "Donald_Trump" "en" "all-access" "2015.10" 1204432
"39" "XHamster" "en" "all-access" "2015.10" 93494
"40" "特別:検索" "ja" "all-access" "2015.10" 3916872
"41" "Заглавная_страница" "ru" "all-access" "2015.11" 49634704
"42" "Заглавная_страница" "ru" "desktop" "2015.11" 27302012
"43" "Заглавная_страница" "ru" "mobile-web" "2015.11" 21280496
"44" "メインページ" "ja" "all-access" "2015.11" 11132801
"45" "メインページ" "ja" "desktop" "2015.11" 7047351
"46" "Служебная:Поиск" "ru" "all-access" "2015.11" 7013511
"47" "Служебная:Поиск" "ru" "desktop" "2015.11" 6711674
"48" "Donald_Trump" "en" "all-access" "2015.11" 1198269
"49" "XHamster" "en" "all-access" "2015.11" 84781
"50" "特別:検索" "ja" "all-access" "2015.11" 3855024
"51" "Заглавная_страница" "ru" "all-access" "2015.12" 46138639
"52" "Заглавная_страница" "ru" "desktop" "2015.12" 23582956
"53" "Заглавная_страница" "ru" "mobile-web" "2015.12" 21500700
"54" "メインページ" "ja" "all-access" "2015.12" 11332438
"55" "メインページ" "ja" "desktop" "2015.12" 7147768
"56" "Служебная:Поиск" "ru" "all-access" "2015.12" 6752230
"57" "Служебная:Поиск" "ru" "desktop" "2015.12" 6457536
"58" "Donald_Trump" "en" "all-access" "2015.12" 2611070
"59" "XHamster" "en" "all-access" "2015.12" 78977
"60" "特別:検索" "ja" "all-access" "2015.12" 3560172
"61" "Заглавная_страница" "ru" "all-access" "2016.01" 45605707
"62" "Заглавная_страница" "ru" "desktop" "2016.01" 22699098
"63" "Заглавная_страница" "ru" "mobile-web" "2016.01" 21783777
"64" "メインページ" "ja" "all-access" "2016.01" 11508317
"65" "メインページ" "ja" "desktop" "2016.01" 7095106
"66" "Служебная:Поиск" "ru" "all-access" "2016.01" 6841623
"67" "Служебная:Поиск" "ru" "desktop" "2016.01" 6537448
"68" "Donald_Trump" "en" "all-access" "2016.01" 2394948
"69" "XHamster" "en" "all-access" "2016.01" 5176941
"70" "特別:検索" "ja" "all-access" "2016.01" 3980211
"71" "Заглавная_страница" "ru" "all-access" "2016.02" 38599053
"72" "Заглавная_страница" "ru" "desktop" "2016.02" 18024998
"73" "Заглавная_страница" "ru" "mobile-web" "2016.02" 19642974
"74" "メインページ" "ja" "all-access" "2016.02" 10693868
"75" "メインページ" "ja" "desktop" "2016.02" 6663639
"76" "Служебная:Поиск" "ru" "all-access" "2016.02" 6496788
"77" "Служебная:Поиск" "ru" "desktop" "2016.02" 6226236
"78" "Donald_Trump" "en" "all-access" "2016.02" 8097919
"79" "XHamster" "en" "all-access" "2016.02" 6242036
"80" "特別:検索" "ja" "all-access" "2016.02" 3644370
"81" "Заглавная_страница" "ru" "all-access" "2016.03" 43738796
"82" "Заглавная_страница" "ru" "desktop" "2016.03" 22688407
"83" "Заглавная_страница" "ru" "mobile-web" "2016.03" 20058382
"84" "メインページ" "ja" "all-access" "2016.03" 11267460
"85" "メインページ" "ja" "desktop" "2016.03" 6988458
"86" "Служебная:Поиск" "ru" "all-access" "2016.03" 6012314
"87" "Служебная:Поиск" "ru" "desktop" "2016.03" 5746732
"88" "Donald_Trump" "en" "all-access" "2016.03" 20952347
"89" "XHamster" "en" "all-access" "2016.03" 6424903
"90" "特別:検索" "ja" "all-access" "2016.03" 3534630
"91" "Заглавная_страница" "ru" "all-access" "2016.04" 38588419
"92" "Заглавная_страница" "ru" "desktop" "2016.04" 19831516
"93" "Заглавная_страница" "ru" "mobile-web" "2016.04" 18111382
"94" "メインページ" "ja" "all-access" "2016.04" 10875382
"95" "メインページ" "ja" "desktop" "2016.04" 6728369
"96" "Служебная:Поиск" "ru" "all-access" "2016.04" 5671845
"97" "Служебная:Поиск" "ru" "desktop" "2016.04" 5423449
"98" "Donald_Trump" "en" "all-access" "2016.04" 3606424
"99" "XHamster" "en" "all-access" "2016.04" 6198600
"100" "特別:検索" "ja" "all-access" "2016.04" 3403535
"101" "Заглавная_страница" "ru" "all-access" "2016.05" 34767998
"102" "Заглавная_страница" "ru" "desktop" "2016.05" 16423112
"103" "Заглавная_страница" "ru" "mobile-web" "2016.05" 17471098
"104" "メインページ" "ja" "all-access" "2016.05" 11773543
"105" "メインページ" "ja" "desktop" "2016.05" 7146377
"106" "Служебная:Поиск" "ru" "all-access" "2016.05" 5356098
"107" "Служебная:Поиск" "ru" "desktop" "2016.05" 5115833
"108" "Donald_Trump" "en" "all-access" "2016.05" 2920297
"109" "XHamster" "en" "all-access" "2016.05" 6657518
"110" "特別:検索" "ja" "all-access" "2016.05" 4857738
"111" "Заглавная_страница" "ru" "all-access" "2016.06" 31209932
"112" "Заглавная_страница" "ru" "desktop" "2016.06" 14621145
"113" "Заглавная_страница" "ru" "mobile-web" "2016.06" 15777427
"114" "メインページ" "ja" "all-access" "2016.06" 11582194
"115" "メインページ" "ja" "desktop" "2016.06" 7117395
"116" "Служебная:Поиск" "ru" "all-access" "2016.06" 4470001
"117" "Служебная:Поиск" "ru" "desktop" "2016.06" 4254207
"118" "Donald_Trump" "en" "all-access" "2016.06" 2293053
"119" "XHamster" "en" "all-access" "2016.06" 6111340
"120" "特別:検索" "ja" "all-access" "2016.06" 3708396
"121" "Заглавная_страница" "ru" "all-access" "2016.07" 176462208
"122" "Заглавная_страница" "ru" "desktop" "2016.07" 160487604
"123" "Заглавная_страница" "ru" "mobile-web" "2016.07" 15214836
"124" "メインページ" "ja" "all-access" "2016.07" 10795205
"125" "メインページ" "ja" "desktop" "2016.07" 6661971
"126" "Служебная:Поиск" "ru" "all-access" "2016.07" 4015488
"127" "Служебная:Поиск" "ru" "desktop" "2016.07" 3762357
"128" "Donald_Trump" "en" "all-access" "2016.07" 5008149
"129" "XHamster" "en" "all-access" "2016.07" 6152574
"130" "特別:検索" "ja" "all-access" "2016.07" 3968654
"131" "Заглавная_страница" "ru" "all-access" "2016.08" 239944781
"132" "Заглавная_страница" "ru" "desktop" "2016.08" 224108708
"133" "Заглавная_страница" "ru" "mobile-web" "2016.08" 15416634
"134" "メインページ" "ja" "all-access" "2016.08" 10565029
"135" "メインページ" "ja" "desktop" "2016.08" 6606075
"136" "Служебная:Поиск" "ru" "all-access" "2016.08" 4201370
"137" "Служебная:Поиск" "ru" "desktop" "2016.08" 3952220
"138" "Donald_Trump" "en" "all-access" "2016.08" 2566470
"139" "XHamster" "en" "all-access" "2016.08" 6331658
"140" "特別:検索" "ja" "all-access" "2016.08" 4027651
"141" "Заглавная_страница" "ru" "all-access" "2016.09" 31961310
"142" "Заглавная_страница" "ru" "desktop" "2016.09" 16308086
"143" "Заглавная_страница" "ru" "mobile-web" "2016.09" 15261420
"144" "メインページ" "ja" "all-access" "2016.09" 11295382
"145" "メインページ" "ja" "desktop" "2016.09" 7098569
"146" "Служебная:Поиск" "ru" "all-access" "2016.09" 5165564
"147" "Служебная:Поиск" "ru" "desktop" "2016.09" 4875939
"148" "Donald_Trump" "en" "all-access" "2016.09" 3069657
"149" "XHamster" "en" "all-access" "2016.09" 6734605
"150" "特別:検索" "ja" "all-access" "2016.09" 4155587
"151" "Заглавная_страница" "ru" "all-access" "2016.10" 50248245
"152" "Заглавная_страница" "ru" "desktop" "2016.10" 34618775
"153" "Заглавная_страница" "ru" "mobile-web" "2016.10" 15244703
"154" "メインページ" "ja" "all-access" "2016.10" 13042015
"155" "メインページ" "ja" "desktop" "2016.10" 8969599
"156" "Служебная:Поиск" "ru" "all-access" "2016.10" 7752392
"157" "Служебная:Поиск" "ru" "desktop" "2016.10" 7406888
"158" "Donald_Trump" "en" "all-access" "2016.10" 6505699
"159" "XHamster" "en" "all-access" "2016.10" 5661599
"160" "特別:検索" "ja" "all-access" "2016.10" 3878976
"161" "Заглавная_страница" "ru" "all-access" "2016.11" 42504016
"162" "Заглавная_страница" "ru" "desktop" "2016.11" 27539931
"163" "Заглавная_страница" "ru" "mobile-web" "2016.11" 14602264
"164" "メインページ" "ja" "all-access" "2016.11" 12478583
"165" "メインページ" "ja" "desktop" "2016.11" 8847777
"166" "Служебная:Поиск" "ru" "all-access" "2016.11" 5021213
"167" "Служебная:Поиск" "ru" "desktop" "2016.11" 4686516
"168" "Donald_Trump" "en" "all-access" "2016.11" 18274101
"169" "XHamster" "en" "all-access" "2016.11" 4942493
"170" "特別:検索" "ja" "all-access" "2016.11" 3778544
"171" "Заглавная_страница" "ru" "all-access" "2016.12" 42553550
"172" "Заглавная_страница" "ru" "desktop" "2016.12" 27297487
"173" "Заглавная_страница" "ru" "mobile-web" "2016.12" 14478894
"174" "メインページ" "ja" "all-access" "2016.12" 14805757
"175" "メインページ" "ja" "desktop" "2016.12" 9129007
"176" "Служебная:Поиск" "ru" "all-access" "2016.12" 4806444
"177" "Служебная:Поиск" "ru" "desktop" "2016.12" 4459412
"178" "Donald_Trump" "en" "all-access" "2016.12" 2548350
"179" "XHamster" "en" "all-access" "2016.12" 4731075
"180" "特別:検索" "ja" "all-access" "2016.12" 4477504

Taking the above example, notepad generated the following content:

"Article" "Language" "Access" "variable" "value"
"1" "Заглавная_страница" "ru" "all-access" "2015.07" 39195116
"2" "Заглавная_страница" "ru" "desktop" "2015.07" 18317353
"3" "Заглавная_страница" "ru" "mobile-web" "2015.07" 19462802
"4" "メインページ" "ja" "all-access" "2015.07" 12007556
"5" "メインページ" "ja" "desktop" "2015.07" 7835865
"6" "Служебная:Поиск" "ru" "all-access" "2015.07" 5075981
"7" "Служебная:Поиск" "ru" "desktop" "2015.07" 4798252
"8" "Donald_Trump" "en" "all-access" "2015.07" 2125559
"9" "XHamster" "en" "all-access" "2015.07" 92189
"10" "特別:検索" "ja" "all-access" "2015.07" 3562492
"11" "Заглавная_страница" "ru" "all-access" "2015.08" 43928436
"12" "Заглавная_страница" "ru" "desktop" "2015.08" 22420437
"13" "Заглавная_страница" "ru" "mobile-web" "2015.08" 20292928
"14" "メインページ" "ja" "all-access" "2015.08" 11848960
"15" "メインページ" "ja" "desktop" "2015.08" 7623353
"16" "Служебная:Поиск" "ru" "all-access" "2015.08" 5687670
"17" "Служебная:Поиск" "ru" "desktop" "2015.08" 5428867
"18" "Donald_Trump" "en" "all-access" "2015.08" 2896610
"19" "XHamster" "en" "all-access" "2015.08" 77452
"20" "特別:検索" "ja" "all-access" "2015.08" 4235434
"21" "Заглавная_страница" "ru" "all-access" "2015.09" 43842621
"22" "Заглавная_страница" "ru" "desktop" "2015.09" 21951052
"23" "Заглавная_страница" "ru" "mobile-web" "2015.09" 20914206
"24" "メインページ" "ja" "all-access" "2015.09" 11697789
"25" "メインページ" "ja" "desktop" "2015.09" 7637508
"26" "Служебная:Поиск" "ru" "all-access" "2015.09" 6458467
"27" "Служебная:Поиск" "ru" "desktop" "2015.09" 6162428
"28" "Donald_Trump" "en" "all-access" "2015.09" 2031231
"29" "XHamster" "en" "all-access" "2015.09" 82311
"30" "特別:検索" "ja" "all-access" "2015.09" 3771139
"31" "Заглавная_страница" "ru" "all-access" "2015.10" 47095921
"32" "Заглавная_страница" "ru" "desktop" "2015.10" 24657339
"33" "Заглавная_страница" "ru" "mobile-web" "2015.10" 21415510
"34" "メインページ" "ja" "all-access" "2015.10" 12051516
"35" "メインページ" "ja" "desktop" "2015.10" 7803228
"36" "Служебная:Поиск" "ru" "all-access" "2015.10" 6965280
"37" "Служебная:Поиск" "ru" "desktop" "2015.10" 6658177
"38" "Donald_Trump" "en" "all-access" "2015.10" 1204432
"39" "XHamster" "en" "all-access" "2015.10" 93494
"40" "特別:検索" "ja" "all-access" "2015.10" 3916872
"41" "Заглавная_страница" "ru" "all-access" "2015.11" 49634704
"42" "Заглавная_страница" "ru" "desktop" "2015.11" 27302012
"43" "Заглавная_страница" "ru" "mobile-web" "2015.11" 21280496
"44" "メインページ" "ja" "all-access" "2015.11" 11132801
"45" "メインページ" "ja" "desktop" "2015.11" 7047351
"46" "Служебная:Поиск" "ru" "all-access" "2015.11" 7013511
"47" "Служебная:Поиск" "ru" "desktop" "2015.11" 6711674
"48" "Donald_Trump" "en" "all-access" "2015.11" 1198269
"49" "XHamster" "en" "all-access" "2015.11" 84781
"50" "特別:検索" "ja" "all-access" "2015.11" 3855024
"51" "Заглавная_страница" "ru" "all-access" "2015.12" 46138639
"52" "Заглавная_страница" "ru" "desktop" "2015.12" 23582956
"53" "Заглавная_страница" "ru" "mobile-web" "2015.12" 21500700
"54" "メインページ" "ja" "all-access" "2015.12" 11332438
"55" "メインページ" "ja" "desktop" "2015.12" 7147768
"56" "Служебная:Поиск" "ru" "all-access" "2015.12" 6752230
"57" "Служебная:Поиск" "ru" "desktop" "2015.12" 6457536
"58" "Donald_Trump" "en" "all-access" "2015.12" 2611070
"59" "XHamster" "en" "all-access" "2015.12" 78977
"60" "特別:検索" "ja" "all-access" "2015.12" 3560172
"61" "Заглавная_страница" "ru" "all-access" "2016.01" 45605707
"62" "Заглавная_страница" "ru" "desktop" "2016.01" 22699098
"63" "Заглавная_страница" "ru" "mobile-web" "2016.01" 21783777
"64" "メインページ" "ja" "all-access" "2016.01" 11508317
"65" "メインページ" "ja" "desktop" "2016.01" 7095106
"66" "Служебная:Поиск" "ru" "all-access" "2016.01" 6841623
"67" "Служебная:Поиск" "ru" "desktop" "2016.01" 6537448
"68" "Donald_Trump" "en" "all-access" "2016.01" 2394948
"69" "XHamster" "en" "all-access" "2016.01" 5176941
"70" "特別:検索" "ja" "all-access" "2016.01" 3980211
"71" "Заглавная_страница" "ru" "all-access" "2016.02" 38599053
"72" "Заглавная_страница" "ru" "desktop" "2016.02" 18024998
"73" "Заглавная_страница" "ru" "mobile-web" "2016.02" 19642974
"74" "メインページ" "ja" "all-access" "2016.02" 10693868
"75" "メインページ" "ja" "desktop" "2016.02" 6663639
"76" "Служебная:Поиск" "ru" "all-access" "2016.02" 6496788
"77" "Служебная:Поиск" "ru" "desktop" "2016.02" 6226236
"78" "Donald_Trump" "en" "all-access" "2016.02" 8097919
"79" "XHamster" "en" "all-access" "2016.02" 6242036
"80" "特別:検索" "ja" "all-access" "2016.02" 3644370
"81" "Заглавная_страница" "ru" "all-access" "2016.03" 43738796
"82" "Заглавная_страница" "ru" "desktop" "2016.03" 22688407
"83" "Заглавная_страница" "ru" "mobile-web" "2016.03" 20058382
"84" "メインページ" "ja" "all-access" "2016.03" 11267460
"85" "メインページ" "ja" "desktop" "2016.03" 6988458
"86" "Служебная:Поиск" "ru" "all-access" "2016.03" 6012314
"87" "Служебная:Поиск" "ru" "desktop" "2016.03" 5746732
"88" "Donald_Trump" "en" "all-access" "2016.03" 20952347
"89" "XHamster" "en" "all-access" "2016.03" 6424903
"90" "特別:検索" "ja" "all-access" "2016.03" 3534630
"91" "Заглавная_страница" "ru" "all-access" "2016.04" 38588419
"92" "Заглавная_страница" "ru" "desktop" "2016.04" 19831516
"93" "Заглавная_страница" "ru" "mobile-web" "2016.04" 18111382
"94" "メインページ" "ja" "all-access" "2016.04" 10875382
"95" "メインページ" "ja" "desktop" "2016.04" 6728369
"96" "Служебная:Поиск" "ru" "all-access" "2016.04" 5671845
"97" "Служебная:Поиск" "ru" "desktop" "2016.04" 5423449
"98" "Donald_Trump" "en" "all-access" "2016.04" 3606424
"99" "XHamster" "en" "all-access" "2016.04" 6198600
"100" "特別:検索" "ja" "all-access" "2016.04" 3403535
"101" "Заглавная_страница" "ru" "all-access" "2016.05" 34767998
"102" "Заглавная_страница" "ru" "desktop" "2016.05" 16423112
"103" "Заглавная_страница" "ru" "mobile-web" "2016.05" 17471098
"104" "メインページ" "ja" "all-access" "2016.05" 11773543
"105" "メインページ" "ja" "desktop" "2016.05" 7146377
"106" "Служебная:Поиск" "ru" "all-access" "2016.05" 5356098
"107" "Служебная:Поиск" "ru" "desktop" "2016.05" 5115833
"108" "Donald_Trump" "en" "all-access" "2016.05" 2920297
"109" "XHamster" "en" "all-access" "2016.05" 6657518
"110" "特別:検索" "ja" "all-access" "2016.05" 4857738
"111" "Заглавная_страница" "ru" "all-access" "2016.06" 31209932
"112" "Заглавная_страница" "ru" "desktop" "2016.06" 14621145
"113" "Заглавная_страница" "ru" "mobile-web" "2016.06" 15777427
"114" "メインページ" "ja" "all-access" "2016.06" 11582194
"115" "メインページ" "ja" "desktop" "2016.06" 7117395
"116" "Служебная:Поиск" "ru" "all-access" "2016.06" 4470001
"117" "Служебная:Поиск" "ru" "desktop" "2016.06" 4254207
"118" "Donald_Trump" "en" "all-access" "2016.06" 2293053
"119" "XHamster" "en" "all-access" "2016.06" 6111340
"120" "特別:検索" "ja" "all-access" "2016.06" 3708396
"121" "Заглавная_страница" "ru" "all-access" "2016.07" 176462208
"122" "Заглавная_страница" "ru" "desktop" "2016.07" 160487604
"123" "Заглавная_страница" "ru" "mobile-web" "2016.07" 15214836
"124" "メインページ" "ja" "all-access" "2016.07" 10795205
"125" "メインページ" "ja" "desktop" "2016.07" 6661971
"126" "Служебная:Поиск" "ru" "all-access" "2016.07" 4015488
"127" "Служебная:Поиск" "ru" "desktop" "2016.07" 3762357
"128" "Donald_Trump" "en" "all-access" "2016.07" 5008149
"129" "XHamster" "en" "all-access" "2016.07" 6152574
"130" "特別:検索" "ja" "all-access" "2016.07" 3968654
"131" "Заглавная_страница" "ru" "all-access" "2016.08" 239944781
"132" "Заглавная_страница" "ru" "desktop" "2016.08" 224108708
"133" "Заглавная_страница" "ru" "mobile-web" "2016.08" 15416634
"134" "メインページ" "ja" "all-access" "2016.08" 10565029
"135" "メインページ" "ja" "desktop" "2016.08" 6606075
"136" "Служебная:Поиск" "ru" "all-access" "2016.08" 4201370
"137" "Служебная:Поиск" "ru" "desktop" "2016.08" 3952220
"138" "Donald_Trump" "en" "all-access" "2016.08" 2566470
"139" "XHamster" "en" "all-access" "2016.08" 6331658
"140" "特別:検索" "ja" "all-access" "2016.08" 4027651
"141" "Заглавная_страница" "ru" "all-access" "2016.09" 31961310
"142" "Заглавная_страница" "ru" "desktop" "2016.09" 16308086
"143" "Заглавная_страница" "ru" "mobile-web" "2016.09" 15261420
"144" "メインページ" "ja" "all-access" "2016.09" 11295382
"145" "メインページ" "ja" "desktop" "2016.09" 7098569
"146" "Служебная:Поиск" "ru" "all-access" "2016.09" 5165564
"147" "Служебная:Поиск" "ru" "desktop" "2016.09" 4875939
"148" "Donald_Trump" "en" "all-access" "2016.09" 3069657
"149" "XHamster" "en" "all-access" "2016.09" 6734605
"150" "特別:検索" "ja" "all-access" "2016.09" 4155587
"151" "Заглавная_страница" "ru" "all-access" "2016.10" 50248245
"152" "Заглавная_страница" "ru" "desktop" "2016.10" 34618775
"153" "Заглавная_страница" "ru" "mobile-web" "2016.10" 15244703
"154" "メインページ" "ja" "all-access" "2016.10" 13042015
"155" "メインページ" "ja" "desktop" "2016.10" 8969599
"156" "Служебная:Поиск" "ru" "all-access" "2016.10" 7752392
"157" "Служебная:Поиск" "ru" "desktop" "2016.10" 7406888
"158" "Donald_Trump" "en" "all-access" "2016.10" 6505699
"159" "XHamster" "en" "all-access" "2016.10" 5661599
"160" "特別:検索" "ja" "all-access" "2016.10" 3878976
"161" "Заглавная_страница" "ru" "all-access" "2016.11" 42504016
"162" "Заглавная_страница" "ru" "desktop" "2016.11" 27539931
"163" "Заглавная_страница" "ru" "mobile-web" "2016.11" 14602264
"164" "メインページ" "ja" "all-access" "2016.11" 12478583
"165" "メインページ" "ja" "desktop" "2016.11" 8847777
"166" "Служебная:Поиск" "ru" "all-access" "2016.11" 5021213
"167" "Служебная:Поиск" "ru" "desktop" "2016.11" 4686516
"168" "Donald_Trump" "en" "all-access" "2016.11" 18274101
"169" "XHamster" "en" "all-access" "2016.11" 4942493
"170" "特別:検索" "ja" "all-access" "2016.11" 3778544
"171" "Заглавная_страница" "ru" "all-access" "2016.12" 42553550
"172" "Заглавная_страница" "ru" "desktop" "2016.12" 27297487
"173" "Заглавная_страница" "ru" "mobile-web" "2016.12" 14478894
"174" "メインページ" "ja" "all-access" "2016.12" 14805757
"175" "メインページ" "ja" "desktop" "2016.12" 9129007
"176" "Служебная:Поиск" "ru" "all-access" "2016.12" 4806444
"177" "Служебная:Поиск" "ru" "desktop" "2016.12" 4459412
"178" "Donald_Trump" "en" "all-access" "2016.12" 2548350
"179" "XHamster" "en" "all-access" "2016.12" 4731075
"180" "特別:検索" "ja" "all-access" "2016.12" 4477504
jeff1evesque commented 6 years ago

The above, translates to the following:

  1. FrontPage
  2. FrontPage
  3. FrontPage
  4. Main page
  5. Main page
  6. Service: Search
  7. Service: Search
  8. Donald Trump
  9. XHampster
  10. Special: Search

From above, it's clear that we can remove all instances except Donald Trump, and XHampster.

jeff1evesque commented 6 years ago

It seems in RStudio we can default the encoding to be UTF8, which removes the above error, by going to File > Save by Encoding ...:

utf8

jeff1evesque commented 6 years ago

We generated the same csv:

write.table(
  average_top20.m,
  file = 'top20.csv',
  append = FALSE,
  quote = TRUE,
  sep = ' ',
  eol = '\n',
  na = 'NA',
  dec = '.',
  row.names = TRUE,
  col.names = TRUE,
  qmethod = c('escape', 'double'),
  fileEncoding = ''
)

Which produced the following content in notepad:

"Article" "Language" "Access" "variable" "value"
"1" "特別:検索" "ja" "all-access" "2015.07" 3562492
"2" "特別:検索" "ja" "desktop" "2015.07" 3491732
"3" "XHamster" "en" "mobile-web" "2015.07" 56508
"4" "メインページ" "ja" "mobile-web" "2015.07" 3384453
"5" "Web_scraping" "en" "all-access" "2015.07" 25171
"6" "Web_scraping" "en" "desktop" "2015.07" 21942
"7" "Donald_Trump" "en" "mobile-web" "2015.07" 1096340
"8" "Proyecto_40" "en" "all-access" "2015.07" 71222
"9" "Proyecto_40" "en" "desktop" "2015.07" 68681
"10" "AMGTV" "en" "all-access" "2015.07" 1544
"11" "Donald_Trump" "en" "desktop" "2015.07" 993185
"12" "特別:検索" "ja" "all-access" "2015.08" 4235434
"13" "特別:検索" "ja" "desktop" "2015.08" 4150079
"14" "XHamster" "en" "mobile-web" "2015.08" 42396
"15" "メインページ" "ja" "mobile-web" "2015.08" 3561256
"16" "Web_scraping" "en" "all-access" "2015.08" 25217
"17" "Web_scraping" "en" "desktop" "2015.08" 22040
"18" "Donald_Trump" "en" "mobile-web" "2015.08" 1630885
"19" "Proyecto_40" "en" "all-access" "2015.08" 173963
"20" "Proyecto_40" "en" "desktop" "2015.08" 168056
"21" "AMGTV" "en" "all-access" "2015.08" 1356
"22" "Donald_Trump" "en" "desktop" "2015.08" 1215657
"23" "特別:検索" "ja" "all-access" "2015.09" 3771139
"24" "特別:検索" "ja" "desktop" "2015.09" 3707563
"25" "XHamster" "en" "mobile-web" "2015.09" 47789
"26" "メインページ" "ja" "mobile-web" "2015.09" 3523796
"27" "Web_scraping" "en" "all-access" "2015.09" 23312
"28" "Web_scraping" "en" "desktop" "2015.09" 20130
"29" "Donald_Trump" "en" "mobile-web" "2015.09" 1124044
"30" "Proyecto_40" "en" "all-access" "2015.09" 152993
"31" "Proyecto_40" "en" "desktop" "2015.09" 147174
"32" "AMGTV" "en" "all-access" "2015.09" 1435
"33" "Donald_Trump" "en" "desktop" "2015.09" 874207
"34" "特別:検索" "ja" "all-access" "2015.10" 3916872
"35" "特別:検索" "ja" "desktop" "2015.10" 3855487
"36" "XHamster" "en" "mobile-web" "2015.10" 57229
"37" "メインページ" "ja" "mobile-web" "2015.10" 3675691
"38" "Web_scraping" "en" "all-access" "2015.10" 7401258
"39" "Web_scraping" "en" "desktop" "2015.10" 7387561
"40" "Donald_Trump" "en" "mobile-web" "2015.10" 612800
"41" "Proyecto_40" "en" "all-access" "2015.10" 186883
"42" "Proyecto_40" "en" "desktop" "2015.10" 179609
"43" "AMGTV" "en" "all-access" "2015.10" 1434
"44" "Donald_Trump" "en" "desktop" "2015.10" 572427
"45" "特別:検索" "ja" "all-access" "2015.11" 3855024
"46" "特別:検索" "ja" "desktop" "2015.11" 3805118
"47" "XHamster" "en" "mobile-web" "2015.11" 49224
"48" "メインページ" "ja" "mobile-web" "2015.11" 3554744
"49" "Web_scraping" "en" "all-access" "2015.11" 5736632
"50" "Web_scraping" "en" "desktop" "2015.11" 5726401
"51" "Donald_Trump" "en" "mobile-web" "2015.11" 649843
"52" "Proyecto_40" "en" "all-access" "2015.11" 171008
"53" "Proyecto_40" "en" "desktop" "2015.11" 162281
"54" "AMGTV" "en" "all-access" "2015.11" 1418
"55" "Donald_Trump" "en" "desktop" "2015.11" 528305
"56" "特別:検索" "ja" "all-access" "2015.12" 3560172
"57" "特別:検索" "ja" "desktop" "2015.12" 3511656
"58" "XHamster" "en" "mobile-web" "2015.12" 48024
"59" "メインページ" "ja" "mobile-web" "2015.12" 3631705
"60" "Web_scraping" "en" "all-access" "2015.12" 4982812
"61" "Web_scraping" "en" "desktop" "2015.12" 4973569
"62" "Donald_Trump" "en" "mobile-web" "2015.12" 1415539
"63" "Proyecto_40" "en" "all-access" "2015.12" 257001
"64" "Proyecto_40" "en" "desktop" "2015.12" 248441
"65" "AMGTV" "en" "all-access" "2015.12" 1487
"66" "Donald_Trump" "en" "desktop" "2015.12" 1152438
"67" "特別:検索" "ja" "all-access" "2016.01" 3980211
"68" "特別:検索" "ja" "desktop" "2016.01" 3927304
"69" "XHamster" "en" "mobile-web" "2016.01" 4888785
"70" "メインページ" "ja" "mobile-web" "2016.01" 3864710
"71" "Web_scraping" "en" "all-access" "2016.01" 6533260
"72" "Web_scraping" "en" "desktop" "2016.01" 6521662
"73" "Donald_Trump" "en" "mobile-web" "2016.01" 1292386
"74" "Proyecto_40" "en" "all-access" "2016.01" 198446
"75" "Proyecto_40" "en" "desktop" "2016.01" 189356
"76" "AMGTV" "en" "all-access" "2016.01" 1656
"77" "Donald_Trump" "en" "desktop" "2016.01" 1064194
"78" "特別:検索" "ja" "all-access" "2016.02" 3644370
"79" "特別:検索" "ja" "desktop" "2016.02" 3596002
"80" "XHamster" "en" "mobile-web" "2016.02" 5905044
"81" "メインページ" "ja" "mobile-web" "2016.02" 3548294
"82" "Web_scraping" "en" "all-access" "2016.02" 33209178
"83" "Web_scraping" "en" "desktop" "2016.02" 33199020
"84" "Donald_Trump" "en" "mobile-web" "2016.02" 4405650
"85" "Proyecto_40" "en" "all-access" "2016.02" 630231
"86" "Proyecto_40" "en" "desktop" "2016.02" 620340
"87" "AMGTV" "en" "all-access" "2016.02" 1492
"88" "Donald_Trump" "en" "desktop" "2016.02" 3618729
"89" "特別:検索" "ja" "all-access" "2016.03" 3534630
"90" "特別:検索" "ja" "desktop" "2016.03" 3486857
"91" "XHamster" "en" "mobile-web" "2016.03" 6087171
"92" "メインページ" "ja" "mobile-web" "2016.03" 3761438
"93" "Web_scraping" "en" "all-access" "2016.03" 47364
"94" "Web_scraping" "en" "desktop" "2016.03" 43008
"95" "Donald_Trump" "en" "mobile-web" "2016.03" 14543110
"96" "Proyecto_40" "en" "all-access" "2016.03" 105701
"97" "Proyecto_40" "en" "desktop" "2016.03" 95565
"98" "AMGTV" "en" "all-access" "2016.03" 1499
"99" "Donald_Trump" "en" "desktop" "2016.03" 6279382
"100" "特別:検索" "ja" "all-access" "2016.04" 3403535
"101" "特別:検索" "ja" "desktop" "2016.04" 3351517
"102" "XHamster" "en" "mobile-web" "2016.04" 5865282
"103" "メインページ" "ja" "mobile-web" "2016.04" 3767504
"104" "Web_scraping" "en" "all-access" "2016.04" 49364
"105" "Web_scraping" "en" "desktop" "2016.04" 45291
"106" "Donald_Trump" "en" "mobile-web" "2016.04" 1751128
"107" "Proyecto_40" "en" "all-access" "2016.04" 71656
"108" "Proyecto_40" "en" "desktop" "2016.04" 63880
"109" "AMGTV" "en" "all-access" "2016.04" 1643
"110" "Donald_Trump" "en" "desktop" "2016.04" 1819843
"111" "特別:検索" "ja" "all-access" "2016.05" 4857738
"112" "特別:検索" "ja" "desktop" "2016.05" 4801751
"113" "XHamster" "en" "mobile-web" "2016.05" 6311479
"114" "メインページ" "ja" "mobile-web" "2016.05" 4133655
"115" "Web_scraping" "en" "all-access" "2016.05" 28661
"116" "Web_scraping" "en" "desktop" "2016.05" 24438
"117" "Donald_Trump" "en" "mobile-web" "2016.05" 1319392
"118" "Proyecto_40" "en" "all-access" "2016.05" 61164
"119" "Proyecto_40" "en" "desktop" "2016.05" 53590
"120" "AMGTV" "en" "all-access" "2016.05" 1546
"121" "Donald_Trump" "en" "desktop" "2016.05" 1555438
"122" "特別:検索" "ja" "all-access" "2016.06" 3708396
"123" "特別:検索" "ja" "desktop" "2016.06" 3652015
"124" "XHamster" "en" "mobile-web" "2016.06" 5786117
"125" "メインページ" "ja" "mobile-web" "2016.06" 3990333
"126" "Web_scraping" "en" "all-access" "2016.06" 27076
"127" "Web_scraping" "en" "desktop" "2016.06" 23178
"128" "Donald_Trump" "en" "mobile-web" "2016.06" 1069239
"129" "Proyecto_40" "en" "all-access" "2016.06" 50662
"130" "Proyecto_40" "en" "desktop" "2016.06" 44775
"131" "AMGTV" "en" "all-access" "2016.06" 2191
"132" "Donald_Trump" "en" "desktop" "2016.06" 1188435
"133" "特別:検索" "ja" "all-access" "2016.07" 3968654
"134" "特別:検索" "ja" "desktop" "2016.07" 3902901
"135" "XHamster" "en" "mobile-web" "2016.07" 5842178
"136" "メインページ" "ja" "mobile-web" "2016.07" 3667074
"137" "Web_scraping" "en" "all-access" "2016.07" 27004
"138" "Web_scraping" "en" "desktop" "2016.07" 23028
"139" "Donald_Trump" "en" "mobile-web" "2016.07" 2853712
"140" "Proyecto_40" "en" "all-access" "2016.07" 3328635
"141" "Proyecto_40" "en" "desktop" "2016.07" 3235175
"142" "AMGTV" "en" "all-access" "2016.07" 1652
"143" "Donald_Trump" "en" "desktop" "2016.07" 2053989
"144" "特別:検索" "ja" "all-access" "2016.08" 4027651
"145" "特別:検索" "ja" "desktop" "2016.08" 3953837
"146" "XHamster" "en" "mobile-web" "2016.08" 6031132
"147" "メインページ" "ja" "mobile-web" "2016.08" 3698145
"148" "Web_scraping" "en" "all-access" "2016.08" 27016
"149" "Web_scraping" "en" "desktop" "2016.08" 22757
"150" "Donald_Trump" "en" "mobile-web" "2016.08" 1359517
"151" "Proyecto_40" "en" "all-access" "2016.08" 6576107
"152" "Proyecto_40" "en" "desktop" "2016.08" 6393428
"153" "AMGTV" "en" "all-access" "2016.08" 4313660
"154" "Donald_Trump" "en" "desktop" "2016.08" 1169842
"155" "特別:検索" "ja" "all-access" "2016.09" 4155587
"156" "特別:検索" "ja" "desktop" "2016.09" 4087283
"157" "XHamster" "en" "mobile-web" "2016.09" 6415768
"158" "メインページ" "ja" "mobile-web" "2016.09" 3973247
"159" "Web_scraping" "en" "all-access" "2016.09" 25359
"160" "Web_scraping" "en" "desktop" "2016.09" 21261
"161" "Donald_Trump" "en" "mobile-web" "2016.09" 1564435
"162" "Proyecto_40" "en" "all-access" "2016.09" 7312148
"163" "Proyecto_40" "en" "desktop" "2016.09" 7110593
"164" "AMGTV" "en" "all-access" "2016.09" 8388911
"165" "Donald_Trump" "en" "desktop" "2016.09" 1467963
"166" "特別:検索" "ja" "all-access" "2016.10" 3878976
"167" "特別:検索" "ja" "desktop" "2016.10" 3815166
"168" "XHamster" "en" "mobile-web" "2016.10" 5379257
"169" "メインページ" "ja" "mobile-web" "2016.10" 3863429
"170" "Web_scraping" "en" "all-access" "2016.10" 25743
"171" "Web_scraping" "en" "desktop" "2016.10" 21721
"172" "Donald_Trump" "en" "mobile-web" "2016.10" 3720394
"173" "Proyecto_40" "en" "all-access" "2016.10" 8413888
"174" "Proyecto_40" "en" "desktop" "2016.10" 8182927
"175" "AMGTV" "en" "all-access" "2016.10" 9619391
"176" "Donald_Trump" "en" "desktop" "2016.10" 2707605
"177" "特別:検索" "ja" "all-access" "2016.11" 3778544
"178" "特別:検索" "ja" "desktop" "2016.11" 3712621
"179" "XHamster" "en" "mobile-web" "2016.11" 4697245
"180" "メインページ" "ja" "mobile-web" "2016.11" 3433041
"181" "Web_scraping" "en" "all-access" "2016.11" 25356
"182" "Web_scraping" "en" "desktop" "2016.11" 21426
"183" "Donald_Trump" "en" "mobile-web" "2016.11" 10653359
"184" "Proyecto_40" "en" "all-access" "2016.11" 9332483
"185" "Proyecto_40" "en" "desktop" "2016.11" 9072781
"186" "AMGTV" "en" "all-access" "2016.11" 10544580
"187" "Donald_Trump" "en" "desktop" "2016.11" 7312633
"188" "特別:検索" "ja" "all-access" "2016.12" 4477504
"189" "特別:検索" "ja" "desktop" "2016.12" 4406317
"190" "XHamster" "en" "mobile-web" "2016.12" 4481687
"191" "メインページ" "ja" "mobile-web" "2016.12" 3426607
"192" "Web_scraping" "en" "all-access" "2016.12" 22177
"193" "Web_scraping" "en" "desktop" "2016.12" 18429
"194" "Donald_Trump" "en" "mobile-web" "2016.12" 1416496
"195" "Proyecto_40" "en" "all-access" "2016.12" 3662535
"196" "Proyecto_40" "en" "desktop" "2016.12" 3429790
"197" "AMGTV" "en" "all-access" "2016.12" 4771438
"198" "Donald_Trump" "en" "desktop" "2016.12" 1084074

This translates to the following:

"Article" "Language" "Access" "variable" "value"
"1" "Special: Search" "ja" "all-access" "2015.07" 3562492
"2" "Special: Search" "ja" "desktop" "2015.07" 3491732
"3" "XHamster" "en" "mobile-web" "2015.07" 56508
"4" "Main page" "ja" "mobile-web" "2015.07" 3384453
"5" "Web_scraping" "en" "all-access" "2015.07" 25171
"6" "Web_scraping" "en" "desktop" "2015.07" 21942
"7" "Donald_Trump" "en" "mobile-web" "2015.07" 1096340
"8" "Proyecto_40" "en" "all-access" "2015.07" 71222
"9" "Proyecto_40" "en" "desktop" "2015.07" 68681
"10" "AMGTV" "en" "all-access" "2015.07" 1544

So, we can remove all instances of Special: Search, and Main page from our dataframe.

jeff1evesque commented 6 years ago

We were able to generate a csv for the top 40-50 results:

write.table(
  average_top40.m,
  file = 'top40.csv',
  append = FALSE,
  quote = TRUE,
  sep = ' ',
  eol = '\n',
  na = 'NA',
  dec = '.',
  row.names = TRUE,
  col.names = TRUE,
  qmethod = c('escape', 'double'),
  fileEncoding = ''
)

Which produced the following content in notepad:

"Article" "Language" "Access" "variable" "value"
"1" "Hauptseite" "de" "desktop" "2015.07" 1734371
"2" "United_States_presidential_election,_2016" "en" "all-access" "2015.07" 518170
"3" "United_States" "en" "all-access" "2015.07" 1389441
"4" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.07" 1307869
"5" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.07" 1301449
"6" "Lali_Espósito" "es" "desktop" "2015.07" 33993
"7" "Prince_(musician)" "en" "all-access" "2015.07" 265779
"8" "Facebook" "en" "all-access" "2015.07" 1508481
"9" "Suicide_Squad_(film)" "en" "all-access" "2015.07" 2401901
"10" "Star_Wars" "en" "all-access" "2015.07" 532750
"11" "Captain_America:_Civil_War" "en" "all-access" "2015.07" 1011155
"12" "Hauptseite" "de" "desktop" "2015.08" 1577355
"13" "United_States_presidential_election,_2016" "en" "all-access" "2015.08" 673805
"14" "United_States" "en" "all-access" "2015.08" 1228022
"15" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.08" 1232406
"16" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.08" 1225882
"17" "Lali_Espósito" "es" "desktop" "2015.08" 37905
"18" "Prince_(musician)" "en" "all-access" "2015.08" 272590
"19" "Facebook" "en" "all-access" "2015.08" 1406504
"20" "Suicide_Squad_(film)" "en" "all-access" "2015.08" 661698
"21" "Star_Wars" "en" "all-access" "2015.08" 622581
"22" "Captain_America:_Civil_War" "en" "all-access" "2015.08" 860774
"23" "Hauptseite" "de" "desktop" "2015.09" 1623685
"24" "United_States_presidential_election,_2016" "en" "all-access" "2015.09" 603415
"25" "United_States" "en" "all-access" "2015.09" 1295477
"26" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.09" 1361523
"27" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.09" 1354479
"28" "Lali_Espósito" "es" "desktop" "2015.09" 30066
"29" "Prince_(musician)" "en" "all-access" "2015.09" 300155
"30" "Facebook" "en" "all-access" "2015.09" 1391545
"31" "Suicide_Squad_(film)" "en" "all-access" "2015.09" 408342
"32" "Star_Wars" "en" "all-access" "2015.09" 632048
"33" "Captain_America:_Civil_War" "en" "all-access" "2015.09" 582479
"34" "Hauptseite" "de" "desktop" "2015.10" 1657135
"35" "United_States_presidential_election,_2016" "en" "all-access" "2015.10" 632384
"36" "United_States" "en" "all-access" "2015.10" 1393463
"37" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.10" 1304983
"38" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.10" 1302871
"39" "Lali_Espósito" "es" "desktop" "2015.10" 29670
"40" "Prince_(musician)" "en" "all-access" "2015.10" 257049
"41" "Facebook" "en" "all-access" "2015.10" 1265080
"42" "Suicide_Squad_(film)" "en" "all-access" "2015.10" 455706
"43" "Star_Wars" "en" "all-access" "2015.10" 1345302
"44" "Captain_America:_Civil_War" "en" "all-access" "2015.10" 514638
"45" "Hauptseite" "de" "desktop" "2015.11" 1632287
"46" "United_States_presidential_election,_2016" "en" "all-access" "2015.11" 690562
"47" "United_States" "en" "all-access" "2015.11" 1440357
"48" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.11" 1979445
"49" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.11" 1978339
"50" "Lali_Espósito" "es" "desktop" "2015.11" 32824
"51" "Prince_(musician)" "en" "all-access" "2015.11" 280434
"52" "Facebook" "en" "all-access" "2015.11" 1251352
"53" "Suicide_Squad_(film)" "en" "all-access" "2015.11" 376339
"54" "Star_Wars" "en" "all-access" "2015.11" 1490466
"55" "Captain_America:_Civil_War" "en" "all-access" "2015.11" 980785
"56" "Hauptseite" "de" "desktop" "2015.12" 1420903
"57" "United_States_presidential_election,_2016" "en" "all-access" "2015.12" 681166
"58" "United_States" "en" "all-access" "2015.12" 1309206
"59" "Служебная:Ссылки_сюда" "ru" "all-access" "2015.12" 754011
"60" "Служебная:Ссылки_сюда" "ru" "desktop" "2015.12" 752896
"61" "Lali_Espósito" "es" "desktop" "2015.12" 28507
"62" "Prince_(musician)" "en" "all-access" "2015.12" 266180
"63" "Facebook" "en" "all-access" "2015.12" 2935017
"64" "Suicide_Squad_(film)" "en" "all-access" "2015.12" 483171
"65" "Star_Wars" "en" "all-access" "2015.12" 7880288
"66" "Captain_America:_Civil_War" "en" "all-access" "2015.12" 754508
"67" "Hauptseite" "de" "desktop" "2016.01" 1605727
"68" "United_States_presidential_election,_2016" "en" "all-access" "2016.01" 956572
"69" "United_States" "en" "all-access" "2016.01" 1487588
"70" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.01" 1694583
"71" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.01" 1693098
"72" "Lali_Espósito" "es" "desktop" "2016.01" 29877
"73" "Prince_(musician)" "en" "all-access" "2016.01" 278346
"74" "Facebook" "en" "all-access" "2016.01" 1341993
"75" "Suicide_Squad_(film)" "en" "all-access" "2016.01" 1248681
"76" "Star_Wars" "en" "all-access" "2016.01" 2685656
"77" "Captain_America:_Civil_War" "en" "all-access" "2016.01" 636454
"78" "Hauptseite" "de" "desktop" "2016.02" 1684600
"79" "United_States_presidential_election,_2016" "en" "all-access" "2016.02" 1468175
"80" "United_States" "en" "all-access" "2016.02" 1370186
"81" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.02" 1104918
"82" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.02" 1102973
"83" "Lali_Espósito" "es" "desktop" "2016.02" 22692
"84" "Prince_(musician)" "en" "all-access" "2016.02" 402043
"85" "Facebook" "en" "all-access" "2016.02" 1235196
"86" "Suicide_Squad_(film)" "en" "all-access" "2016.02" 537547
"87" "Star_Wars" "en" "all-access" "2016.02" 732380
"88" "Captain_America:_Civil_War" "en" "all-access" "2016.02" 763384
"89" "Hauptseite" "de" "desktop" "2016.03" 1495212
"90" "United_States_presidential_election,_2016" "en" "all-access" "2016.03" 1567744
"91" "United_States" "en" "all-access" "2016.03" 1525295
"92" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.03" 1417020
"93" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.03" 1415361
"94" "Lali_Espósito" "es" "desktop" "2016.03" 23952
"95" "Prince_(musician)" "en" "all-access" "2016.03" 312140
"96" "Facebook" "en" "all-access" "2016.03" 1253651
"97" "Suicide_Squad_(film)" "en" "all-access" "2016.03" 761134
"98" "Star_Wars" "en" "all-access" "2016.03" 619159
"99" "Captain_America:_Civil_War" "en" "all-access" "2016.03" 1561975
"100" "Hauptseite" "de" "desktop" "2016.04" 1592192
"101" "United_States_presidential_election,_2016" "en" "all-access" "2016.04" 714701
"102" "United_States" "en" "all-access" "2016.04" 1790943
"103" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.04" 1739891
"104" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.04" 1738514
"105" "Lali_Espósito" "es" "desktop" "2016.04" 19084
"106" "Prince_(musician)" "en" "all-access" "2016.04" 17593761
"107" "Facebook" "en" "all-access" "2016.04" 1118960
"108" "Suicide_Squad_(film)" "en" "all-access" "2016.04" 1162461
"109" "Star_Wars" "en" "all-access" "2016.04" 871077
"110" "Captain_America:_Civil_War" "en" "all-access" "2016.04" 2995688
"111" "Hauptseite" "de" "desktop" "2016.05" 1339050
"112" "United_States_presidential_election,_2016" "en" "all-access" "2016.05" 952964
"113" "United_States" "en" "all-access" "2016.05" 1622682
"114" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.05" 1096173
"115" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.05" 1094887
"116" "Lali_Espósito" "es" "desktop" "2016.05" 20236
"117" "Prince_(musician)" "en" "all-access" "2016.05" 1801197
"118" "Facebook" "en" "all-access" "2016.05" 1115910
"119" "Suicide_Squad_(film)" "en" "all-access" "2016.05" 612550
"120" "Star_Wars" "en" "all-access" "2016.05" 631501
"121" "Captain_America:_Civil_War" "en" "all-access" "2016.05" 8169422
"122" "Hauptseite" "de" "desktop" "2016.06" 1339732
"123" "United_States_presidential_election,_2016" "en" "all-access" "2016.06" 890686
"124" "United_States" "en" "all-access" "2016.06" 1428307
"125" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.06" 1118183
"126" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.06" 1116627
"127" "Lali_Espósito" "es" "desktop" "2016.06" 18831
"128" "Prince_(musician)" "en" "all-access" "2016.06" 661565
"129" "Facebook" "en" "all-access" "2016.06" 1060695
"130" "Suicide_Squad_(film)" "en" "all-access" "2016.06" 802305
"131" "Star_Wars" "en" "all-access" "2016.06" 412372
"132" "Captain_America:_Civil_War" "en" "all-access" "2016.06" 1245143
"133" "Hauptseite" "de" "desktop" "2016.07" 1251503
"134" "United_States_presidential_election,_2016" "en" "all-access" "2016.07" 1101900
"135" "United_States" "en" "all-access" "2016.07" 1322539
"136" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.07" 2008057
"137" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.07" 2007638
"138" "Lali_Espósito" "es" "desktop" "2016.07" 401977
"139" "Prince_(musician)" "en" "all-access" "2016.07" 303731
"140" "Facebook" "en" "all-access" "2016.07" 970732
"141" "Suicide_Squad_(film)" "en" "all-access" "2016.07" 2401933
"142" "Star_Wars" "en" "all-access" "2016.07" 462223
"143" "Captain_America:_Civil_War" "en" "all-access" "2016.07" 676101
"144" "Hauptseite" "de" "desktop" "2016.08" 1297924
"145" "United_States_presidential_election,_2016" "en" "all-access" "2016.08" 770667
"146" "United_States" "en" "all-access" "2016.08" 1282844
"147" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.08" 912865
"148" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.08" 912354
"149" "Lali_Espósito" "es" "desktop" "2016.08" 2511461
"150" "Prince_(musician)" "en" "all-access" "2016.08" 257315
"151" "Facebook" "en" "all-access" "2016.08" 1398709
"152" "Suicide_Squad_(film)" "en" "all-access" "2016.08" 7500312
"153" "Star_Wars" "en" "all-access" "2016.08" 521633
"154" "Captain_America:_Civil_War" "en" "all-access" "2016.08" 607393
"155" "Hauptseite" "de" "desktop" "2016.09" 1211923
"156" "United_States_presidential_election,_2016" "en" "all-access" "2016.09" 983638
"157" "United_States" "en" "all-access" "2016.09" 1283570
"158" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.09" 1500842
"159" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.09" 1499240
"160" "Lali_Espósito" "es" "desktop" "2016.09" 6033355
"161" "Prince_(musician)" "en" "all-access" "2016.09" 242354
"162" "Facebook" "en" "all-access" "2016.09" 1162557
"163" "Suicide_Squad_(film)" "en" "all-access" "2016.09" 1329633
"164" "Star_Wars" "en" "all-access" "2016.09" 630348
"165" "Captain_America:_Civil_War" "en" "all-access" "2016.09" 826461
"166" "Hauptseite" "de" "desktop" "2016.10" 1338389
"167" "United_States_presidential_election,_2016" "en" "all-access" "2016.10" 1843035
"168" "United_States" "en" "all-access" "2016.10" 1442285
"169" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.10" 1503042
"170" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.10" 1499910
"171" "Lali_Espósito" "es" "desktop" "2016.10" 7987393
"172" "Prince_(musician)" "en" "all-access" "2016.10" 274514
"173" "Facebook" "en" "all-access" "2016.10" 1341568
"174" "Suicide_Squad_(film)" "en" "all-access" "2016.10" 1092952
"175" "Star_Wars" "en" "all-access" "2016.10" 620393
"176" "Captain_America:_Civil_War" "en" "all-access" "2016.10" 427860
"177" "Hauptseite" "de" "desktop" "2016.11" 1338350
"178" "United_States_presidential_election,_2016" "en" "all-access" "2016.11" 9164298
"179" "United_States" "en" "all-access" "2016.11" 1844238
"180" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.11" 1543246
"181" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.11" 1539596
"182" "Lali_Espósito" "es" "desktop" "2016.11" 4155536
"183" "Prince_(musician)" "en" "all-access" "2016.11" 305243
"184" "Facebook" "en" "all-access" "2016.11" 1135987
"185" "Suicide_Squad_(film)" "en" "all-access" "2016.11" 946676
"186" "Star_Wars" "en" "all-access" "2016.11" 704311
"187" "Captain_America:_Civil_War" "en" "all-access" "2016.11" 391090
"188" "Hauptseite" "de" "desktop" "2016.12" 1271224
"189" "United_States_presidential_election,_2016" "en" "all-access" "2016.12" 1879095
"190" "United_States" "en" "all-access" "2016.12" 1384389
"191" "Служебная:Ссылки_сюда" "ru" "all-access" "2016.12" 1522947
"192" "Служебная:Ссылки_сюда" "ru" "desktop" "2016.12" 1522041
"193" "Lali_Espósito" "es" "desktop" "2016.12" 3312866
"194" "Prince_(musician)" "en" "all-access" "2016.12" 483724
"195" "Facebook" "en" "all-access" "2016.12" 1528216
"196" "Suicide_Squad_(film)" "en" "all-access" "2016.12" 1118541
"197" "Star_Wars" "en" "all-access" "2016.12" 2707050
"198" "Captain_America:_Civil_War" "en" "all-access" "2016.12" 515161

This further translates to the following:

"Article" "Language" "Access" "variable" "value"
"1" "Hauptseite" "de" "desktop" "2015.07" 1734371
"2" "United_States_presidential_election,_2016" "en" "all-access" "2015.07" 518170
"3" "United_States" "en" "all-access" "2015.07" 1389441
"4" "Official: Links_ here" "ru" "all-access" "2015.07" 1307869
"5" "Official: Links_ here" "ru" "desktop" "2015.07" 1301449
"6" "Lali_Espósito" "es" "desktop" "2015.07" 33993
"7" "Prince_(musician)" "en" "all-access" "2015.07" 265779
"8" "Facebook" "en" "all-access" "2015.07" 1508481
"9" "Suicide_Squad_(film)" "en" "all-access" "2015.07" 2401901
"10" "Star_Wars" "en" "all-access" "2015.07" 532750

So, we know we can remove instances of article 4 and 5, throughout this dataframe.

jeff1evesque commented 6 years ago

Unfortunately, windows is too good to work. While being able to save a file containing utf8 encoding, this doesn't guaranty that the R code can interpret this:

utf8

Instead, if our munge_ist687.R contains UTF8 characters, based on the above filtering, then our returned dataframe object is empty. Somewhere along the blackhole of windows, it doesn't get interpretted.

jeff1evesque commented 6 years ago

Actually, just by enabling UTF8 in R, will return an empty dataframe, even if we don't include our above filtered UTF8 article names, or article pattern names in our munge_ist687.R.

jeff1evesque commented 6 years ago

04e6419: it seems that the reshape2 package requires stringi:

...
> load_package(c('reshape2', 'zoo', 'Hmisc', 'ggplot2', 'grid', 'gridExtra'))
Loading required package: reshape2
Error: package or namespace load failed for ‘reshape2’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
 there is no package called ‘stringi’
Loading required package: zoo

Attaching package: ‘zoo’

The following objects are masked from ‘package:base’:

    as.Date, as.Date.numeric

Loading required package: Hmisc
Loading required package: lattice
Loading required package: survival
Loading required package: Formula
Loading required package: ggplot2
Error: package or namespace load failed for ‘Hmisc’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):
 there is no package called ‘stringi’
Loading required package: grid
Loading required package: gridExtra
 reshape2       zoo     Hmisc   ggplot2      grid gridExtra 
    FALSE      TRUE     FALSE      TRUE      TRUE      TRUE
jeff1evesque commented 6 years ago

Our install_packages function attempts to install dependencies. However, the stringi generates a popup, which the function may not be able to satisfy:

error

This is likely why it is not installed automatically via our custom ist687utility package.

jeff1evesque commented 6 years ago

We will remove the Portal:Current_events, from our result set. However, we may need to revisit this for further analysis.