First, many thanks for the wonderful pdftools package. Second, if my question is more suitable for another forum (i.e. stackoverflow) please let me know.
I am trying to extract info from about a 1000 voting records. Overall, pdftools works as intended. However, there are one or two hundred pdfs where the extracted text is not properly read. Instead of the text, the first lines (= header) consist of some 'arbitrary' characters .
As far as I can tell the only difference between those which work and those which don't is
a) the font (those which do not work contain Helvetica) and
b) the way how the pdf was created. It seems that those pdf which work fine were created in Excle while those which don't work were created with the 'Microsoft Reporting Service PDF Rendering Extension 10.0.0.0'.
It might be a coincidence, but there seems to be a pattern. I was wondering whether this behavior is known and whether there is a way how to address it? I saw that there was already an issue flagged with regard to Helvetica. I have Helvetica and Helvetica bold installed and imported with the extrafont package into R (not sure whether the last step is actually necessary).
Many thanks!
Below my reprex.
library(tidyverse, suppressWarnings())
library(pdftools)
link.ok <- "http://parlament.ba/oLaw/GetOwisDocument/?documentId=103320&data=9A4B80C1CB2A8BA6617949C64776F770&lang=bs"
link.notok <- "http://parlament.ba/oLaw/GetOwisDocument/?documentId=125266&data=BE111A5A0468C6C5A5A926FE35A37965&lang=bs"
link.notok2 <- "http://parlament.ba/oLaw/GetOwisDocument/?documentId=132385&data=9AAAAC17DF6FCD761F951F58799E63ED&lang=bs"
c(link.ok, link.notok, link.notok2) %>%
map(., pdftools::pdf_text)
#> [[1]]
#> [1] " Rezultati glasanja\r\nSjednica: 41. sjednica Doma naroda Parlamentarne skupštine Bosne i Hercegovine\r\nDatum održavanja: 2014-09-10 12:18\r\nNacin glasanja: otvoreno\r\nRedni broj: Tacka 4\r\nGlasanje o: Prijedlogu zakona o izmjenama i dopunama Zakona o politici direktnih\r\n stranih ulaganja u Bosnu i Hercegovinu (prvo citanje), predlagac: Vijece\r\n ministara BiH, zakon broj: 01,02-02-1-30/14, od 17.7.2014.;\r\nTip glasanje: Prvi krug\r\n Prisutno 13\r\n Ukupno 13\r\n ZA 12\r\n PROTIV 0\r\n SUZDRŽAN 1\r\n Sulejman Tihic Delegat FBiH Nije prisutan\r\n Halid Genjac Delegat FBiH ZA\r\n Nermina Kapetanovic Delegat FBiH ZA\r\n Seudin Hodžic Delegat FBiH Nije prisutan\r\n Mehmed Bradaric Delegat FBiH ZA\r\n Dragan Covic Delegat FBiH ZA\r\n Martin Raguž Delegat FBiH ZA\r\n Stjepan Krešic Delegat FBiH ZA\r\n Krunoslav Vrdoljak Delegat FBiH ZA\r\n Borjana Krišto Delegat FBiH ZA\r\n Ognjen Tadic Delegat RS ZA\r\n Dragutin Rodic Delegat RS ZA\r\n Mladen Ivanic Delegat RS SUZDRŽAN\r\n Krstan Simic Delegat RS ZA\r\n Staša Košarac Delegat RS ZA\r\n ZA PROTIV SUZDRŽAN NIJE PRISUTAN UKUPNO\r\n FBiH 8 0 0 2 10\r\n RS 4 0 1 0 5\r\n UKUPNO 12 0 1 2 15\r\n"
#>
#> [[2]]
#> [1] " 20\021\003VMHGQLFD\0033UHGVWDYQLþNRJ\003GRPD\0033DUODPHQWDUQH\003VNXSãWLQH\003%RVQH\003L\003+HUFHJRYLQH\r\n - 24.11.2015. -\r\nRedni broj glasanja: 12\r\nDatum i vrijeme glasanja: 24.11.2015. 21:13\r\n1DþLQ\003JODVDQMD\035 Javno\r\n5HGQL\003EURM\003WDþNH\035 4\r\n 3ULMHGORJ\003]DNRQD\003R\003D]LOX\017\003SUHGODJDþ\035\0039LMHüH\003PLQLVWDUD\003%L+\017\003]DNRQ\003EURM\035\003\023\024\017\023\025\020\r\n7DþND\003GQHYQRJ\003UHGD\035\r\n \023\025\020\024\020\024\025\022\024\030\017\003RG\003\031\021\033\021\025\023\024\030\021\003 GUXJR\003þLWDQMH\r\n Amandman I. poslanika Damira Arnauta\r\nGlasanje o: Amandman II. poslanika Damira Arnauta\r\n Amandman III. poslanika Damira Arnauta\r\nTip glasanja: Prvi krug\r\nPrisutan 37\r\nNije prisutan 5\r\nUkupno 42\r\nZA 26\r\nPROTIV 11\r\nSUZDRŽAN 0\r\n1. Borjana Krišto Federacija BiH Nije prisutan\r\n2. 0ODGHQ\003\003%RVLü\003 Republika Srpska Za\r\n3. âHILN\003\003'åDIHURYLü Federacija BiH Za\r\n4. $VLP\003\0036DUDMOLü\003 Federacija BiH Za\r\n5. 6DIHU\003\003'HPLURYLü\003 Federacija BiH Za\r\n6. $PLU\003\003)D]OLü Federacija BiH Za\r\n7. 1HUPLQD\003\003.DSHWDQRYLü\003 Federacija BiH Za\r\n8. âHPVXGLQ\003\0030HKPHGRYLü\003 Federacija BiH Za\r\n9. +D]LP\0035DQþLü Federacija BiH Za\r\n10. 6DONR\003\0036RNRORYLü\003 Federacija BiH Za\r\n11. 6HQDG\003\003âHSLü\003 Federacija BiH Za\r\n12. 6DGLN\003\003$KPHWRYLü\003 Republika Srpska Za\r\n13. äHOMNR\003.RPãLü Federacija BiH Nije prisutan\r\n14. 'DPLU\003\003%HüLURYLü\003 Federacija BiH Za\r\n15. 0HQVXUD\003%HJDQRYLü\003 Federacija BiH Za\r\n16. Jovan Vukovljak Republika Srpska Protiv\r\n17. 0LURVODY\003\0030LORYDQRYLü\003 Republika Srpska Protiv\r\n18. /D]DU\003\0033URGDQRYLü\003 Republika Srpska Protiv\r\n19. 0LOLFD\003\0030DUNRYLü\003 Republika Srpska Nije prisutan\r\n20. 'XãDQND\003\0030DMNLü\003 Republika Srpska Protiv\r\n21. Staša Košarac Republika Srpska Protiv\r\n"
#> [2] "22. 1LNROD\003âSLULü Republika Srpska Protiv\r\n23. Maja Gasal-Vražalica Federacija BiH Za\r\n24. +DQND\003\0039DM]RYLü\003 Federacija BiH Za\r\n25. Predrag Kožul Federacija BiH Za\r\n26. 0RQLND\003\0037RPLü\003 Federacija BiH Za\r\n27. 1LNROD\003\003/RYULQRYLü\003 Federacija BiH Za\r\n28. Diana Zelenika Federacija BiH Za\r\n29. 1HQDG\003\003/DOLü\003 Republika Srpska Protiv\r\n30. CRUÿR\003\003.UþPDU\003 Republika Srpska Protiv\r\n31. %RULVODY\003\003%RMLü\003 Republika Srpska Nije prisutan\r\n32. $OHNVDQGUD\003\0033DQGXUHYLü\003 Republika Srpska Protiv\r\n33. 0LUVDG\003\003CRQODJLü\003 Federacija BiH Za\r\n34. Damir Arnaut Federacija BiH Za\r\n35. 0LUVDG\003\003,VDNRYLü\003 Federacija BiH Za\r\n36. )HKLP\003\003âNDOMLü\003 Federacija BiH Nije prisutan\r\n37. 0LUVDG\003\0030HãLü\003 Federacija BiH Za\r\n38. 'HQLV\003\003%HüLURYLü\003 Federacija BiH Protiv\r\n39. 6DãD\003\0030DJD]LQRYLü\003 Federacija BiH Za\r\n40. =DLP\003\003%DFNRYLü\003 Federacija BiH Za\r\n41. -DVPLQ\003\003(PULü\003 Federacija BiH Za\r\n42. 0RPþLOR\003\0031RYDNRYLü\003 Republika Srpska Protiv\r\n ZA PROTIV SUZDRŽAN NIJE PRISUTAN UKUPNO\r\nFBIH 24 1 0 3 28\r\nRS 2 10 0 2 14\r\nUKUPNO 26 11 0 5 42\r\n"
#>
#> [[3]]
#> [1] " 25\021\003VMHGQLFD\0033UHGVWDYQLþNRJ\003GRPD\0033DUODPHQWDUQH\003VNXSãWLQH\003%RVQH\003L\003+HUFHJRYLQH\r\n - 16.02.2016. -\r\nRedni broj glasanja: 6\r\nDatum i vrijeme glasanja: 16.02.2016. 16:21\r\n1DþLQ\003JODVDQMD\035 Javno\r\n5HGQL\003EURM\003WDþNH\035 4\r\n Prijedlog zakona o izmjenama i dopunama Zakona o službi u Oružanim\r\n7DþND\003GQHYQRJ\003UHGD\035 VQDJDPD\003%RVQH\003L\003+HUFHJRYLQH\017\003SUHGODJDþ\035\0039LMHüH\003PLQLVWDUD\003%L+\017\003]DNRQ\003EURM\035\003\r\n \023\024\017\023\025\020\023\025\020\024\020\025\031\022\024\030\017\003RG\003\024\032\021\024\025\021\025\023\024\030\021\003 SUYR\003þLWDQMH\r\nTip glasanja: Prvi krug\r\nPrisutan 35\r\nNije prisutan 7\r\nUkupno 42\r\nZA 30\r\nPROTIV 0\r\nSUZDRŽAN 5\r\n1. âHILN\003\003'åDIHURYLü Federacija BiH Za\r\n2. Borjana Krišto Federacija BiH Za\r\n3. 0ODGHQ\003\003%RVLü\003 Republika Srpska Za\r\n4. $PLU\003\003)D]OLü Federacija BiH Za\r\n5. 6DIHU\003\003'HPLURYLü\003 Federacija BiH Za\r\n6. 1HUPLQD\003\003.DSHWDQRYLü\003 Federacija BiH Nije prisutan\r\n7. âHPVXGLQ\003\0030HKPHGRYLü\003 Federacija BiH Nije prisutan\r\n8. +D]LP\0035DQþLü Federacija BiH Nije prisutan\r\n9. $VLP\003\0036DUDMOLü\003 Federacija BiH Za\r\n10. 6DONR\003\0036RNRORYLü\003 Federacija BiH Za\r\n11. 6HQDG\003\003âHSLü\003 Federacija BiH Za\r\n12. 6DGLN\003\003$KPHWRYLü\003 Republika Srpska Za\r\n13. äHOMNR\003.RPãLü Federacija BiH Nije prisutan\r\n14. 'DPLU\003\003%HüLURYLü\003 Federacija BiH Suzdržan\r\n15. 0HQVXUD\003%HJDQRYLü\003 Federacija BiH Suzdržan\r\n16. Jovan Vukovljak Republika Srpska Za\r\n17. 0LURVODY\003\0030LORYDQRYLü\003 Republika Srpska Za\r\n18. /D]DU\003\0033URGDQRYLü\003 Republika Srpska Za\r\n19. 0LOLFD\003\0030DUNRYLü\003 Republika Srpska Za\r\n20. 'XãDQND\003\0030DMNLü\003 Republika Srpska Za\r\n21. Staša Košarac Republika Srpska Za\r\n22. 1LNROD\003âSLULü Republika Srpska Nije prisutan\r\n23. Maja Gasal-Vražalica Federacija BiH Suzdržan\r\n24. +DQND\003\0039DM]RYLü\003 Federacija BiH Suzdržan\r\n"
#> [2] "25. Predrag Kožul Federacija BiH Za\r\n26. 0RQLND\003\0037RPLü\003 Federacija BiH Za\r\n27. 1LNROD\003\003/RYULQRYLü\003 Federacija BiH Za\r\n28. Diana Zelenika Federacija BiH Nije prisutan\r\n29. 1HQDG\003\003/DOLü\003 Republika Srpska Za\r\n30. CRUÿR\003\003.UþPDU\003 Republika Srpska Nije prisutan\r\n31. %RULVODY\003\003%RMLü\003 Republika Srpska Za\r\n32. $OHNVDQGUD\003\0033DQGXUHYLü\003 Republika Srpska Za\r\n33. 0LUVDG\003\003CRQODJLü\003 Federacija BiH Za\r\n34. Damir Arnaut Federacija BiH Za\r\n35. 0LUVDG\003\003,VDNRYLü\003 Federacija BiH Za\r\n36. )HKLP\003\003âNDOMLü\003 Federacija BiH Za\r\n37. 0LUVDG\003\0030HãLü\003 Federacija BiH Za\r\n38. 'HQLV\003\003%HüLURYLü\003 Federacija BiH Suzdržan\r\n39. 6DãD\003\0030DJD]LQRYLü\003 Federacija BiH Za\r\n40. =DLP\003\003%DFNRYLü\003 Federacija BiH Za\r\n41. -DVPLQ\003\003(PULü\003 Federacija BiH Za\r\n42. 0RPþLOR\003\0031RYDNRYLü\003 Republika Srpska Za\r\n ZA PROTIV SUZDRŽAN NIJE PRISUTAN UKUPNO\r\nFBIH 18 0 5 5 28\r\nRS 12 0 0 2 14\r\nUKUPNO 30 0 5 7 42\r\n"
sessionInfo()
#> R version 3.5.1 (2018-07-02)
#> Platform: i386-w64-mingw32/i386 (32-bit)
#> Running under: Windows 10 x64 (build 16299)
#>
#> Matrix products: default
#>
#> locale:
#> [1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252
#> [3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C
#> [5] LC_TIME=German_Germany.1252
#>
#> attached base packages:
#> [1] stats graphics grDevices utils datasets methods base
#>
#> other attached packages:
#> [1] pdftools_2.2 forcats_0.4.0 stringr_1.3.1 dplyr_0.8.0.1
#> [5] purrr_0.3.2 readr_1.3.1 tidyr_0.8.3 tibble_2.1.1
#> [9] ggplot2_3.1.1 tidyverse_1.2.1
#>
#> loaded via a namespace (and not attached):
#> [1] withr_2.1.2 rvest_0.3.2 tidyselect_0.2.5 lattice_0.20-35
#> [5] pkgconfig_2.0.2 xml2_1.2.0 compiler_3.5.1 readxl_1.3.1
#> [9] Rcpp_1.0.1 cli_1.1.0 plyr_1.8.4 cellranger_1.1.0
#> [13] highr_0.7 httr_1.4.0 askpass_1.1 tools_3.5.1
#> [17] nlme_3.1-137 broom_0.5.2 rmarkdown_1.12 R6_2.4.0
#> [21] knitr_1.22 scales_1.0.0 assertthat_0.2.0 digest_0.6.18
#> [25] gtable_0.2.0 evaluate_0.13 qpdf_1.1 stringi_1.4.3
#> [29] backports_1.1.2 htmltools_0.3.6 hms_0.4.2 munsell_0.5.0
#> [33] grid_3.5.1 colorspace_1.3-2 glue_1.3.1 lubridate_1.7.4
#> [37] rlang_0.3.4 magrittr_1.5 generics_0.0.2 lazyeval_0.2.2
#> [41] yaml_2.2.0 crayon_1.3.4 haven_2.1.0 xfun_0.6
First, many thanks for the wonderful pdftools package. Second, if my question is more suitable for another forum (i.e. stackoverflow) please let me know.
I am trying to extract info from about a 1000 voting records. Overall, pdftools works as intended. However, there are one or two hundred pdfs where the extracted text is not properly read. Instead of the text, the first lines (= header) consist of some 'arbitrary' characters .
As far as I can tell the only difference between those which work and those which don't is a) the font (those which do not work contain Helvetica) and b) the way how the pdf was created. It seems that those pdf which work fine were created in Excle while those which don't work were created with the 'Microsoft Reporting Service PDF Rendering Extension 10.0.0.0'.
It might be a coincidence, but there seems to be a pattern. I was wondering whether this behavior is known and whether there is a way how to address it? I saw that there was already an issue flagged with regard to Helvetica. I have Helvetica and Helvetica bold installed and imported with the extrafont package into R (not sure whether the last step is actually necessary).
Many thanks!
Below my reprex.