Open sdspieg opened 3 years ago
Dear Stephan, I promise I will try to do it asap.
Grazie mille! :)
Any progress on this?
We are working on it. Please don't ask for ETA.
Just added support to lens.org CSV files.
Let me know any issues and bugs.
to import a lens.org file using convert2df: M <- convert2df(file="mylensfile.csv", dbsource="lens", format="csv)
using biblioshiny, you have just to choose"lens.org" when selecting the database in the "import raw file" menu.
Great! Thanks much. A few comments.
value
should be greater than or equal to min
(value = 1, min = 3)."Great! Thanks much. A few comments.
- the conversion seems to work partially. My biggest Lens CSV file has 49,687 scholarly docs, and the conversion results show only 15,381 (RStudio shows an error message: "Warning: In sliderInput():
value
should be greater than or equal tomin
(value = 1, min = 3)."- annual scientific production, sources, works (for those 15k)
- authors works for most fields that are in the CSV (e.g. affiliations are not in there, so that does not work), but for. Also, the Author local impact took 45' to compute [to be continued...]
convert2df works fine with your CSV file (you sent me it in a previous message)
Moreover, biblioshiny works well too (using the same CSV file)
Please provide more info about errors you are facing.
Regarding the high computational time of some functions (i.e. author local impact), that's normal when analyzing a such big collection.
I have completely rewritten the function Hindex that performs author and source impact analysis. Now, with your collection of quite 50.000 rows, author impact calculation takes just 30 seconds (on my MacBook air i5) instead of 45 minutes!!
sadly, lens it crashes for me with a message: "Error in strsplit(DATA$AU, split = "; ") : non-character argument" It may need some additional cleaning step?
sadly, lens it crashes for me with a message: "Error in strsplit(DATA$AU, split = "; ") : non-character argument" It may need some additional cleaning step?
I need more info about your issue. Please, provide your code and CSV lens file.
Hi, I am attaching a file (had to save it as .xlsx as it was not possible to upload .csv here) - it was originally downloaded from Lens.org as a .csv export, as recommended earlier in this issue. Unfortunately, exporting to .csv messes up encoding of special characters, e.g.: "Neal R. Haddaway; Colin D. Brown; Sönke Eggers; Jonas Josefsson; Brian Kronvang; Nicola P. Randall; Jaana Uusi-Kämppä" .
Working with .bib exports has no such problem (but does not work for importing data into bibliometrix). However, it looks like not all fields get exported. For example, the field with references gets lost, which makes any bibliometric analyses basing on lists of citations impossible.
bib_lens <- convert2df("lens-export.csv", dbsource = "lens", format = "csv") #using a recent version of the function from GitHub, not CRAN
dim(bib_lens) #crashes with error message: Error in strsplit(DATA$AU, split = "; ") : non-character argument
Hi, I am attaching a file (had to save it as .xlsx as it was not possible to upload .csv here) - it was originally downloaded from Lens.org as a .csv export, as recommended earlier in this issue. Unfortunately, exporting to .csv messes up encoding of special characters, e.g.: "Neal R. Haddaway; Colin D. Brown; Sönke Eggers; Jonas Josefsson; Brian Kronvang; Nicola P. Randall; Jaana Uusi-Kämppä" .
Working with .bib exports has no such problem (but does not work for importing data into bibliometrix). However, it looks like not all fields get exported. For example, the field with references gets lost, which makes any bibliometric analyses basing on lists of citations impossible.
bib_lens <- convert2df("lens-export.csv", dbsource = "lens", format = "csv") #using a recent version of the function from GitHub, not CRAN dim(bib_lens) #crashes with error message: Error in strsplit(DATA$AU, split = "; ") : non-character argument
You raised two points: Differences among Lens export file formats and an importing issue with CSV. Focusing on the first point, Lens allows exporting collections using three different file formats: RIS, CSV, and BibTeX. Currently, none of the three formats allow you to export the entire set of metadata (i.e no format provides author's affiliation info). Comparing the three files (CSV, RIS, and BibTeX), CSV appears to be more complete. That's why we choose to include it in bibliometrix.
Regarding the importing issue, I need the original CSV you downloaded from Lens. Please zip and upload it on GitHub. I cannot identify the issue through the XLSX file.
lens-export.csv.zip thank you for your reply - here is the zipped file
I just tried to import your lens file and all works fine on my pc. Maybe it could depend on your R enconding setting that returns an error when trying to import foreign characters. Please execute the following code in R console and let me know what is your encoding:
getOption("encoding")
lens-export.csv.zip thank you for your reply - here is the zipped file
Hi, thanks a lot - it looks like it might have something to do with encoding on my computer. I would appreciate any suggestions on how to get around this issue.
Here is my encoding and session info:
getOption("encoding") [1] "native.enc"
sessionInfo() R version 3.6.0 (2019-04-26) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Sierra 10.12.6
Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale: [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8
attached base packages: [1] stats graphics grDevices utils datasets methods base ...
Hi, thanks a lot - it looks like it might have something to do with encoding on my computer. I would appreciate any suggestions on how to get around this issue.
Here is my encoding and session info:
getOption("encoding") [1] "native.enc"
sessionInfo() R version 3.6.0 (2019-04-26) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Sierra 10.12.6
Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib
locale: [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8
attached base packages: [1] stats graphics grDevices utils datasets methods base ...
I am trying to fix your issue. Please try if now lens importing works fine.
Hello and thank you for your work. I have a problem importing from lens.org to biblioshiny. Everything seems fine, no errors but in the Most Global Cited Documents analysis, all the documents appear with 0 citations. The same happens in Average citations per documents. How can I fix it? thanks
This is not an issue. Lens.org database does not export citation counts. So the bibliometrix importing routine set TC column to 0 for all documents.
This is not an issue. Lens.org database does not export citation counts. So the bibliometrix importing routine set TC column to 0 for all documents.
Thanks!
Hi, I am attaching a file (had to save it as .xlsx as it was not possible to upload .csv here) - it was originally downloaded from Lens.org as a .csv export, as recommended earlier in this issue. Unfortunately, exporting to .csv messes up encoding of special characters, e.g.: "Neal R. Haddaway; Colin D. Brown; Sönke Eggers; Jonas Josefsson; Brian Kronvang; Nicola P. Randall; Jaana Uusi-Kämppä" . Working with .bib exports has no such problem (but does not work for importing data into bibliometrix). However, it looks like not all fields get exported. For example, the field with references gets lost, which makes any bibliometric analyses basing on lists of citations impossible.
bib_lens <- convert2df("lens-export.csv", dbsource = "lens", format = "csv") #using a recent version of the function from GitHub, not CRAN dim(bib_lens) #crashes with error message: Error in strsplit(DATA$AU, split = "; ") : non-character argument
You raised two points: Differences among Lens export file formats and an importing issue with CSV. Focusing on the first point, Lens allows exporting collections using three different file formats: RIS, CSV, and BibTeX. Currently, none of the three formats allow you to export the entire set of metadata (i.e no format provides author's affiliation info). Comparing the three files (CSV, RIS, and BibTeX), CSV appears to be more complete. That's why we choose to include it in bibliometrix.
Regarding the importing issue, I need the original CSV you downloaded from Lens. Please zip and upload it on GitHub. I cannot identify the issue through the XLSX file.
Hi!
I'm not sure when this was implemented, but currently Lens allows to export data in .json and .jsonl formats, both of which provide author affiliation data. Are there any plans to include options for parsing these files in future releases of bibliometrix?
Thanks for this amazing software!
Did something break or change? When I try to import an exported csv-file through the gui I first see this `Converting your lens collection into a bibliographic dataframe
Rows: 17309 Columns: 32
── Column specification ──────────────────────────────────────────
Delimiter: ","
chr (26): Lens ID, Title, Publication Type, Source Title, ISS...
dbl (4): Publication Year, PMID, Citing Patents Count, Citin...
lgl (1): Is Open Access
date (1): Date Published
ℹ Use spec()
to retrieve the full column specification for this data.
ℹ Specify the column types or set show_col_types = FALSE
to quiet this message.`
But then it just keeps spinning and spinning... Could you please take a look? https://drive.google.com/open?id=1G6uZEijKJ94sJxmXOneFXGjF5DqLtFNH&usp=drive_fs
Update - it just took very long. But it DID complete successfully
Hi @massimoaria, I'm having the same issue here, with the following error message. Is there anything you can suggest?
M <- convert2df(file, dbsource = "lens", format = "csv")
Error in strsplit(DATA$AU, split = "; ") : non-character argument
Did something break or change? When I try to import an exported csv-file through the gui I first see this `Converting your lens collection into a bibliographic dataframe
Rows: 17309 Columns: 32 ── Column specification ────────────────────────────────────────── Delimiter: "," chr (26): Lens ID, Title, Publication Type, Source Title, ISS... dbl (4): Publication Year, PMID, Citing Patents Count, Citin... lgl (1): Is Open Access date (1): Date Published
ℹ Use
spec()
to retrieve the full column specification for this data. ℹ Specify the column types or setshow_col_types = FALSE
to quiet this message.`But then it just keeps spinning and spinning... Could you please take a look?
https://drive.google.com/open?id=1G6uZEijKJ94sJxmXOneFXGjF5DqLtFNH&usp=drive_fs
Did something break or change? When I try to import an exported csv-file through the gui I first see this `Converting your lens collection into a bibliographic dataframe
Rows: 17309 Columns: 32 ── Column specification ────────────────────────────────────────── Delimiter: "," chr (26): Lens ID, Title, Publication Type, Source Title, ISS... dbl (4): Publication Year, PMID, Citing Patents Count, Citin... lgl (1): Is Open Access date (1): Date Published
ℹ Use
spec()
to retrieve the full column specification for this data. ℹ Specify the column types or setshow_col_types = FALSE
to quiet this message.`But then it just keeps spinning and spinning... Could you please take a look?
https://drive.google.com/open?id=1G6uZEijKJ94sJxmXOneFXGjF5DqLtFNH&usp=drive_fs
I'm having the same problem. Does anyone know how to solve it?
Did something break or change? When I try to import an exported csv-file through the gui I first see this
Converting your lens collection into a bibliographic dataframe Rows: 17309 Columns: 32 ── Column specification ────────────────────────────────────────── Delimiter: "," chr (26): Lens ID, Title, Publication Type, Source Title, ISS... dbl (4): Publication Year, PMID, Citing Patents Count, Citin... lgl (1): Is Open Access date (1): Date Published ℹ Use
spec()to retrieve the full column specification for this data. ℹ Specify the column types or set
show_col_types = FALSEto quiet this message.
But then it just keeps spinning and spinning... Could you please take a look?https://drive.google.com/open?id=1G6uZEijKJ94sJxmXOneFXGjF5DqLtFNH&usp=drive_fs
Did something break or change? When I try to import an exported csv-file through the gui I first see this
Converting your lens collection into a bibliographic dataframe Rows: 17309 Columns: 32 ── Column specification ────────────────────────────────────────── Delimiter: "," chr (26): Lens ID, Title, Publication Type, Source Title, ISS... dbl (4): Publication Year, PMID, Citing Patents Count, Citin... lgl (1): Is Open Access date (1): Date Published ℹ Use
spec()to retrieve the full column specification for this data. ℹ Specify the column types or set
show_col_types = FALSEto quiet this message.
But then it just keeps spinning and spinning... Could you please take a look?https://drive.google.com/open?id=1G6uZEijKJ94sJxmXOneFXGjF5DqLtFNH&usp=drive_fs
I'm having the same problem. Does anyone know how to solve it?
Hi, using the last bibliometrix version 4.3.0, Lens files work fine on my PC. Here is a reproducible example using your dataset:
library(bibliometrix)
#> Please note that our software is open source and available for use, distributed under the MIT license.
#> When it is used in a publication, we ask that authors properly cite the following reference:
#>
#> Aria, M. & Cuccurullo, C. (2017) bibliometrix: An R-tool for comprehensive science mapping analysis,
#> Journal of Informetrics, 11(4), pp 959-975, Elsevier.
#>
#> Failure to properly cite the software is considered a violation of the license.
#>
#> For information and bug reports:
#> - Take a look at https://www.bibliometrix.org
#> - Send an email to info@bibliometrix.org
#> - Write a post on https://github.com/massimoaria/bibliometrix/issues
#>
#> Help us to keep Bibliometrix and Biblioshiny free to download and use by contributing with a small donation to support our research team (https://bibliometrix.org/donate.html)
#>
#>
#> To start with the Biblioshiny app, please digit:
#> biblioshiny()
file <- "/Users/massimoaria/Downloads/lens-export(5).csv"
M <- convert2df(file, dbsource = "lens", format = "csv")
#>
#> Converting your lens collection into a bibliographic dataframe
#> Warning: One or more parsing issues, call `problems()` on your data frame for details,
#> e.g.:
#> dat <- vroom(...)
#> problems(dat)
#> Rows: 17309 Columns: 32
#> ── Column specification ────────────────────────────────────────────────────────
#> Delimiter: ","
#> chr (26): Lens ID, Title, Publication Type, Source Title, ISSNs, Publisher,...
#> dbl (4): Publication Year, PMID, Citing Patents Count, Citing Works Count
#> lgl (1): Is Open Access
#> date (1): Date Published
#>
#> ℹ Use `spec()` to retrieve the full column specification for this data.
#> ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.
#> Done!
summary(biblioAnalysis(M))
#>
#>
#> MAIN INFORMATION ABOUT DATA
#>
#> Timespan 1855 : 2023
#> Sources (Journals, Books, etc) 3956
#> Documents 17309
#> Annual Growth Rate % 4.07
#> Document Average Age 11.1
#> Average citations per doc 3.138
#> Average citations per year per doc 0.3135
#> References 70533
#>
#> DOCUMENT TYPES
#> 3175
#> book 613
#> book chapter 1463
#> component 3
#> conference proceedings 484
#> conference proceedings article 649
#> dataset 19
#> dissertation 215
#> editorial 11
#> journal 2
#> journal article 10305
#> journal issue 39
#> letter 9
#> libguide 21
#> news 9
#> other 163
#> preprint 30
#> reference entry 12
#> report 73
#> review 14
#>
#> DOCUMENT CONTENTS
#> Keywords Plus (ID) 719
#> Author's Keywords (DE) 719
#>
#> AUTHORS
#> Authors 23488
#> Author Appearances 36260
#> Authors of single-authored docs 6632
#>
#> AUTHORS COLLABORATION
#> Single-authored docs 9468
#> Documents per Author 0.737
#> Co-Authors per Doc 2.09
#> International co-authorships % 0
#>
#>
#> Annual Scientific Production
#>
#> Year Articles
#> 1855 1
#> 1861 1
#> 1871 1
#> 1872 1
#> 1876 1
#> 1877 1
#> 1879 1
#> 1880 3
#> 1883 2
#> 1885 3
#> 1888 1
#> 1894 7
#> 1902 1
#> 1904 1
#> 1905 2
#> 1908 1
#> 1912 1
#> 1915 4
#> 1916 3
#> 1917 1
#> 1921 2
#> 1922 4
#> 1923 2
#> 1924 1
#> 1925 3
#> 1926 2
#> 1927 3
#> 1928 2
#> 1929 1
#> 1930 1
#> 1931 2
#> 1932 2
#> 1935 2
#> 1940 2
#> 1941 5
#> 1942 2
#> 1943 18
#> 1944 7
#> 1945 2
#> 1946 3
#> 1947 6
#> 1948 4
#> 1949 1
#> 1950 4
#> 1951 1
#> 1952 1
#> 1953 1
#> 1954 2
#> 1956 1
#> 1957 13
#> 1958 4
#> 1959 11
#> 1960 5
#> 1961 2
#> 1962 5
#> 1963 18
#> 1964 3
#> 1965 8
#> 1966 6
#> 1967 7
#> 1968 14
#> 1969 6
#> 1970 10
#> 1971 11
#> 1972 8
#> 1973 7
#> 1974 12
#> 1975 5
#> 1976 10
#> 1977 11
#> 1978 3
#> 1979 7
#> 1980 7
#> 1981 8
#> 1982 25
#> 1983 18
#> 1984 7
#> 1985 11
#> 1986 5
#> 1987 9
#> 1988 7
#> 1989 7
#> 1990 7
#> 1991 10
#> 1992 70
#> 1993 98
#> 1994 118
#> 1995 128
#> 1996 144
#> 1997 124
#> 1998 163
#> 1999 145
#> 2000 164
#> 2001 172
#> 2002 179
#> 2003 673
#> 2004 252
#> 2005 198
#> 2006 260
#> 2007 300
#> 2008 348
#> 2009 403
#> 2010 340
#> 2011 440
#> 2012 584
#> 2013 510
#> 2014 694
#> 2015 868
#> 2016 808
#> 2017 712
#> 2018 983
#> 2019 998
#> 2020 1359
#> 2021 1446
#> 2022 1802
#> 2023 820
#>
#> Annual Percentage Growth Rate 4.07
#>
#>
#> Most Productive Authors
#>
#> Authors Articles Authors Articles Fractionalized
#> 1 В 285 VOLOVIK N 66.0
#> 2 А 186 ВЛАДИМИРОВИЧ 45.7
#> 3 VOLOVIK N 84 BLANK S 44.7
#> 4 ВЛАДИМИРОВИЧ 82 SUCHKOV MA 38.0
#> 5 НИКОЛАЕВИЧ 78 TSYGANKOV AP 32.8
#> 6 С 76 В 32.6
#> 7 Н 75 НИКОЛАЕВИЧ 31.3
#> 8 АЛЕКСАНДРОВИЧ 65 АЛЕКСАНДРОВИЧ 25.3
#> 9 И 55 А 23.7
#> 10 М 55 LO B 22.3
#>
#>
#> Top manuscripts per citations
#>
#> Paper
#> 1 AIE , 2011, WORLD ENERGY OUTLOOK
#> 2 GLÄNZEL W, 2001, SCIENTOMETRICS
#> 3 GILL P, 1994, NATURE GENETICS
#> 4 FITT BDL, 2006, EUROPEAN JOURNAL OF PLANT PATHOLOGY
#> 5 BOLDYREV VV, 2000, JOURNAL OF MATERIALS SYNTHESIS AND PROCESSING
#> 6 LARSON DW, 2010, INTERNATIONAL SECURITY
#> 7 ZHAO L, 2019, INFOMAT
#> 8 WANG K, 2019, ACS NANO
#> 9 SPASOV AA, 1999, PHARMACEUTICAL CHEMISTRY JOURNAL
#> 10 BARRETT S, 2003, INTERNATIONAL ENVIRONMENTAL AGREEMENTS: POLITICS, LAW AND ECONOMICS
#> DOI TC TCperYear NTC
#> 1 10.1787/weo-2011-en 961 68.6 143.5
#> 2 10.1023/a:1010512628145 611 25.5 70.1
#> 3 10.1038/ng0294-130 600 19.4 67.6
#> 4 10.1007/s10658-005-2233-5 402 21.2 43.5
#> 5 10.1023/a:1011347706721 327 13.1 35.7
#> 6 10.1162/isec.2010.34.4.63 287 19.1 60.9
#> 7 10.1002/inf2.12032 282 47.0 93.2
#> 8 10.1021/acsnano.9b03454 273 45.5 90.2
#> 9 10.1007/bf02510042 269 10.3 26.6
#> 10 10.1023/b:inea.0000005767.67689.28 261 11.9 76.2
#>
#>
#> Most Relevant Sources
#>
#> Sources
#> 1
#> 2 BULLETIN OF THE RUSSIAN MILITARY MEDICAL ACADEMY
#> 3 ЭНЕРГЕТИЧЕСКАЯ ПОЛИТИКА
#> 4 5TH KOREA-RUSSIA INTERNATIONAL SYMPOSIUM ON SCIENCE AND TECHNOLOGY. PROCEEDINGS. KORUS 2001 (CAT. NO.01EX478)
#> 5 SOCIAL SCIENCE RESEARCH NETWORK
#> 6 PROCEEDINGS KORUS 2000. THE 4TH KOREA-RUSSIA INTERNATIONAL SYMPOSIUM ON SCIENCE AND TECHNOLOGY
#> 7 RUSSIAN MILITARY MEDICAL ACADEMY REPORTS
#> 8 SSRN ELECTRONIC JOURNAL
#> 9 EMERALD EXPERT BRIEFINGS
#> 10 STUDIES ON RUSSIAN ECONOMIC DEVELOPMENT
#> Articles
#> 1 3620
#> 2 870
#> 3 325
#> 4 239
#> 5 220
#> 6 211
#> 7 207
#> 8 203
#> 9 175
#> 10 174
#>
#>
#> Most Relevant Keywords
#>
#> Author Keywords (DE) Articles Keywords-Plus (ID) Articles
#> 1 RUSSIA 11 RUSSIA 11
#> 2 POLICY 8 POLICY 8
#> 3 POLITICS 8 POLITICS 8
#> 4 CLIMATE CHANGE 7 CLIMATE CHANGE 7
#> 5 ENERGY 7 ENERGY 7
#> 6 COVID-19 6 COVID-19 6
#> 7 UKRAINE 5 UKRAINE 5
#> 8 GOVERNMENT 4 GOVERNMENT 4
#> 9 ECONOMICS 3 ECONOMICS 3
#> 10 FOREIGN DIRECT INVESTMENT 3 FOREIGN DIRECT INVESTMENT 3
Created on 2024-07-04 with reprex v2.1.0
When I try, this happens.:
Converting your lens collection into a bibliographic dataframe Rows: 166 Columns: 9 ── Column specification ──────────────────────────────────────── Delimiter: "," chr (7): ID da lente, Título, Tipo de Publicação, Título da ... dbl (2): Ano de Publicação, Citando Contagem de Trabalhos ℹ Use `spec()` to retrieve the full column specification for this data. ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message. **Error in strsplit(DATA$AU, split = "; ") : argumento modo não caractere Além disso: Warning message: One or more parsing issues, call `problems()` on your data frame for details, e.g.: dat <- vroom(...) problems(dat) --** > | > >Please provide your lens file
Any plans to also enable us to load exported lens.org files into bibliometrix? It's a great bibliometric resource, because it allows us to download as many documents as we want - admittedly 'only' with Lens IDs, but those should still suffice to even build co-citation networks... And its coverage is also quite a bit bigger than Dimensions - which you already support (thanks!). Pretty please/per favore, caro Massimo ? :)