IQSS / dataverse-installations

code that powers a map of Dataverse installations around the world
https://iqss.github.io/dataverse-installations
Apache License 2.0
6 stars 10 forks source link

What version of Dataverse are the installations on the map running? #3

Open pdurbin opened 5 years ago

pdurbin commented 5 years ago

As of this writing there are 46 installations of Dataverse on the map at https://dataverse.org

Screen Shot 2019-07-03 at 2 34 02 PM

Currently there are 17 installations of Dataverse listed at https://dataverse.org/metrics

Screen Shot 2019-07-03 at 2 35 09 PM

The list of URLs of installations on the map is available in JSON format: https://services.dataverse.harvard.edu/miniverse/map/installations-json

The list of URLs of installations configured for metrics gathering is also available in JSON format: https://dataversemetrics.odum.unc.edu/dataverse-metrics/config.json

For any given installation, you can check the version by visiting, for example, https://dataverse.harvard.edu/api/info/version

The task for this issue is to write a script that reports the version of Dataverse that's being run by each installation on the map. It should also report whether or not it has been configured for metrics gathering or not.

One goal is to continue to add installations running Dataverse 4.9 or higher to the metrics aggregator. The process for this to currently to ping @donsizemore at http://chat.dataverse.org . I suspect that some of the 29 installations of Dataverse that aren't being polled for metrics are running 4.9 or higher.

Another goal is to simply get a sense of what versions the community is running.

pdurbin commented 5 years ago

Here's the quick and dirty one liner I wrote a while ago for determining the version of Dataverse that is being run by installations on the map:

curl -s https://services.dataverse.harvard.edu/miniverse/map/installations-json | jq '.installations[].url' -r | while read i; do echo -n "$i "; curl -s $i/api/info/version | jq '.data.version' -r; done

The problem is that the output is a mess:

https://dvn.library.ubc.ca/dvn/ https://dataverse.ada.edu.au/ 4.6.1 https://data.aussda.at/ 4.6.2 https://dataverse.bhp.org.bw/ null https://catalogues.cdsp.sciences-po.fr/ 4.8.5 https://data.cifor.org 4.14 http://data.cimmyt.org/ parse error: Invalid numeric literal at line 1, column 10 https://dataverse.cirad.fr/ 4.8.4 https://dataverse.library.dal.ca parse error: Invalid numeric literal at line 1, column 10 https://data.inra.fr/ 4.9.2 https://dataspace.ust.hk/ parse error: Invalid numeric literal at line 1, column 6 (23) Failed writing body https://edatos.consorciomadrono.es/ 4.11 https://dataverse.nl/ 4.10.1 https://dataverse.no/ 4.11 https://researchdata.ntu.edu.sg 4.9.4 https://dvn.fudan.edu.cn/home/ https://data.gro.uni-goettingen.de/ 4.14 https://dataverse.harvard.edu 4.15 https://heidata.uni-heidelberg.de/ 4.14 https://repositoriopesquisas.ibict.br/ parse error: Invalid numeric literal at line 1, column 6 http://dataverse.icrisat.org/ 4.8.6 https://dataverse.mpi-sws.org/dataverse/icwsm parse error: Invalid numeric literal at line 1, column 6 https://research-data.ifsttar.fr/dataverse/data parse error: Invalid numeric literal at line 1, column 6 https://datasets.socialhistory.org/ parse error: Invalid numeric literal at line 1, column 6 https://data.cipotato.org/dataverse.xhtml parse error: Invalid numeric literal at line 1, column 6 https://archive.data.jhu.edu/ 4.13 https://dataverse.lib.virginia.edu/ 4.12 https://data.lipi.go.id 4.14 http://dataverse.acg.maine.edu/dvn/ parse error: Invalid numeric literal at line 1, column 6 http://data.mel.cgiar.org 4.8.5 https://researchdata.nie.edu.sg 4.10.1 http://opendata.pku.edu.cn/ 4.14 https://data.qdr.syr.edu 4.14-qdr2 http://dataverse.ufabc.edu.br 4.8.5 http://dataverse.ileel.ufu.br 4.11 http://research-data.urosario.edu.co/ 4.9.4 https://dataverse.scholarsportal.info/ v4.10.1.5-SP https://dataverse.tdl.org/ 4.9.4-tdl https://dataverse.library.ualberta.ca/dvn/ parse error: Invalid numeric literal at line 1, column 6 https://dataverse.ucla.edu/dataverse/dataverse parse error: Invalid numeric literal at line 1, column 6 https://dataverse.lib.unb.ca/ 4.14 https://dataverse.unc.edu/ 4.9.4 https://dataverse.unimi.it/ 4.14 https://dataverse.lib.umanitoba.ca/ null http://dataverse.sta.uwi.edu/ parse error: Invalid numeric literal at line 1, column 10 https://dataverse.vtti.vt.edu/ null

pdurbin commented 5 years ago

I just wrote a little Python script in pull request #4 that I think is better than the one liner above.

The "True" and "False" in the output below indicates if the installation is on https://dataverse.org/metrics

https://dvn.library.ubc.ca UNKNOWN False https://dataverse.ada.edu.au 4.6.1 False https://data.aussda.at 4.6.2 False https://dataverse.bhp.org.bw 4.9.4 False https://catalogues.cdsp.sciences-po.fr 4.8.5 False https://data.cifor.org 4.14 True https://data.cimmyt.org 4.7.1 False https://dataverse.cirad.fr UNKNOWN False https://dataverse.library.dal.ca UNKNOWN False https://data.inra.fr 4.9.2 True https://dataspace.ust.hk UNKNOWN False https://edatos.consorciomadrono.es 4.11 True https://dataverse.nl 4.10.1 False https://dataverse.no 4.11 True https://researchdata.ntu.edu.sg 4.9.4 True https://dvn.fudan.edu.cn UNKNOWN False https://data.gro.uni-goettingen.de 4.14 True https://dataverse.harvard.edu 4.15 True https://heidata.uni-heidelberg.de 4.14 True https://repositoriopesquisas.ibict.br UNKNOWN False https://dataverse.icrisat.org UNKNOWN False https://dataverse.mpi-sws.org UNKNOWN False https://research-data.ifsttar.fr 4.10.1 False https://datasets.socialhistory.org UNKNOWN False https://data.cipotato.org 4.8.1 False https://archive.data.jhu.edu UNKNOWN False https://dataverse.lib.virginia.edu 4.12 True https://data.lipi.go.id 4.14 True https://dataverse.acg.maine.edu UNKNOWN False https://data.mel.cgiar.org UNKNOWN False https://researchdata.nie.edu.sg 4.10.1 False https://opendata.pku.edu.cn UNKNOWN False https://data.qdr.syr.edu 4.14-qdr2 True https://dataverse.ufabc.edu.br UNKNOWN False https://dataverse.ileel.ufu.br UNKNOWN False https://research-data.urosario.edu.co UNKNOWN False https://dataverse.scholarsportal.info v4.10.1.5-SP True https://dataverse.tdl.org 4.9.4-tdl True https://dataverse.library.ualberta.ca 4.9.4 False https://dataverse.ucla.edu 4.14 False https://dataverse.lib.unb.ca 4.14 True https://dataverse.unc.edu 4.9.4 True https://dataverse.unimi.it 4.14 True https://dataverse.lib.umanitoba.ca 4.10.1 False https://dataverse.sta.uwi.edu 4.9.2 False https://dataverse.vtti.vt.edu 4.12 False

donsizemore commented 5 years ago

@pdurbin I just added the hosts from http://irclog.iq.harvard.edu/dataverse/2019-07-03#i_99836 to metrics.dataverse.org. Totals jumped!

skasberger commented 4 years ago

There is a function in pyDataverse, which retrieves the Dataverse build version. This could be used, to automatize the version collection mentioned in the december community call.

https://github.com/AUSSDA/pyDataverse/blob/master/src/pyDataverse/api.py#L1038

donsizemore commented 2 years ago

I too wrote a no-good very-bad horrible script (though its output is hostname,version CSV). Happy to contribute to the dataverse-metrics/global repo or to this repo in case it's helpful.

pdurbin commented 1 year ago

@donsizemore sure, please make a pull request. Thanks!

Unless that was an April Fools joke. šŸ˜„

Sorry for the slow reply! šŸ˜…

donsizemore commented 1 year ago

@pdurbin it wasn't a joke but Jim says the new version of dataverse-metrics also reports this information. It is on my list to upgrade metrics.dataverse.org, but that's waiting on https://github.com/IQSS/dataverse/issues/9016

pdurbin commented 1 year ago

Ah right, this thing...

versions

... from https://github.com/IQSS/dataverse-metrics/issues/50

Sure, sounds fine.