Dataset updates made by Derilinx

[shandaoz]

Activity stream has shown that Derilinx has been continuously updating 3 datasets in the past few days. The datasets are: Ethiopia - Multi-Tier Framework (MTF) Survey , Cambodia - Multi-Tier Framework (MTF) Survey and Rwanda - Multi-Tier Framework (MTF) Survey . In the case of these 3 datasets, multiple authors are changed to one author only. Why have the updates been made to these datasets? Are there any other datasets that have had their metadata or attached files changed as a result of this strange on-going update?

[jodiegardiner]

This is definitely a bug. I will address asap.

[shandaoz]

That would be much appreciated! ENERGYDATA received an email from an user this morning who claim that their recent data file uploads disappear about a couple of days, and are reverted back to the 'last update date" of November 14, 2018. https://energydata.info/dataset/esmap-solar-measurements-in-vietnam

The user also asks if they could upload the data files to a (S)FTP server or at least being able to zip the QC data files, because they are about to upload a 2GB files, which would take a long time. Their original message are included below.

[shandaoz]

I fear there are more datasets that may have lost files or had changes reverted since December. Would there be a way to track down these datasets?

[jodiegardiner]

Hi Shandao,

I spent a lot of time looking at this, specifically the reason behind several updates showing on a single day for a single dataset. I have identified the cause (there is a short loop executing when a resource update triggers. This is starting a process off which then updates the resource and then triggers another activity stream notification). This is in-hand and a fix for this should be implemented soon.

There's a separate issue with data being overwritten. This linked dataset is harvested from DDH and as agreed, DDH is the source of truth. Updates to datasets should be made on DDH and then they will proliferate to the ENERGYDATA portal. Currently, if a dataset has been previously harvested TO DDH THEN is updated on ENERGYDATA, any changes made to that dataset will be overwritten when the harvester next runs. We can talk today if this is not the way it should work now.

[shandaoz]

Hi Jodie,

Thanks for looking into this! For the a lot of the over ridden data on ENERGYDATA, their data contributors are from outside of the bank. Where can we see the emails of all members under World Bank Organization, so we can notify them and have them update the data on DDH?

[jodiegardiner]

Hi Shandao,

That info can be found in the database of course, that seemed the easiest way to get at it quickly.

Here is the output for all users who are part of the World Bank Org in CKAN:

Albertine Potter van Loon vpottervanloon@worldbank.org Christopher Arderne carderne@worldbank.org christian-peratsakis-207 christian.peratsakis@socrata.com Clara Ivanescu civanescu@worldbank.org Roman Affolter meteo@cspservices.de Dany Jones djones@worldbank.org TG trenton.gilbert@dnvgl.com Shant Dokouzian shant.dokouzian@dnvgl.com Fowzi Dahhan fowzi.dahhan@dnvgl.com Helga Treichel htreichel@worldbank.org Jonathan Davidar jdavidar@worldbank.org joana-zerbin-7559 joana.zerbin@suntrace.de KTH_Division of Energy Systems Analysis admin@desa.kth.se Luc Dewilde luc.dewilde@3E.eu Margot King margot.king@geosun.co.za Dimitris Mentis mentis@kth.se María Vicenta Guisado Otero mvguisado@cener.com Oliver Knight oknight@worldbank.org Olaf Veerman olaf.veerman@gmail.com Pep Bardouille pbardouille@ifc.org Rafael Jimenez Alcaide rjimenezalcaide@worldbank.org Branislav Schnierer branislav.schnierer@solargis.com Shandao Zhou shandaoz@gwmail.gwu.edu Shandao Test shandaoz@gwu.edu Shujun Zhong sz469@cornell.edu Tim Herzog therzog1@worldbank.org Tigran Parvanyan tparvanyan@ifc.org Yann Tanvez yann.tanvez@gmail.com MTF Team ylin3@worldbank.org Haroun Beltaifa hbeltaifa@worldbank.org