Dataset request: updated electricity data from EMA.

[lieuzhenghong]

I'll dispense with the form, but in essence the data from the EMA that gives energy consumption/production here, here, and here, and here, amongst other places, haven't been updated since 2017.

I'd love to see all of these monthly datasets updated monthly. (As an aside, is there/shouldn't there be some automated pipeline to get data regularly from all of these agencies rather than one-time data dumps?)

Can you give a short explanation on how you're planning to use this dataset?

Mainly exploratory data analysis. I'd love to see how the coronavirus has affected energy production and use (which is of course impossible if the data stops during 2017), and I'd also like to explore how much energy Singapore could conceivably generate with renewable energy.

Cheers, Zhenghong

[lennardl]

@lieuzhenghong Thanks for the feedback, we have informed EMA on the outdated datasets. As for the automated pipeline, it would be left to the individual government agencies decision to automate their process of updating their data.

[lieuzhenghong]

Thanks a lot @lennardl . Is there any process where I can follow up on this or do I just sit here and hope that they deign to provide us the updated dataset?

[lennardl]

@lieuzhenghong We have heard back from EMA and they are compiling the data. I will update this ticket when its updated, have followed up to ask when they would be ready.

[lieuzhenghong]

Hi @lennardl , have they gotten back to you guys yet? 1 month+ already leh....

[lennardl]

@liangyuanruo We brought this up with EMA last month while they acknowledged they would update it. I understand its not updated yet I'll write in again. Apologizes for this.

[lieuzhenghong]

Hi @lennardl , I see the issue has been closed, but the data hasn't been updated. Did EMA stonewall you?

[lennardl]

@lieuzhenghong Short answer yes

[lieuzhenghong]

Sorry to hear. Is there anything I can do about it?

[lennardl]

@liangyuanruo I've heard back from EMA they have the data and will update it by the end of the month

[lieuzhenghong]

Thanks for going to all this trouble! This whole ask-EMA-for-data-everytime-we-want-an-update approach just doesn't seem sustainable. There should be some sort of API on their end --- or at the very least a cronjob.

Who should I talk to to give feedback on this process? I'm just a regular guy but I can send a strongly-worded email to whoever is in charge. It looks like Mr Don Yeo is the "CHIEF TRANSFORMATION OFFICER" of the "DIGITAL TRANSFORMATION OFFICE" --- I could send him an email.

[lennardl]

@lieuzhenghong I do agree however not all agencies system are built for that purpose. In my current capacity, I can only advice that writing in should work as direct public feedback are taken seriously

[lieuzhenghong]

What’s your email address? I’ll BCC you

Sent from my iPhone

On 18 Nov 2020, at 9:46 AM, lennardl notifications@github.com wrote:

 @lieuzhenghong I do agree however not all agencies system are built for that purpose. In my current capacity, I can only advice that writing in should work as direct public feedback are taken seriously

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[lennardl]

You can use feedback@data.gov.sg as I access that account as well

[lieuzhenghong]

Data looks like it's been updated but you (or someone else) has broken the links: all the previous URIs give 404s.

[lieuzhenghong]

Also, some of the data that was previously available has now been removed: https://data.gov.sg/dataset/half-hourly-system-demand for instance. @lennardl

[lennardl]

@lieuzhenghong I know that EMA has just included a whole bunch of datasets and removed some old ones. Can you see if any fits your purpose here: https://data.gov.sg/search?organization=energy-market-authority