Belpex Data API

[JrtPec]

Let's try to get Belpex data in Open Grid. See https://transparency.entsoe.eu for graphs, need to research if there is an API or something available

[toonhooy]

I have looked for it one time, never found an API. They do have the option to get acces to all historical market data (hourly prices, trade-information, volumes, etc.) but it is a manual one (Yes, this means downloading excel sheets...). You have to fill in a form, afterwards you can acces the data.

https://www.belpex.be/services/market-data-services/

[JrtPec]

Can't be too hard to write a web scraper then. I mean, just look at this URL:

https://transparency.entsoe.eu/transmission-domain/r2/dayAheadPrices/show?name=&defaultValue=false&viewType=TABLE&areaType=BZN&atch=false&dateTime.dateTime=24.02.2016+00:00|CET|DAY&biddingZone.values=CTY|10YBE----------2!BZN|10YBE----------2&dateTime.timezone=CET_CEST&dateTime.timezone_input=CET+(UTC+1)+/+CEST+(UTC+2)

[JrtPec]

I'm just gonna post this here for record keeping purposes, not sure if it leads somewhere:

A data cell in the table is filled by this JavaScript call:

<span onclick="showDetail('eu.entsoe.emfip.transmission_domain.r2.presentation.entity.DayAheadPricesMongoEntity', '56caf701e4b094a8e4d4f9ee', '2016-02-23T23:00:00.000Z', 'PRICE', 'CET');">
23.99
</span>

So I opened up the console to find out where showDetail is defined:

function showDetail(a, b, c, d, e) {
    if(typeof mobileView !== "undefined" && mobileView){
        return;
    }
    if (arguments.length == 5) {
        showDetailByMongoEntity(a, b, c, d, e);
    } else {
        showDetailByModel(a, b, c, d);
    }
}

Digging deeper:

function showDetailByMongoEntity(className, id, timestamp, fieldName, timeZone) {
    var data = { className: className, id: id, timestamp: timestamp, fieldName: fieldName, timeZone: timeZone};
    showDataViewDetail(data, afterLoadDetailDialogContentProcess);
}

function showDataViewDetail(data, afterCallback) {

    var urlElements = ajaxUrl.match(/^(\/([^\/]*))/);
    var url;
    if (urlElements != null) {
        if (typeof specificDetailUrl === 'undefined') {
            url = urlElements[0] + "/getDetail";
        } else {
            url = urlElements[0] + specificDetailUrl;
        }
    }

    ajaxRequestResponse({
        contentType: "application/json;charset=UTF-8",
        method: "POST",
        async: false,
        url: url,
        data: JSON.stringify(data),
        success: function (detail) {
            $('<div id="dialog-content"></div>').appendTo('body').append(detail).triggerCallback(afterCallback()).initDialog({
                title: dataViewDetailName,
                modal: true,
                width: 850,
                topCloseButton: {text: "Close"},
                close: function () {
                    $('#dialog-content').remove()
                }
            });
        }
    })
}

So there is a simple HTTP call! It takes a class name, id, timestamp, field name and timezone. The only thing that isn't obvious about that is the id... So unless we can figure that out this is a dead end.

[JrtPec]

It took me literally 30 minutes building a web scraper using beautifulsoup. Belpex values for everyone in #107. Now let's find something nice to do with them :-)