response status code 400 -- No RESULT OBTAINED --

Hi, I tried to use entsoe-py, I can make valid base_requests, but when I try to get a query_price I get the response:

response status code 400 -- No RESULT OBTAINED --

So this is possible a feature request, what about adding a logger to entsoe-py to see what is going on and to be able to to see what went wrong? I am pretty sure I invoked the query_price(...) method with the correct parameters. Also some examples would be nice to see how the module actually works. I had some problems to get the imports working in my virtual environment.... Otherwise nice work and I hope I get it going...

Could you please tell me what exact request you are making, with what parameters? I'll check if I can replicate the problem, and where the problems lie. (I have a script running that checks prices every day, haven't had any problems since last summer).
It's an open source project for a reason, so feel free to add examples, more explanations or logging functionality. I'll try and add some examples, if I find the time...

I have forked the project and have started to add some unit tests (using node2) and I have started to add sphinx documentation, I am not yet ready to send you a particular error for the price request, but the unit test for basic_request works reliable. I try to add more unit tests and see what fails (or if it just was my problem).

fork is at https://github.com/bfauser/entsoe-py

Great! Please feel free to create a pull request once you have something ready that can be included in the package.

Dear JrtPec

sphinx is (almost) working, so I can create documentation as html pages (some problems still with the setup.py file, which does not like to work nicely with sphinx. I have added documentation in doc/config.py and some new *.rts files in the githuib fork, but not the whole (large) documentation. There is also a bit information how to run nose2 tests, which is not (yet) automated in the setup.py file. I will not commit this fork to any package site, I feel this is your package and you should decide what to use and commit (and I would not have the submit-rights to so so anyhow?)

While the basic_request for Germany 'DE' works well, I would like to make a query_price request, which in turn calls internally the basic_request. I have a few issues with that. I would like to get the Germany-Austria and the Switzerland Day-Ahead hourly prices (and later also other stuff, like different production series including forecasts) However the request result is just None. I tried to figure out what basic_request is compiled by the price request, and called that directly, but alas, that failed to (request = None, not even a status code?) If you could let me know what kind of request works for you, I can check if I can try that request.

Reading the API doc was not entierly helpful, as example request from the Entso-e web site (API doc) give me a result, but that says my request is invalid :( "documentType cannot be used in ..." type errors.

I will see if I find time (and am able to) add logging capabilities, which might help in finding errors I make with the package.

Is it sane to have constants inside the main level of the package or should they be moved into a config or resource object? I find it a bit poluting to have things like URL floating around inside the entsoe name space (if imported as > import entsoe that fills the main namespace...)

Indeed if I can manage to something which I find worth doing, I'll let you know via a pull request.

I just pushed a new version (0.1.13) that includes some bugfixes:

HTTP Errors that contained the text "No data found", used to return None. This was indeed confusing, so now the error will be raised. (although there is not much info in the errors)
I was able to request day-ahead prices for AT and CH, but indeed not for DE. DE was translated to domain code 10Y1001A1001A83F (Germany), which apparently doesn't work for prices. (I also noticed that LU didn't work). I have worked around it by mapping country codes 'DE' and 'LU' to domain code 10Y1001A1001A63L (DE-AT-LU BZL). I hope this is correct? (Do Germany and Luxemburg have the same prices? Why don't I have problems with country code 'AT'?) You can find these codes in the API doc under section A.10

I don't really get your last remark. __init__.py in the Entsoe module only imports the Entsoe-class, so doing import entsoe shouldn't be importing URL. You would need to do from entsoe.entsoe import URL for that. That being said, there is indeed a point to be made about putting all the different mappings and other globals in a separate module...

Dear JrtPec,

thank you very much for looking into this. I have found a similar thing with the codes. In the section A.10 of the API documentation all codes are summarized in one list. However the codes (some but not all, alas) have some additional marker, e.g. BZ — Bidding Zone BZA — Bidding Zone Aggregation CA — Control Area MBA — Market Balance Area It seems that prices of auctions are only available in a bidding zone, which makes sense. I found the DE-AT-LU code too. My experience with EPEX data is, that DE-AT are in the same auction, hence share a price, I was not aware that LU is in the same lot. I am on the way to sort these codes into the 4 types, and create 4 dictionaries with these data. Hence a BIDDING_ZONE, etc dictionary.... and then use the appropriate domain tag in the queries.

Downside is, that e.g. Norway and Italy have many BiddingZones, hence a single CountrCode, will not do, and one is forced to provide the exact key [or an abreviation which is meaningful, but alas arbitrary] to get the BiddingZone right. I am inclined to include also a check if the Zone provided is valid (in the right dictionary) before even handing out a html request.

I also see that you have used hard codes time zones. However you already load pytz module, which is aware of all of the Olsen Time zones (at least), so it might be better to see if the time zones could be used directly from there. All results and request times are UCT, which is the correct way to handle this, but some users might want to set local time intervals. I am not sure I want this, hence I will not implement it. However the pandas time stamps are time zone aware, and have an ugly +00:01 etc like form, which made it hard to compare them with naive times stamps in my jupter example. It may be useful to have a timestamp_naive=True|False flag in the converters?

I will see if I can add a json export function, as I might need json data in some of my API backends.

I have added in my fork an example folder where I posted a jupyter notebook which shows how to get prices and generation data (you need to past your API key in before it works). Basic logging functionality is added (works in nose2 testing but not checked if it works when the package is properly imported as >pip install entsoe-py package), and I am on my way to run some more cross validation checks. I plan to see which codes will actually allow which types of requests to check if my allocation of the Area codes into the 4 dictionaries was correct. However I need only data from DE and CH (at the moment) so my testing will possibly not be complete. But I plan to automate it and see how fare I come.

I am a bit confused about the generation query, which yields for different countries different types of generated stuff (when psrType=None is provided. When I provide a psrType I get no response. CH has no Oil, Hard Coal etc, and I wonder why, but have to check with the Entso-e documentation why that is so. FR, BE etc seem to do much better. That CH has no Wind Offshore seems to be clear ;)

I am not a software developer, so my python skills are not that good. So I had a hard time get nose2, sphinx and other stuff right. I have the problem that in my jupyter notebook I have to import

from .entsoe.entsoe (relative) which looks not so good. I guess the first entsoe goes away when I install the package via > pip install -e . but have to check that. The imports are done relative to the locaton of teh jupyter notebook inside the example/ folder of the package (see my fork).

So the init.py question: I have quite a few APIs in a Flask app, so many things are called app, or SECRET_KEY etc and I would like to keep these things apart. For that reason I will see how e.g. Flask has a config object (accessible as app.config.property) where it stores all of its config data. (and similarly does SQLAlchemy) I would like to move the dicrionaries, URLs, etc into such a structure. OR at leas move them as class methods into the Entsoe class inside entsoe.py. If you are willing to give some advice what you think of this idea we can make sure that your and mine sorce base do not diverge (too much).,

Hence two questions remain (at least): 1) do you agree I split the codes into the 4 mentioned types of codes (some are overlapping, valid in 2,3, or even all four cases), but the queries could easily type check if a domain is applicable by looking if the key is in the appropriate dictionary. 2) I would like to move the constants somewhere, either into a third file constants.py, or (mab be better, not a further import) into the Entsoe class but as a class property of a config (dictionary) structure, so all instances share these data (usually only one instance will be in use I guess, but its cleaner that way)

I you agree I can try to do this and send you a pull request when its done. I will try to make changes in such a way that existing code using your entsoe-py package will be least affected.

One final remark. I found time series having a 'PT15M' (15minutes) period (frequency). I tried to add that in the parser, but the pandas series stuff was not willing to accept this. There is a list of all (currently used) periods on Entso-e, which makes it clear that 1h periods can be encoded in two ways 'PT60M' and 'PT1H', I think it would be better to have a dictionary with all these codes and a translation into pandas series frequencies (if possible), this would replace the if then else clause in the parser.py file. And would make later changes very easiy, if new frequencies pop up.

Ah! A after-last-question-remark ;) I noticed that Entso-e also gives the currency of a price request, and that may not have a tag but also a tag (or such), anyhow I have not found a series which does so till now. I would at least like to see the unit of the quantity in a time series, e.g. 'MWH' for MegaWattHours, (our system uses SI units so I have to change to Watts) 'EUR (per MWH)' for currency (CH has EUR prices amazingly, while some other countries have their own currency) I would like to pass that information on, but they do not fit into the padas data frame, (unless I add a new column for each time series with units, [bad ides lots of wasted data fields] ... any idea how to pass that on? I will do this in the json response, but pandas is nice and cool for some quick and dirty hacking in jupyter notebooks to show how to use the entsoe-py API wrapper.

Last remark, if you want to contact me directly (not over github) my email is <<Bertfried[dot]Fauser[at] gmail[dot}com>>.

Cheers and a nice evening ;)

Hi Bertfried,

I definitely agree with splitting the domains up in 4 different dictionaries. They could be in a separate module named domains.py or something.
In countries with multiple bidding zones, only the country code will not do. Nothing we can do but to ask the user to be specific.

About the timezones, I tried this:

import pytz
from entsoe.entsoe import TIMEZONE_MAPPINGS
for code in TIMEZONE_MAPPINGS:
print(pytz.country_timezones(code.split('-')[0])

You see that this approach could work for all country codes, except Russia...

I don't agree with returning timezone-naive data. It adds ambiguity where there is none. We could however add a flag localized=False or something, so you can get data in UTC.
You can export any Pandas DataFrame to Json
I haven't added a dictionary to convert psrTypes yet, so right now you have to use the code as input. Codes are in the documentation in section A5.
You can only do the import entsoe line if the module is in your PYTHONPATH (Try import sys; print(sys.path). You can do this by either installing it via pip (not a good idea if you are developing it); launching your Jupyter Notebook in the folder right above the module; or add the path to the module into your pythonpath
You have to make a distinction between configuration variables, and global variables. The URL and others are globals, but they don't really need to be configurable by a user. This means you can just add them in a separate python file and import them from there if you want.
You can send me a pull request whenever you have finished anything. Just a tip/request: if you are working on different topics, please create separate branches and pull requests to split them up. This way everything doesn't end up in one big chunk, which is harder to review.
I have already added the PT15M compatibility to version 0.1.13. I agree to have it changed into a dictionary.
You can assign variables to a Pandas DataFrame like this (as long as the name is not reserved):
```
import pandas as pd
df = pd.DataFrame()
df.unit = 'kWh'
df.currency = 'CHF'
```
These attributes will only stick to the DataFrame as long as no operations are performed on it. I don't think this is a recommended way, but it does the trick and I've not yet seen anything better.

Cheers!

EnergieID / entsoe-py

response status code 400 -- No RESULT OBTAINED -- #2