search

acruzgarcia / OMIEData

Python package to import data from OMIE (Iberian Peninsula's Electricity Market Operator): https://www.omie.es/
https://pypi.org/project/OMIEData/
MIT License
24 stars 7 forks source link

'DataFrame' object has no attribute 'concat' #8

Open FilipeDoria opened 5 months ago

FilipeDoria commented 5 months ago

Hey! I just tried using the package, unsuccessfully. Maybe something changed within OMIE endpoints? I tried the example code you suggested:

`import datetime as dt import matplotlib.pyplot as plt

from OMIEData.DataImport.omie_marginalprice_importer import OMIEMarginalPriceFileImporter from OMIEData.Enums.all_enums import DataTypeInMarginalPriceFile

dateIni = dt.datetime(2024, 1, 1) dateEnd = dt.datetime(2024, 3, 22)

df = OMIEMarginalPriceFileImporter(date_ini=dateIni, date_end=dateEnd).read_to_dataframe(verbose=True) df.sort_values(by='DATE', axis=0, inplace=True) print(df) `

And got this error: _'DataFrame' object has no attribute 'concat'https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_15_01_2024_15_01_2024.TXT Requesting https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_16_01_2024_16_01_2024.TXT ... There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2024/MES_01/TXT/INT_PBC_EV_H_1_16_01_2024_16_01_2024.TXT_

Can you have a look at it?

acruzgarcia commented 2 months ago

Hi, sorry i didnt ser this issue until now!

Please update the pandas package. The version has to be > 2.0.1

javier-alvarez-ctic commented 2 months ago

Hi, I have the Pandas Version 2.2.2 installed and I have the same issue.

I tried to downgrade the version and use version 2.0.1 or 2.0.2 (thats require downgrade Numpy too) but I have the same issue (Im using Python 3.11.3)

This package still working? I need to download data from OMIE (energy prices from Spain)

Thanks!!

(.venv) PS XXXXXXXXX> python .\Omie.py 2.0.2 Requesting https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_01_01_2020_01_01_2020.TXT ... There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_01_01_2020_01_01_2020.TXT 'DataFrame' object has no attribute 'concat'https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_01_01_2020_01_01_2020.TXT Requesting https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_02_01_2020_02_01_2020.TXT ... There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_02_01_2020_02_01_2020.TXT 'DataFrame' object has no attribute 'concat'https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_02_01_2020_02_01_2020.TXT Empty DataFrame Columns: [DATE, CONCEPT, H1, H2, H3, H4, H5, H6, H7, H8, H9, H10, H11, H12, H13, H14, H15, H16, H17, H18, H19, H20, H21, H22, H23, H24] Index: []

javier-alvarez-ctic commented 2 months ago

Hi again!

You only need to change the OMIEData/DataImport/omie_data_importer_from_responses.py code from: df = df.concat([df, self.fileReader.get_data_from_response(response=response)], ignore_index=True) To this: df = pd.concat([df, self.fileReader.get_data_from_response(response=response)],ignore_index=True)

I have created a new Pull request, please accept it if is ok with you.

Thanks!

acruzgarcia commented 2 months ago

Hi, this is a recurrent issue that has been replied before. You need to update pandas package to be a version higher or equal to 2.0.1.

Please pip install pandas. Thanks

javier-alvarez-ctic commented 2 months ago

Hi. As I said in my first message, I do actually have Pandas installed in my environment...

In fact, I have a version higher than 2.0.1, exactly version 2.2.2. I have tried to downgrade the version to 2.0.1 and 2.0.2 (the same and a little higher) and the problem persists.

I made the change I said in my second message and everything works now, please keep this in mind.

Thank you.

acruzgarcia commented 2 months ago

Yes, thanks for this. Your fix looks like the correct one. The current code must have been corrupted in the last update. I will fix when I have some spare time.

Sorry for the initial confusion. I did not correctly read your message.

Cheers, Alberto.

acruzgarcia commented 2 months ago

In fact I have just seen that you did a pull request so I will review ASAP. Thanks!

javier-alvarez-ctic commented 1 month ago

I hope this message finds you well. I wanted to ask if there have been any updates or progress on the issue. I understand that the problem has been identified, but I noticed that the changes have not been pushed yet, nor has a new version of the package been released.

I am trying to use the package in a Docker container that automatically downloads dependencies, and these changes are crucial to the configuration.

Any information on next steps would be greatly appreciated.

Thanks for your time and help.

javier-alvarez-ctic commented 1 month ago

Hi! Any update?

moemmia commented 2 weeks ago

Hey! Any update on this topic? Same issue here 🙋🏻‍♂️

marcforn commented 2 weeks ago

Hi, I'm having the same issue. Any update on merging the PR?

acruzgarcia commented 2 weeks ago

The current codebase in OMIEData/DataImport/omie_data_importer_from_responses.py is already as you Javier suggested, so

df = pd.concat([df, self.fileReader.get_data_from_response(response=response)],ignore_index=True)

that is, (with pd. instead of df.).

I think the problem that people is having is because ther are using a non-updated version of OMIEData. The one in pyOPI.org. I will update the version to use the current codebase.

javier-alvarez-ctic commented 2 weeks ago

Yes, you are right. The code in the downloaded package is wrong but the code in the repository has fixed the bug. I'm sorry I didn't check before making the pull request.

acruzgarcia commented 2 weeks ago

Hi, I have already uploaded the fixed package in

https://pypi.org/project/OMIEData/

The issue should be fixed... at last. Could anyone of you please confirm so I can close the issue? Thanks!

marcforn commented 2 weeks ago

I've updated to latest version and when running the example code:

image

Now I get different error:

There was error processing file: https://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_12_01_2020_12_01_2020.TXT
unsupported locale settinghttps://www.omie.es/sites/default/files/dados/AGNO_2020/MES_01/TXT/INT_PBC_EV_H_1_12_01_2020_12_01_2020.TXT

Pip show omiedata: image

acruzgarcia commented 2 weeks ago

Hi,

The "unsupported local settings" issues were solved in the past. Please check the closed issues and let me know if that helps.

marcforn commented 1 week ago

I've tried to apply the changes and does not work on a mac

FilipeDoria commented 5 days ago

I just made a clean install and keep having the same issue. image

acruzgarcia commented 5 days ago

Hi, I have just executed the same code as you and it works. Can you please verify that you have OMIEData 0.3.0.0 installed.

From the same session you are, you can just type the following:

import pip pip.main(['list'])

you should get an output like the following, with the OMIEData version you are using:

import pip pip.main(['list']) WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip. Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue. To avoid this problem you can invoke Python with '-m pip' instead of running pip directly. Package Version

certifi 2024.8.30 charset-normalizer 3.4.0 contourpy 1.3.1 cycler 0.12.1 DateTime 5.5 fonttools 4.55.0 idna 3.10 kiwisolver 1.4.7 matplotlib 3.9.2 numpy 2.1.3 OMIEData 0.3.0.0 packaging 24.2 pandas 2.2.3 pillow 11.0.0 pip 22.3.1 pyparsing 3.2.0 python-dateutil 2.9.0.post0 pytz 2024.2 requests 2.32.3 setuptools 65.5.1 six 1.16.0 tzdata 2024.2 urllib3 2.2.3 wheel 0.38.4 zope.interface 7.1.1

If you use 0.3.0.0 it will work. Please let me know if this helps.