launch app with command python read.py.old or python gui.py in terminal (used powershell in vscode)
select folder containing csv
enter 1 single DUID to test, for instance ARWF1
click run
results returned and printed correctly in terminal
app crashed
exceptions log
.\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old:20: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support skipfooter; you can avoid this warning by specifying engine='python'.
df = pd.read_csv(PathAndNameFile, skiprows=1, skipfooter=1, on_bad_lines='skip')#This will fetch the BIDDAYOFFER_D
I BID BIDDAYOFFER_D 2 SETTLEMENTDATE DUID BIDTYPE BIDSETTLEMENTDATE ... T2 T3 T4 NORMALSTATUS LASTCHANGED MR_FACTOR ENTRYTYPE Unnamed: 32
0 D BID BIDDAYOFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 NaN 31/05/2021 11:39 NaN DAILY NaN
1198 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 31/05/2021 11:39 241 2/06/2021 4:05 NaN
2386 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 31/05/2021 11:39 241 2/06/2021 4:10 NaN
3574 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 31/05/2021 11:39 241 2/06/2021 4:15 NaN
4762 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 31/05/2021 11:39 241 2/06/2021 4:20 NaN
... .. ... ... .. ... ... ... ... ... .. .. .. ... ... ... ... ...
337402 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 2/06/2021 17:37 241 3/06/2021 3:40 NaN
338590 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 2/06/2021 17:37 241 3/06/2021 3:45 NaN
339778 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 2/06/2021 17:37 241 3/06/2021 3:50 NaN
340966 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 2/06/2021 17:37 241 3/06/2021 3:55 NaN
342154 D BID BIDPEROFFER_D 2 2/06/2021 0:00 ARWF1 ENERGY 2/06/2021 0:00 ... 0 0 0 0 2/06/2021 17:37 241 3/06/2021 4:00 NaN
[289 rows x 33 columns]
.\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old:24: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support skipfooter; you can avoid this warning by specifying engine='python'.
df2 = pd.read_csv(PathAndNameFile, skiprows=len(df. index)+2, skipfooter=1, on_bad_lines='skip') #This will fetch the BIDPEROFFER_D
Traceback (most recent call last):
File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc
return self._engine.get_loc(casted_key)
File "pandas_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc
File "pandas_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'DUID'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File ".\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old", line 98, in
loadCSV(f,DUIDsets,folder)
File ".\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old", line 26, in loadCSV
print(df2.loc[df2['DUID'] == targetDUID])
File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\frame.py", line 3505, in getitem
indexer = self.columns.get_loc(key)
File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc
raise KeyError(key) from err
KeyError: 'DUID'
Tested on:
read.py.old(original source before segmentation)gui.pyandapp.py(after code segmentation)Steps to reproduce:
python read.py.oldorpython gui.pyin terminal (used powershell in vscode)ARWF1[289 rows x 33 columns] .\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old:24: ParserWarning: Falling back to the 'python' engine because the 'c' engine does not support skipfooter; you can avoid this warning by specifying engine='python'. df2 = pd.read_csv(PathAndNameFile, skiprows=len(df. index)+2, skipfooter=1, on_bad_lines='skip') #This will fetch the BIDPEROFFER_D Traceback (most recent call last): File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\indexes\base.py", line 3621, in get_loc return self._engine.get_loc(casted_key) File "pandas_libs\index.pyx", line 136, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\index.pyx", line 163, in pandas._libs.index.IndexEngine.get_loc File "pandas_libs\hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item File "pandas_libs\hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item KeyError: 'DUID'
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File ".\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old", line 98, in
loadCSV(f,DUIDsets,folder) File ".\Wil-Project-AEMO-CSV-Efficient-Reading-Sorting-and-Export-\read.py.old", line 26, in loadCSV print(df2.loc[df2['DUID'] == targetDUID]) File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\frame.py", line 3505, in getitem indexer = self.columns.get_loc(key) File "%localappdata%\Programs\Python\Python310\lib\site-packages\pandas\core\indexes\base.py", line 3623, in get_loc raise KeyError(key) from err KeyError: 'DUID'