the L1 control files for Loxton site seems not working for PyfluxPro v3.4.11

[sunxm19]

When following the example data in wiki and running the L1 file (without any revision) for Loxton site, I encounted a few errors:

Accordingly, I revised the L1.txt as followes:

• changed "CSAT" to "CSAT3" for Diag_SONIC, Tv_SONIC_Av, Tv_SONIC_Sd, Ux_SONIC_Av, Ux_SONIC_Sd, Uy_SONIC_Av, Uy_SONIC_Sd, Uz_SONIC_Av, Uz_SONIC_Sd, Wd_SONIC_Av, Ws_SONIC_Av, UxUy.
• changed "CSAT3 and Li-7500" to "CSAT3, Li-7500" for Fco2_EP, Fco2_EP_QC, Fe_EP, Fe_EP_QC.

But for the remaining errors, I am not sure how to deal with it.

Thanks for your attention.

Xiangmin (Sam) from Illinois

[pisaac-ozflux]

Hi Sam,

Thanks for your email and apologies for the confusion. This is an old repository of example files and is no longer maintained. I will set it to private after this conversation so that it is no longer visible.

The current examples are in the PFP_examples repository at https://github.com/OzFlux/PFP_examples.git. You'll need to delete your existing Examples folder and clone the current PFP_examples repository from the URL above. If you make sure that your PyFluxPro and PFP_examples folders are at the same level e.g. C:\Users\Sam\PyFluxPro\ and C:\Users\Sam\PFP_examples\ then should not need to change the file paths and names in the example control files.

For the errors that you found, you did the right thing and had almost finished. The remaining 2 fixes required were:

1. Set the instrument attribute for Fh_EP and Fh_EP_QC to CSAT3. These variables are only associated with the sonic anemometer and do not involve measurements from the IRGA.
2. Set the instrument attribute for Sco2_profile to None. This is a "dummy" profile measurement the site PI has included that uses the change in CO2 concentration from the IRGA to approximate a true profile measurement.

Feel free to contact me directly by email if you need more help.

Cheers, Peter

[sunxm19]

Hi, Peter:

Thank you very much for your prompt reply with details.

I followed your instructions to revise this L1 files (old L1) and get PyfluxPro processed successfully.

However, when I run the L1 file in the new link you just shared (new L1), I got the following errors, as attached below:

Do you know any issues causing these issues?

Thanks for your attention. Xiangmin (Sam)

21:34:34 INFO Sonic type set to CSAT3 21:34:34 INFO IRGA type set to Li-7500 21:34:34 WARNING ????? 21:34:34 WARNING Global: recommended global attribute metadata_link not found 21:34:34 WARNING ????? 21:34:34 INFO Global: setting Conventions to CF-1.8 21:34:34 INFO Global: setting data_link to http://data.ozflux.org.au/ 21:34:34 INFO Global: setting featureType to timeSeries 21:34:34 INFO Global: setting license_name to CC BY 4.0 21:34:34 INFO Global: setting license to https://creativecommons.org/licenses/by/4.0/ 21:34:34 INFO Global: setting publisher_name to TERN Ecosystem Processes,OzFlux 21:34:34 INFO Global: setting ozflux_link to http://ozflux.org.au/ 21:34:34 INFO Starting L1 processing 21:34:34 INFO Parsing the L1 control file 21:34:34 INFO Reading Excel workbook Loxton_L1.xls 21:34:34 ERROR Error running L1, see below for details ... 21:34:34 ERROR Traceback (most recent call last): File "C:\eddy_covariance\PyFluxPro\scripts\pfp_top_level.py", line 405, in do_run_l1 ds1 = pfp_levels.l1qc(cfg) File "C:\eddy_covariance\PyFluxPro\scripts\pfp_levels.py", line 35, in l1qc dfs = pfp_io.ReadInputFile(l1_info) File "C:\eddy_covariance\PyFluxPro\scripts\pfp_io.py", line 774, in ReadInputFile data = ReadExcelWorkbook(l1_info) File "C:\eddy_covariance\PyFluxPro\scripts\pfp_io.py", line 642, in ReadExcelWorkbook xl_file = pandas.ExcelFile(file_name, engine=engine) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\pandas\io\excel_base.py", line 1419, in init self._reader = self._engines[engine](self._io, storage_options=storage_options) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\pandas\io\excel_xlrd.py", line 25, in init super().init(filepath_or_buffer, storage_options=storage_options) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\pandas\io\excel_base.py", line 518, in init self.book = self.load_workbook(self.handles.handle) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\pandas\io\excel_xlrd.py", line 38, in load_workbook return open_workbook(file_contents=data) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\xlrd__init__.py", line 172, in open_workbook bk = open_workbook_xls( File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\xlrd\book.py", line 79, in open_workbook_xls biff_version = bk.getbof(XL_WORKBOOK_GLOBALS) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\xlrd\book.py", line 1284, in getbof bof_error('Expected BOF record; found %r' % self.mem[savpos:savpos+8]) File "C:\Users\User.conda\envs\pfp_env\lib\site-packages\xlrd\book.py", line 1278, in bof_error raise XLRDError('Unsupported format, or corrupt file: ' + msg) xlrd.biffh.XLRDError: Unsupported format, or corrupt file: Expected BOF record; found b'version '

[pisaac-ozflux]

Hi Sam,

Feel free to email me directly at pisaacozfluxgmailcom, it's easier to do this via email than through GitHub issues.

The error message suggests that the Excel workbook you are trying to read with PyFluxPro is corrupted in some way.

Have you swapped from your old Examples folder to the current PFP_examples repository? If so, then that should work correctly, it was tested yesterday and I will test it again after this email.

I would try 3 things:

1. Try opening the Excel workbook specified in the L1 file in Excel to see if it is corrupted.
2. Try cloning the PFP_examples repository again in case something was corrupted in the cloning process.
3. If all else fails, we can do a Zoom session so I can see exactly what is going on.

Cheers, Peter

[sunxm19]

Thanks Peter for your prompt reply with thoughtful details. Previously, I directly downloaded the data (code-->download ZIP), which caused the issue of corrupted L1 file. While coloning with Github Desktop, everything works good. I should check more careful before reaching out to you. Thanks again for your help. (I will contact you via email in future inquiry).