search

ratt-ru / tricolour

Holds an offline, MS direct version of the SDP online flagger.
Other
8 stars 4 forks source link

UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 452: ordinal not in range(128) #66

Closed o-smirnov closed 4 years ago

o-smirnov commented 4 years ago

Description

Startup log throws logging error. Seems it doesn't like the French Revolution.

What I Did

# # running /opt/code/dist/tricolourexe/tricolourexe --config /home/oms/input/mk_rfi_flagging_target_fields_firstpass.yaml --flagging-strategy polarisation --data-column DATA --windo
w-backend numpy --field-names J0538-4405,0408-65 /home/oms/msdir/1576687564_sdp_l0_cal.ms                                                                                             
# --- Logging error ---                                                                                       
# Traceback (most recent call last):                                                                          
#   File "logging/__init__.py", line 982, in emit                                                             
# UnicodeEncodeError: 'ascii' codec can't encode character '\xe9' in position 452: ordinal not in range(128)            
# Call stack:                                                                                                           
#   File "tricolour/apps/tricolour/tricolourexe.py", line 535, in <module>                                              
#   File "tricolour/apps/tricolour/tricolourexe.py", line 262, in main                                                  
#   File "tricolour/apps/tricolour/tricolourexe.py", line 268, in _main                                                 
#   File "logging/__init__.py", line 1279, in info                                                                      
#   File "logging/__init__.py", line 1415, in _log                                                                      
#   File "logging/__init__.py", line 1425, in handle                                                                    
#   File "logging/__init__.py", line 1487, in callHandlers                                                              
#   File "logging/__init__.py", line 855, in handle                                                                     
#   File "logging/__init__.py", line 1048, in emit                                                                      
#   File "logging/__init__.py", line 986, in emit                                                                       
# Message: '\n*******************************************************************************\n                  _______   _           _\n                 |__   __| (_)         | |\n
                    | |_ __ _  ___ ___ | | ___  _   _ _ __\n                    | | \'__| |/ __/ _ \\| |/ _ \\| | | | \'__|\n                    | | |  | | (_| (_) | | (_) | |_| | |\
n                    |_|_|  |_|\\___\\___/|_|\\___/ \\__,_|_|\n\nViva la r\xe9volution!\n\nSARAO END USER LICENSE AGREEMENT\n\nA DASK distributed RFI flagger by\nScience Data Process
ing and the\nRadio Astronomy Research Group\nCopyright 2019 South African Radio Astronomy Observatory (SARAO, SKA-SA).\nAll rights reserved\n\nThe above copyright notice and this per
mission notice shall be included\nin all copies or substantial portions of the Software.\n\nTHE SOFTWARE IS PROVIDED GRATIS, "AS IS", WITHOUT WARRANTY OF ANY KIND,\nEXPRESS OR IMPLIE
D, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF\nMERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABL
E\nFOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,\nTORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH\nTHE SOFTWARE OR THE USE OR OTHER DEALI
NGS IN THE SOFTWARE.\nNO PORTION OF THIS SOFTWARE MAY BE REVERSE ENGINEERED OR INCLUDED\nIN OTHER PRODUCTS WITHOUT PRIOR WRITTEN PERMISSION FROM SARAO.\nTHIS SOFTWARE, EITHER IN PORT
ION OR IN ITS ENTIRETY,\nMAY NOT BE SOLD OR OTHERWISE COMMERCIALLY EXPLOITED BY ANY THIRD PARTY\n\nBy using this software you, the end user, agree to these terms and conditions\n****
***************************************************************************\n'                                                                                                        
# Arguments: ()                                                                                                                                                                       
# tricolour - 2020-04-14 17:36:40,582 INFO -                                                                                         
# *******************************************************************************                                                                                                     
#                   _______   _           _                                                                             
#                  |__   __| (_)         | |                                                                            
#                     | |_ __ _  ___ ___ | | ___  _   _ _ __                                                            
#                     | | '__| |/ __/ _ \| |/ _ \| | | | '__|                                                           
#                     | | |  | | (_| (_) | | (_) | |_| | |                                                              
#                     |_|_|  |_|\___\___/|_|\___/ \__,_|_|                                                              
#                                                                                                                       
# Viva la r\xe9volution!
IanHeywood commented 4 years ago

Weirdly, I get this when running on CHPC but not on IDIA, using the same container (tricolour-1.2.5.simg from the Stimela docker hub). It doesn't halt proceedings on CHPC though, however it is interpreted as an error, and the subsequent output is written to the error file not the log by PBS.

sjperkins commented 4 years ago

Does this prevent the application running or does it merely pollute the log? I think the former implies the need for a release while the latter implies rolling the fix into a future release.

o-smirnov commented 4 years ago

It only pollutes the log.

IanHeywood commented 4 years ago

Or produces an error file instead of a log when run via PBS.

sjperkins commented 4 years ago

Or produces an error file instead of a log when run via PBS.

That's not great and would further motivate a new release.

sjperkins commented 4 years ago

Here's the tell:

#   File "tricolour/apps/tricolour/tricolourexe.py", line 535, in <module>                                              
#   File "tricolour/apps/tricolour/tricolourexe.py", line 262, in main                                                  
#   File "tricolour/apps/tricolour/tricolourexe.py", line 268, in _main

That's @bennahugo's obfuscated version (circa v0.1.6). There was a 0.1.7 release early January that addressed the unicode issues.

I suspect that you may simply need to update the tricolour in the stimela container.

Or simply pip install tricolour now!

sjperkins commented 4 years ago

This makes me realise that the application version needs to be logged on startup. We had similar misunderstandings with crystalball. Logged in #67

sjperkins commented 4 years ago

I'm closing this as it's been addressed in https://github.com/ska-sa/tricolour/pull/64, but do reopen the issue if I've made an error in this regard.