BDF reader should accept non-ascii chars in comments

[GoogleCodeExporter]

What is the new feature/capability that you'd like?

BDF reader should accept non-ascii characters in comment cards and string 
fields like title and subtitle.

Why is it useful?

Some decks may contain non-ascii characters (e.g., degree symbol). Presently 
these characters need to be removed and replaced with ascii placeholders. 
Otherwise, these characters cause an error in the BDF.read_bdf method, e.g., 

Traceback (most recent call last):
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\gui\gui_qt.py", line 1002, in on_load_geometry
    has_results = load_function(infile_name, self.last_dir)
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\converters\nastran\nastranIO.py", line 99, in load_nastran_geo
metry
    model.read_bdf(bdf_filename, include_dir=dirname, punch=punch, xref=True)
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\bdf\bdf.py", line 748, in read_bdf
    self._read_bulk_data_deck()
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\bdf\bdf.py", line 1182, in _read_bulk_data_deck
    (lines, comment) = self._card_streams[self._ifile].next()
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\bdf\bdf.py", line 1067, in _stream_card
    (i, line, comment) = self._get_line()
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\bdf\bdf.py", line 1239, in _get_line
    return self._line_streams[self._ifile].next()
  File "C:\Anaconda\lib\site-packages\pynastran-0.7.0_dev-py2.7.egg\pyNastran\bdf\bdf.py", line 1253, in _stream_line
    line = line.rstrip('\t\r\n ')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x94 in position 29: 
ordinal not in range(128)

List a sample API for the new functionality:
example:  nodeObject = mesh.getNode(nodeID)

 * use unicode for comments and user-defined string varibales
 * maybe use try/except/finally to parse these lines in the deck?

Any other requirements on the functionality?

Original issue reported on code.google.com by jeffrey....@gmail.com on 29 Oct 2014 at 1:02

[GoogleCodeExporter]

Unicode is definitely a problem.  While it should be supported, it will create 
a lot of headaches.  Since there are so many encodings, the burden will have to 
be passed to the user as an encoding option, but reading utf-8 or other 
encodings is certainly possible.  Unless I'm wrong about how Nastran in France 
is used, it shouldn't affect the code beyond the comments (e.g. PBARL still 
uses ASCII strings for shape names).

The io module has an open method that is used in Python 3.  I've created two 
sample problems with a simple test script to familiarize myself with unicode.  
While it works just fine on a list of Spanish characters, it has a lot of 
trouble with Greek letters.  When a given encoding fails, it tries all of the 
encodings.  What's strange is none of these encodings decode the characters 
properly.  I'm sure some Greek letters work, but the ones I grabbed from 
Wikipedia did not.  However, they looked fine in Notepad++ as UTF-8 and it's 
possible that another representation of the letter would be fine.  (e.g. there 
are multiple unicode glyphs that look like A).

If you can figure out what's wrong with the printing of the Greek letters, that 
would help a lot.  Otherwise, the strangely buggy approach that doesn't support 
Greek letters will be used.  Also, do you have a sample BDF and a sample script?

Original comment by mesheb82 on 29 Oct 2014 at 6:14

Attachments:

• file.bdf
• file2.bdf
• unicode_test.py

[GoogleCodeExporter]

I'll check out the tests you put together. Thanks!

Original comment by jeffrey....@gmail.com on 4 Nov 2014 at 2:57

[GoogleCodeExporter]

moved to github

Original comment by mesheb82 on 14 Mar 2015 at 8:13

[GoogleCodeExporter]

Original comment by mesheb82 on 8 Apr 2015 at 6:34

• Changed state: MovedToGithub