Importing a file with just comments crashes the parser

[GoogleCodeExporter]

What steps will reproduce the problem?
1. File / New
2. File / Import
3. attached file, which happens to have only comments in it. See attached 
\trunk\src\__init__.py

What is the expected output? 

nothing, as there are no classes.

What do you see instead?

crash related to token parsing.

Original issue reported on code.google.com by abu...@gmail.com on 19 Jul 2012 at 2:16

Attachments:

• init.py

[GoogleCodeExporter]

Importing...
[u'E:\\Devel_Andy\\pyNsource\\trunk\\src\\__init__.py']
Traceback (most recent call last):
  File "E:\Devel_Andy\pyNsource\trunk\src\pyNsourceGui.py", line 476, in FileImport
    self.observers.CMD_FILE_IMPORT_SOURCE(filenames, None)
  File "E:\Devel_Andy\pyNsource\trunk\src\architecture_support.py", line 20, in __call__
    for alias, obj in self.items() if callable(obj) ] )
  File "E:\Devel_Andy\pyNsource\trunk\src\app\controller.py", line 49, in CMD_FILE_IMPORT_SOURCE
    self.run(c)
  File "E:\Devel_Andy\pyNsource\trunk\src\app\controller.py", line 13, in run
    self.app.cmd_mgr.run(cmd)
  File "E:\Devel_Andy\pyNsource\trunk\src\command_pattern.py", line 101, in run
    item.execute()
  File "E:\Devel_Andy\pyNsource\trunk\src\app\cmds\files.py", line 22, in execute
    p.Parse(f)
  File "E:\Devel_Andy\pyNsource\trunk\src\core_parser.py", line 508, in Parse
    HandleComposites.Parse(self, file)
  File "E:\Devel_Andy\pyNsource\trunk\src\core_parser.py", line 29, in Parse
    self._ParseLoop()
  File "E:\Devel_Andy\pyNsource\trunk\src\core_parser.py", line 47, in _ParseLoop
    assert token, ("Not expecting blank token, once have detected in & out dents. tokentype=%d, token=%s" %(tokentype, token))
AssertionError: Not expecting blank token, once have detected in & out dents. 
tokentype=54, token=

Original comment by abu...@gmail.com on 19 Jul 2012 at 2:17

[GoogleCodeExporter]

Fixed by switching in the new ast based parser.  See 
http://code.google.com/p/pynsource/source/browse/trunk/src/app/cmds/filemgmt.py 

#pmodel, debuginfo = old_parser(f)
pmodel, debuginfo = new_parser(f)

Original comment by abu...@gmail.com on 19 Aug 2012 at 11:30

• Changed state: Verified