AttributeError: 'ShellSpoutSpec' object has no attribute 'componentId'

[marijnvriens]

Intermittently we get a traceback when running $ sparse submit --name ruleengine The submit will fail with the following traceback.

Traceback (most recent call last):\n
  File \"/opt/cysoc/bin/sparse\", line 8, in <module>\n
    sys.exit(main())\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/cli/sparse.py\", line 87, in main\n
    args.func(args)\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/cli/submit.py\", line 395, in main\n
    active=args.active,\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/cli/submit.py\", line 206, in submit_topology\n
    topology_class = get_topology_from_file(topology_file)\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/util.py\", line 573, in get_topology_from_file\n
    mod = importlib.import_module(mod_name)\n
  File \"/opt/cysoc/lib/python3.5/importlib/__init__.py\", line 126, in import_module\n
    return _bootstrap._gcd_import(name[level:], package, level)\n
  File \"<frozen importlib._bootstrap>\", line 986, in _gcd_import\n
  File \"<frozen importlib._bootstrap>\", line 969, in _find_and_load\n
  File \"<frozen importlib._bootstrap>\", line 958, in _find_and_load_unlocked\n
  File \"<frozen importlib._bootstrap>\", line 673, in _load_unlocked\n
  File \"<frozen importlib._bootstrap_external>\", line 673, in exec_module\n
  File \"<frozen importlib._bootstrap>\", line 222, in _call_with_frames_removed\n
  File \"topologies/ruleengine.py\", line 25, in <module>\n
    class RuleEngine(Topology):\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/dsl/topology.py\", line 38, in __new__\n
    TopologyType.clean_spec_inputs(spec, specs)\n
  File \"/opt/cysoc/lib/python3.5/site-packages/streamparse/dsl/topology.py\", line 114, in clean_spec_inputs\n
    if isinstance(stream_id.componentId, ComponentSpec):\n
AttributeError: 'ShellSpoutSpec' object has no attribute 'componentId'

Diving into the streamparse code a bit, this is remarkable because the line above (113) contains for stream_id, grouping in list(iteritems(spec.inputs)): The error is for a ShellSpoutSpec, yet the code iterates over the spec.input , something that spouts should not have.

This is streamparse 3.16.0 .

Any hints are very appreciated.