Topology inheritance

[fedelemantuano]

Is it possible to use a base abstract topology and then use it to create new topologies like Python inheritance?

If yes, what is the best way?

Thanks

[Darkless012]

Can you state your use-case. I can't see why would you need that. On the other hand I believe you can reach your goal by using python modules (eg. imports)

[fedelemantuano]

I'd like to have a core topology, then use it with different output bolts (redis, elasticsearch, file system, mongodb, etc...) My use case is SpamScope. I tryed to use Python module in standard way but didn't work. You can see this tag. I used abstract bolt and then a real bolt.

[Darkless012]

I see no reason why this shouldn't work. (looking on topologies/spamscope_\<topology>.py)

Can you post the exception/error here?

[fedelemantuano]

Ok. I'll send you the error. Maybe my problem was here.

[Darkless012]

Oh I see... you can't reference the variable. Also similar issue here I guess: https://stackoverflow.com/questions/13404476/inherited-class-variable-modification-in-python

[fedelemantuano]

I'm trying do that but now I have this issue:

  File "/usr/local/bin/sparse", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/streamparse/cli/sparse.py", line 79, in main
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/streamparse/cli/submit.py", line 276, in main
    timeout=args.timeout)
  File "/usr/local/lib/python2.7/dist-packages/streamparse/cli/submit.py", line 143, in submit_topology
    topology_class = get_topology_from_file(topology_file)
  File "/usr/local/lib/python2.7/dist-packages/streamparse/util.py", line 439, in get_topology_from_file
    mod = importlib.import_module(mod_name)
  File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
    __import__(name)
  File "topologies/spamscope_redis.py", line 26, in <module>
    class OutputRedisTopology(Topology, AbstractTopology):
  File "/usr/local/lib/python2.7/dist-packages/streamparse/dsl/topology.py", line 36, in __new__
    TopologyType.clean_spec_inputs(spec, specs)
  File "/usr/local/lib/python2.7/dist-packages/streamparse/dsl/topology.py", line 114, in clean_spec_inputs
    stream_comp = specs[stream_id.componentId]
KeyError: 'phishing'

phishing is a key of my topology that it doesn't know because is in abstract module.