gems-uff / noworkflow

Supporting infrastructure to run scientific experiments without a scientific workflow management system.
http://gems-uff.github.io/noworkflow
MIT License
120 stars 28 forks source link

Capture Definition Provenance from other files and class definitions #74

Open JoaoFelipe opened 8 years ago

JoaoFelipe commented 8 years ago

Add an option to collect definition provenance from other modules other than the main script.

Currently, we capture only the content hash of the main script as definition provenance. The other local modules are captured as deployment provenance. So, we do not capture definition provenance from them that are useful for execution provenance (such as global variables names)

We could expand FunctionDef table to not only store function definitions, but also classes and files, since they have the same attributes: trial_id, id, name, code_hash. We could also add an extra attribute: parent. This way, the following code would generate 4 definition objects

# some_file.py
class SomeClass(object):
    def some_method(self):
        def some_function():
            pass

An extra attribute, qualname, could be provided for qualified name reference, according to https://www.python.org/dev/peps/pep-3155/

Definition objects:

Definition(id=1, name="some_file.py", parent=None, qualname="")
Definition(id=2, name="SomeClass", parent=1, qualname="SomeClass")
Definition(id=3, name="some_method", parent=2, qualname="SomeClass.some_method")
Definition(id=4, name="some_function", parent=3, qualname="SomeClass.some_method.<local>.some_function")

Note that the qualname is not fully qualified, so we don't have the filename on it.

JoaoFelipe commented 8 years ago

Related to #66