Netflix / metaflow

:rocket: Build and manage real-life ML, AI, and data science projects with ease!
https://metaflow.org
Apache License 2.0
8.01k stars 751 forks source link

Cannot initialize FlowSpec? #126

Closed mxlei01 closed 4 years ago

mxlei01 commented 4 years ago

Tried this in a jupyter notebook:

from metaflow import FlowSpec, step

class TravelFlow(FlowSpec):
    pass

if __name__ == '__main__':
    TravelFlow()

And it gives me an error:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-e93218f57f8d> in <module>
      1 if __name__ == '__main__':
----> 2     TravelFlow()

~/.pyenv/versions/3.7.5/envs/375/lib/python3.7/site-packages/metaflow/flowspec.py in __init__(self, use_cli)
     68         self._cached_input = {}
     69 
---> 70         self._graph = FlowGraph(self.__class__)
     71         self._steps = [getattr(self, node.name) for node in self._graph]
     72 

~/.pyenv/versions/3.7.5/envs/375/lib/python3.7/site-packages/metaflow/graph.py in __init__(self, flow)
    130     def __init__(self, flow):
    131         self.name = flow.__name__
--> 132         self.nodes = self._create_nodes(flow)
    133         self.doc = deindent_docstring(flow.__doc__)
    134         self._traverse_graph()

~/.pyenv/versions/3.7.5/envs/375/lib/python3.7/site-packages/metaflow/graph.py in _create_nodes(self, flow)
    137     def _create_nodes(self, flow):
    138         module = __import__(flow.__module__)
--> 139         tree = ast.parse(inspect.getsource(module)).body
    140         root = [n for n in tree\
    141                 if isinstance(n, ast.ClassDef) and n.name == self.name][0]

~/.pyenv/versions/3.7.5/lib/python3.7/inspect.py in getsource(object)
    971     or code object.  The source code is returned as a single string.  An
    972     OSError is raised if the source code cannot be retrieved."""
--> 973     lines, lnum = getsourcelines(object)
    974     return ''.join(lines)
    975 

~/.pyenv/versions/3.7.5/lib/python3.7/inspect.py in getsourcelines(object)
    953     raised if the source code cannot be retrieved."""
    954     object = unwrap(object)
--> 955     lines, lnum = findsource(object)
    956 
    957     if istraceback(object):

~/.pyenv/versions/3.7.5/lib/python3.7/inspect.py in findsource(object)
    766     is raised if the source code cannot be retrieved."""
    767 
--> 768     file = getsourcefile(object)
    769     if file:
    770         # Invalidate cache if needed.

~/.pyenv/versions/3.7.5/lib/python3.7/inspect.py in getsourcefile(object)
    682     Return None if no way can be identified to get the source.
    683     """
--> 684     filename = getfile(object)
    685     all_bytecode_suffixes = importlib.machinery.DEBUG_BYTECODE_SUFFIXES[:]
    686     all_bytecode_suffixes += importlib.machinery.OPTIMIZED_BYTECODE_SUFFIXES[:]

~/.pyenv/versions/3.7.5/lib/python3.7/inspect.py in getfile(object)
    645         if getattr(object, '__file__', None):
    646             return object.__file__
--> 647         raise TypeError('{!r} is a built-in module'.format(object))
    648     if isclass(object):
    649         if hasattr(object, '__module__'):

TypeError: <module '__main__'> is a built-in module

Not sure what the problem is?

savingoyal commented 4 years ago

Currently, you can't author your flow inside a notebook.

MasterSkepticista commented 3 months ago

Hi, has this changed? Or are there plans to support flows within notebooks?

madhur-ob commented 3 months ago

@MasterSkepticista Please see https://docs.metaflow.org/metaflow/managing-flows/introduction This should be related and was released just yesterday.