Enable autoreload in interactive mode

[gpleiss]

Is there a way to enable IPython autoreload when using --interactive mode? I've been using this mode frequently in development, and it'd be nice to easily test changes on the fly.

[jtratner]

@dbieber - I wonder if this requires loading the module as __main__

[dbieber]

@gpleiss With some caveats, yes.

You can enable autoreload in the usual way after entering interactive mode:

%load_ext autoreload
%autoreload 2

This will successfully enable autoreload for everything that was imported, but it won't autoreload things in the __main__ module. So when you change things in the file in which fire.Fire() was called, they won't be reflected in IPython. <-- I imagine this is exactly what you want, though; is that right?

One workaround is to move fire.Fire() to a separate file where you import the things you care about, so that everything you might change is being imported and will be reloaded.

I wonder if there's a way to make autoreload think that things from the __main__ module (only those provided by Fire, not those created afterward) are actually from the module that corresponds to the file, so that when the file changes, these objects are updated.

If you're interested in exploring this, the place to start is interact.py:_EmbedIPython. The list of variables is variables, and you can check the module of certain types of objects with variable.__module__.

@jtratner can you elaborate?

[jtratner]

@jtratner can you elaborate?

You basically covered it in your explanation, but for #29 - using load_module rather than load_source changes how autoreload would work.

[gpleiss]

Cool thanks! The separate file solution is a good workaround. I'll explore the __main__ module as well.

[dbieber]

Thought I'd share some of my (failed) experimentation:

I modified interact.py adding the following methods:

def make_reloadable(variables, filename):
  module = reload_filename(filename)
  for name, variable in variables.items():
    variable.__module__ = module.__name__
    setattr(module, name, variable)

def new_reload(m):
  print('reloading module', m)
  try:
    return original_reload(m)
  except:
    return reload_filename(m.__file__)

def reload_filename(filename):
  module_name = os.path.splitext(os.path.basename(filename))[0]
  dirname = os.path.dirname(filename)
  print(dirname)
  try:
    fp, pathname, description = imp.find_module(module_name, [dirname])
  except ImportError:
    pass

  try:
    module = imp.load_module(module_name, fp, pathname, description)
  finally:
    if fp:
      fp.close()
  return module

Overwriting reload:

import __builtin__
original_reload = __builtin__.reload
__builtin__.reload = new_reload

And modifying this method:

def _EmbedIPython(variables, argv=None):
  """Drops into an IPython REPL with variables available for use.

  Args:
    variables: A dict of variables to make available. Keys are variable names.
        Values are variable values.
    argv: The argv to use for starting ipython. Defaults to an empty list.
  """
  argv = argv or []

  items = {}
  for name, variable in variables.items():
    try:
      if variable.__module__ == '__main__':
        items[name] = variable
    except AttributeError:
      pass

  main_module = sys.modules.get('__main__')
  make_reloadable(items, main_module.__file__)

  IPython.start_ipython(argv=argv, user_ns=variables)

I also updated autoreload.py (around line 375) to preserve __file__:

   # reload module
    try:
        # clear namespace first from old cruft
        old_dict = module.__dict__.copy()
        old_name = module.__name__
        old_file = module.__file__
        module.__dict__.clear()
        module.__dict__['__name__'] = old_name
        module.__dict__['__file__'] = old_file
        module.__dict__['__loader__'] = old_dict['__loader__']
    except (TypeError, AttributeError, KeyError):
        pass

Also tried adding this to autoreload after the call to reload(module).

for key in old_dict:
  if key not in module.__dict__:
    module.__dict__[key] = old_dict[key]

No luck. autoreload's update_generic is being called on the objects we want to reload, so not sure what the problem is at the moment. Going to take a break from digging now. Maybe someone will find this useful.