Provide an interface equivilent to exec

[stuaxo]

Hi, I'm one of the developers on shoebot (like nodebox or processing). Adding live code reloading would be a killer feature :)

At the moment, for each frame we render, I exec the 'bot code' (== the code in a processing sketch).

Would it be possible to provide a very simple interface, that is similar to exec ?

I've got some ideas to speed up the changes too - especially if the editor knows which region of the source it's changing, it may be possible to map parts of the code to regions of the source, so that small parts could change, this would be great for changing simple values more quickly.

But first things first ... so, yes - if I could something like

from pyhotreload import hotexec

...
hotexec(python_code)

It would be pretty nice :)

[mdsitton]

So first the slow part of the hotreload code isn't the reloading code, Its the file monitoring code. Which I've been thinking about using watchdog for. https://github.com/gorakhargosh/watchdog

So it would be pretty easy to implement once i make a couple of changes. Basically would need to add in some methods to represent a lot of built-in functions i use on the modules being reloaded. ex: example setattr getattr

Then the exec version you want could be implemented as a subclass of the HotReload class.

Edit: Created the methods.

I would go ahead and see if you subclassing it can get the result you want. Looking at your project any solution for it would be pretty specific to that project i think. It wouldn't make to much sense to put it here on this repository.

[stuaxo]

I guess what I really want is an easy way of using hot reload from and IDE / editor (in my case gedit).

From an IDE plugin, I can easily tell hotreload which file has changed (and possibly even which bit of it).

I was imagining that there would be a method to exec/reexec if the code has changed - I could just call this every time an edit occurs.

[stuaxo]

In other words, I'm thinking the hotreload could be useful for many projects if it was easy to hook into editors / IDEs.

[stuaxo]

Watchdog looks pretty good :) expecially the cross platform part.

[mdsitton]

My only issue with using watchdog though is that i doubt it will work on pypy :P

[mdsitton]

So i split the file detection from the reloading part of the code. Basically to use it you subclass BaseReload.

You could use BaseReload on its own i guess if you just want to pass in a path to a file to be reloaded, as long as it can be found by your current python install.

You would do this:

from hotreload import BaseReload

hotreload = BaseReload()

if hotreload.init_module(pathToFile):
    hotreload.reload()

You would only initialize BaseReload once, and reuse the same instance for all reloads you do.

In order to have something usable with exec like how you were originally looking for you would need to replace the init_module, getmoduleattr, setmoduleattr, and delmoduleattr methods to work with a dictionary of instead of module objects. Which should be pretty easy.

Im actually going to do some tests with exec now, just to see if it actually will work.

[stuaxo]

Cool .. one last thing - is there a way I could pass in code as a string (I can fake a filename if needed) ?

This is useful in editors when the file may not have been saved yet.

[stuaxo]

I've been playing with livecode reloading a little more + got a toy version working in shoebot.

https://www.youtube.com/watch?v=foxzx0JFU5g

https://github.com/shoebot/shoebot/tree/cmdline_reload

In shoebot I have this working by having two states, to the code KNOWN_GOOD and TENUOUS.

If code is KNOWN_GOOD and there is an exception, it happens as normal.

When code changes in the editor, I send it to shoebot, the new code is TENUOUS - if an exception happens, the code reverts to the last KNOWN_GOOD code. If it runs successfully for one frame, that code becomes KNOWN_GOOD.

The reloading is knowhere near as advanced as pyhotreload though.

Having having looked again at pyhotreload, the thing needed is a way to tell it to stop watching one of the files, and replace it's content with an arbitrary string.

(In shoebot I do this from a commandline that can be loaded).

If it's possible to add the above API, I can add a commandline example, like above + a demo gedit plugin.

• Basically, as the user types, the editor just sends new versions of the code (as opposed to IDEs that keep the file constantly saved)

APIs needed - stop watching ( file_path ) set content ( file_path, content ) start watching ( file_path )

[mdsitton]

Here is my current thoughts on this.

I wrote pyHotReload to reload whole modules, where as shoebot doesn't use modules for the code that you want to reload. However implementing these ideas might be interesting.

The main issue right now is that since shoebot passes around, a string, filename, or bytecode objects then runs it through exec. Since exec either runs the code in the current scope or with the supplied dictionary as the scope.

Im thinking about refactoring the code a bit to have it not specifically operate on the module level.

Here is an outline of my plans for doing that:

• Refactor how to code reloads, making it more generic.
  • Don't directly operate on modules, create some type of intermediate representation instead.
  • The ir for what is being reloaded would be diff'd, creating two different diff's
  • One for reloading the code.
  • The second would be in-case of a run-time crash so we can undo the new changes.
    • This is how i would implement reverting to the old code after a failed runtime check.
  • The ir could also be used for easier debugging of cases where code isn't reloaded properly

Also if i end up adding support for set_content(), it would probably implement some type of file shadow/override system that would allow you to temporally specify that file as an alias of the given code or file. This would also potentially allow me to do file rename detection without changing module names internally.

For a while I've been wanting to add a way to specify certain folder/files to be watched instead of watching everything.

[stuaxo]

Yup, that all makes a lot of sense.

[mdsitton]

I haven't had as much time as i would like to work on this. Hopefully I can get to working on it some in the next few weeks.