Fixes #19 and closes #35. Pandas job submission now works.
This commit alters the function submission system imports to import all
modules from the calling script, making it much more robust.
Additionally, this alters the way that get() handles outputs, it now
returns the function return value for functions and only STDOUT for
scripts. By default it now saves the output, STDOUT, and STDERR to the
parent Job class and then deleted all intermediate files.
To allow more fine-grained control, I implemented a number of new method
arguments to control when outputs are saved and when files are deleted.
I also added an auto_delete attribute to the class, which defaults to
False, that can be used to set behavior class-wide.
To handle these changes I had to alter all of the test files, I
additionally made changes to them to make it easier to get verbose
output.
Finally, I implemented a new pandas based test that thoroughly tests
the new function submission system. Because pandas is hard to install on
older versions of python, I made this new test separate from the
py.test system.
To make a full suite of tests easier, I updated pyenv_tests.sh to run
the pandas tests also.
Makes code obviously 2/3 compatible, fixes some minor codacy issues and updates the README.
Fixes #19 and closes #35. Pandas job submission now works.
This commit alters the function submission system imports to import all modules from the calling script, making it much more robust.
Additionally, this alters the way that
get()handles outputs, it now returns the function return value for functions and only STDOUT for scripts. By default it now saves the output, STDOUT, and STDERR to the parent Job class and then deleted all intermediate files.To allow more fine-grained control, I implemented a number of new method arguments to control when outputs are saved and when files are deleted. I also added an
auto_deleteattribute to the class, which defaults toFalse, that can be used to set behavior class-wide.To handle these changes I had to alter all of the test files, I additionally made changes to them to make it easier to get verbose output.
Finally, I implemented a new
pandasbased test that thoroughly tests the new function submission system. Because pandas is hard to install on older versions of python, I made this new test separate from thepy.testsystem.To make a full suite of tests easier, I updated
pyenv_tests.shto run the pandas tests also.Makes code obviously 2/3 compatible, fixes some minor codacy issues and updates the README.