Install modules via an EventGhost function

[kdschlosser]

I tried to run PIP from a shell and i got a traceback.

Traceback (most recent call last): File "", line 1, in File "pipinit.pyc", line 14, in File "pip\utilsinit.pyc", line 22, in
c
File "pip\compatinit.pyc", line 26, in ImportError: No module named ipaddr

I had an idea for a solution to getting modules installed that a plugin may require without having to have the user of said plugin jump through hoops to have it run.

and it would be as simple as trying to import the module and if not successful to install the module via pip or setuptools

setup tools imports properly but gives this traceback upon trying to install a module.

Traceback (most recent call last): File "", line 1, in File "setuptools\command\easy_install.pyc", line 2271, in main File "distutils\core.pyc", line 151, in setup File "distutils\dist.pyc", line 953, in run_commands File "distutils\dist.pyc", line 971, in run_command File "distutils\cmd.pyc", line 109, in ensure_finalized File "setuptools\command\easy_install.pyc", line 273, in finalize_options File "setuptools\command\easy_install.pyc", line 1322, in _expand File "distutils\cmd.pyc", line 311, in get_finalized_command File "distutils\dist.pyc", line 845, in get_command_obj File "setuptools\dist.pyc", line 511, in get_command_class File "pkg_resourcesinit.pyc", line 2230, in load File "pkg_resourcesinit.pyc", line 2236, in resolve ImportError: No module named install

I am not sure if there is a way to make these two modules work inside of EG. It would be nice to have that ability and would solve the difficulties with wanting to use outside modules for a plugin and not having the user have to install python and install modules from python to get a plugin to work. that's a process i do not believe most people would want to do. or even know how to do and opens it's self to a whole bunch of other issues.

[blackwind]

I like what you're thinking. Here's what you need to know:

When py2exe builds python27.zip, it looks through all our source files and only bundles what it thinks is needed, so when you import something and it gives you an error, it's because not all of that module was included. To fix, open Build.py, add the module to includedModules, and run python Build.py -b to rebuild the zip.

If you can come up with a small function that installs a module, I'll accept a pull request for it. I'd like it to work like this:

try:
    import mymodule
except ImportError:
    eg.Utils.InstallModule("mymodule")

No fancy GUIs or anything, please -- just the function.

[kdschlosser]

alright sweet, need to know a couple of things first.

Can the zip file be rebuilt on the fly? without the need to restart EG?

would I add the modules that are to be installed into the included list? or would it have to go and download the module each and every time?

and if the modules are to be included into the included list is this list in a separate file or inside of a python code file?

and if the list is inside of a python code file would it be possible to have them put into a separate file? it would be easier to maintain that list if it's in a separate file.

and is there any function of EG that can notify a plugin of it's removal? that way it could delete the imported module from the list and rebuild the zip file on uninstall. no need to make EG all kinds of bloated if a plugin is not being used at that point in time.

need to know those couple of things before i go and dive into this to see if i can get it to work properly.

[blackwind]

would I add the modules that are to be installed into the included list?

You'd add setuptools and anything else you need to create the function. Importing setuptools doesn't work right now, so you need to add it to the list and rebuild. End-users don't have access to the builder, so includedModules isn't relevant to them.

To be clear, any module you install via the setuptools or pip modules should be added to lib27\site-packages and NOT python27.zip. Only the builder touches python27.zip.

[kdschlosser]

OK cool that was one of the questions. and i can import setuptools and it runs. but it gets a traceback. I would also like to have the plugin remove the installed module if the plugin is uninstalled. is there a function that is already builtin that would notify a plugin of it's uninstall? calling the init of the plugin means it's installed. and the creator of the plugin could do the import there and assign the installed module to a self. variable. i know this would be slightly inconvenient but would allow for module installation only if the plugin was installed and not when eg loads the file.

i am trying to get a feel for how you would like to have the installation feature work.

maybe i will code up some different ways to achieve this and you could choose which one you would like to use.

[topic2k]

My 2 cents: Integrate this feature into plugin loader. That way we know which plugin was missing a package and we can keep a database with that information. On start of EG it can be checked if the plugin(directory) still exists or not and in the latter case remove the package if the last plugin using it is deleted.

and is there any function of EG that can notify a plugin of it's removal?

There is the plugin method __close__() that will be called when EG quits or the plugin is removed from config tree, but there is no notification about (from directory/harddisk) removed plugins.

[blackwind]

Looks like you guys are thinking much bigger than I was. Let's recalibrate expectations on this:

What I wanted from this function is a solution to @ThomasBott's snippet sharing problem. There's absolutely no reason plugin developers should be pulling modules on-the-fly, as they can just dump their modules into the plugin folder and distribute them with their plugin in the first place. Plugin developers using this function will only result in headaches like:

• PyPI was temporarily down when the user installed the plugin.
• An untested version of the module breaks the plugin.
• One plugin requires one version of the module, another plugin overwrites it with the version it requires.

This is an all-out support nightmare waiting to happen, and yet, if we provide the function, eventually, some developer somewhere is going to use it even if we scream at them throughout the code not to, so let me add an additional caveat here: If we're going to do this, the function needs to work only from Python Command and Python Script actions. How that might be accomplished, I'm not sure.

[kdschlosser]

that bit is easy. if you just want it to work with python script actions. or command actions.

i have also started making an override for the pip and for the setuptools so that on import it will call the override instead and the override uses inspect to see who the calling class is and if it isn't the eg.Utils.InstallModule it will not allow the installation of the module. this i wanted to put into place that way it can be monitored as to what has been imported and what hasn't by using the inspect once again to get the calling file and store the file and module name in the eg.persistentdata. and during boot it would create a temporary dict with that information and once boot has completed then compare the data and if the data isn't the same to uninstall anything that is in the eg.persistentdata and not in the dict and set the persistent data to the temp dict. have the new module overridden to point to a definition in the InstallModule class that way it would be able to keep track of what installed module is in use or not.

I basically just went through something similar with the creation of the Python Script editor plugin i made.

this is also a convenient way for not allowing installation of modules that would override the core installed modules. i believe if this is done properly there would be no support nightmare for EG but maybe for the plugins that use this feature if they don't keep their plugins up to date with the latest versions of a specific installed module.

kinda puts it out of the hands of the EG dev team. and can have a specific disclaimer about using it.

I will code up something that will make this a simple to use thing but still allow control from the core code but also have it check for modules already installed. and what plugin is calling for the module and if it is not the plugin that installed the module to have it do one of 2 things. either simply post a warning in the log. or in the debug log file. where ever you think it would be the best place. but as a default doesn't pip and setuptools automatically install the latest version of a module anyways? and we could restrict to only allowing that specifically. that would also take care of a clashing version problem.

we can also add a list of modules that are not to be installed because of known problems with EG so this kind of solution i think would be best over just having the user install python and modules. at least this way it becomes easier to manage the problems as they arise. I also think it would really broaden the abilities of EG and really allow the plugin devs to maximize what their plugins can do. it takes off the shackles of restriction by only being able to use the core libs.

let me do this. I will make something up. and if you like it. great. and if not. it's not going to hurt my feelings at all. and if it needs work. then we brainstorm and see if we can come up with the best possible solution.

[kdschlosser]

and even if the module is installed via a python script or command wouldn't it be installed globally?? or is it just installed for that one exec instance??

[blackwind]

and even if the module is installed via a python script or command wouldn't it be installed globally?? or is it just installed for that one exec instance??

Globally, to lib27\site-packages as noted above, with the exception that stock modules (agithub, pycurl, qrcode, requests, tornado, and websocket) shouldn't be overwritten. So...

• A user like ThomasBott adds a Python Script action that calls eg.Utils.InstallModule(). The function allows the install to take place.
• He copies his action and shares the XML with another user. Because it's still a Python Script action, the function allows the install to take place.
• A plugin developer tries to call the function from within his plugin. He gets an error message telling him to just bundle the damn module.
• A plugin developer tries to be clever and install the module by calling setuptools or pip directly. He also gets an error message telling him to just bundle the damn module.

It occurs to me we should also allow installs from PyCrust ("Python Shell") and that we should also provide eg.Utils.UninstallModule(), which also wouldn't touch stock modules.

There's no need for eg.PersistentData or plugin lists or any of the other complexities you've described. These functions are for regular users only, solely to remove the necessity of installing Python to add modules.

[blackwind]

In fact, we can't install to lib27\site-packages thanks to UAC, so that makes things even simpler. %PROGRAMDATA%\EventGhost\lib27\site-packages. This folder will come after lib27\site-packages in sys.path, so a user couldn't override stock modules and break things even if they tried.

[kdschlosser]

ok so you just want this to be available to the python scripts. and you want an error to occur if tried from anywhere else.

and you also want an uninstall routine for it.

this is all very simple so far.

should i still l do the overrides that make sure that pip and setuptools cannot be called from anywhere except the eg.InstallModule?

and I am going to send you a PM about one other thing as well.

[blackwind]

Correct to all of the above.

should i still l do the overrides that make sure that pip and setuptools cannot be called from anywhere except the eg.InstallModule?

I'd be interested to see what you can come up with in that regard. The simpler it is, the more likely I'll accept it.

[kdschlosser]

I am going to make it as simple as possible. so what i think i am going to do is for simplicity sake is have the user import the pip module. which won't really be the pip module it will be my override instead. and when they try to use anything in the pip module i will have it do the check at that point to see if it's the eg.Utils.InstallModule that is calling and if it isn't throw an error then. I would like to keep with EG coding styles so I need some guidance on the error message you would like to have displayed.

[blackwind]

I don't think we have any real standard as far as error messages go. As long as you've read CONTRIBUTING.md (and all the links it references) and follow it to the letter, I probably won't complain. I might tweak your messages myself after merging, but I probably won't complain.

[kdschlosser]

I get this traceback when i ass setup tools to the included list. you may know the solution.

Exception in thread Thread-9: Traceback (most recent call last): File "C:\Python27\lib\threading.py", line 810, in *bootstrap_inner self.run() File "C:\Python27\lib\threading.py", line 763, in run self.__target(_self.__args, _self.kwargs) File "C:/Users/Administrator/PycharmProjects/EGModuleInstall/_build\builder\Gui.py", line 139, in DoMain builder.Tasks.Main(self.buildSetup) File "C:/Users/Administrator/PycharmProjects/EGModuleInstall/_build\builder\Tasks.py", line 150, in Main task.DoTask() File "C:/Users/Administrator/PycharmProjects/EGModuleInstall/_build\builder\BuildImports.py", line 128, in DoTask for module in GetPackageModules(package): File "C:/Users/Administrator/PycharmProjects/EGModuleInstall/_build\builder\BuildImports.py", line 201, in GetPackageModules mod = import**(package) File "C:\Python27\lib\site-packages\setuptoolsinit.py", line 14, in from setuptools.extension import Extension File "C:\Python27\lib\site-packages\setuptools\extension.py", line 10, in from .dist import _get_unpatched File "C:\Python27\lib\site-packages\setuptools\dist.py", line 39, in _Distribution = _get_unpatched(_Distribution) File "C:\Python27\lib\site-packages\setuptools\dist.py", line 35, in _get_unpatched "distutils has already been patched by %r" % cls AssertionError: distutils has already been patched by <class py2exe.Distribution at 0x033AACE0>

[blackwind]

Nothing I've seen before, no. You'll have to do some debugging and/or Googling.

[kdschlosser]

ok, so here is an update.

this is what i have thus far.

I have easy_install working

pip is almost there

i have it set to eg.InstallModule and eg.UninstallModule

how it functions.

eg.InstallModule('setuptools', 'MODULENAME', upgrade=bool)

same deal with uninstall without the upgrade

it runs the site module after import to refresh in imported modules

how i have this thing set up to control access to it from the Shell, Script, and Command is it uses inspect to grab some information and if it doesn't line up rejects the request to install.

now i also did the same for every module included in the pip and easy install package. they can only be called form them selves each other and only from the 3 above mentioned places.

we don't want just random use of the installers themselves.

I need to know of the location you want the newly aquired module installs to go. and it needs to have full permissions. the use of easy_install to remove a package requires me to code in a way to delete the egg file.

i have not provided a means to specify a version on install or a way to specify one on upgrade. i do not know if this should be done because of conflicting versions

I did have to add one line outside of the Utils file and it's in the build.py file the first line. i had to import setuptools before py2exe got it's hands on it and patched that distutils causing the traceback i posted previously about.

i also had to redirect the stdout and stderr to be able to remove the ansi escape codes and to strip all the extra whitespace from the output. as well as add the flush and the isatty since the default EG redirect doesn't provide provisions for this and is something both pip and setuptools uses.

i think i should be finished with this feature in a couple of days possibly. just depends on what life tosses at me, so we will see.

i am going to add a verbose option to it as well. it's nice to be able to have the extra information if needed

[blackwind]

i have it set to eg.InstallModule and eg.UninstallModule

eg.Utils.InstallModule and eg.Utils.UninstallModule, I hope.

eg.InstallModule('setuptools', 'MODULENAME', upgrade=bool)

Is there a reason you're allowing the module to be installed three different ways? Shouldn't the pip method be sufficient?

I need to know of the location you want the newly aquired module installs to go.

join(
    eg.folderPath.ProgramData,
    eg.APP_NAME,
    "lib%d%d" % sys.version_info[:2],
    "site-packages"
)

i have not provided a means to specify a version on install or a way to specify one on upgrade.

Let's keep this simple. Users who need more functionality can install Python.

[kdschlosser]

3 different ways?

it's pip and setuptools. and setuptools has to be functioning for pip to work.

and you instructed me to have this feature work from Python Script, Shell and Command.

it wouldn't work from Command if you had to do

from eg.Utils import InstallModule InstallModule(.....)

thats 2 lines of code. and python command is for a single line statement

and the method i came up with is the easiest way i could think of. and I am sure it can be improved upon. this is a first draft.

and i really don't think it could be much easier to use then the way it is unless you limit the choice of what installer to use

[blackwind]

it's pip and setuptools. and setuptools has to be functioning for pip to work.

You mentioned easy_install as well. In any case, the number is irrelevant -- what I'm asking is, is there a reason to support multiple methods? If not, remove that parameter and just use pip.

thats 2 lines of code. and python command is for a single line statement

Try print eg.Utils.DecodeMarkdown("* Test") from a Python Command. And don't forget about the semicolon. from eg.Utils import DecodeMarkdown; print DecodeMarkdown("* Test") works just as well.

and i really don't think it could be much easier to use then the way it is unless you limit the choice of what installer to use

When I said "let's keep this simple", I was referring to the quote above that:

i have not provided a means to specify a version on install or a way to specify one on upgrade.

In other words, don't bother allowing a version to be specified.

[kdschlosser]

Easy_install is setup tools. Easy_install is part of the setup tools package. And from my own personal experience is that if I have an issue with one of them the other will work. And since I had to make the thing work using both anyways because pip uses setup tools it was simple to add the option to use setup tools as well. As you had stated if pip for some reason fails then it becomes a support issue. This at least provides a second method to give a go at. If you would like to separate the 2 into eg.Utils.PIP.Install() eg.Utils.SetupTools.Install()

I kind of like that better personally but this is not my decision to make. And by separating like that and with the sub type it makes it easier to initialize and handle the who's doing what. The only thing is with setup tools it doesn't remove the egg on uninstall I am not sure what pip does haven't gotten it working yet. I need to look about in the installed module to see if the egg name is stored there or if I can get the name from setup tools somehow.

[kdschlosser]

Or even do eg.Utils.Package.Install() eg.Utils.Package.Uninstall() eg.Utils.Package.Upgrade()

If you only want to use one package installer setup tools is ready to go. I am not sure if I will even be able to get pip working because it keeps on spawning a second session of EventGhost. And that causes it to hang up. But it gives no errors so I am not sure how to go about figuring out what the issue is.

[blackwind]

Keep working on pip for now. If you get it working, we'll use only pip; if you don't, we'll survive with only easy_install.

[kdschlosser]

well good news is. I have got pip to finally function. mostly. LOL

still working on the Uninstall bits.

it's funny though because if i install a package using easy_install from EG it installs fine. and if i uninstall the same package from pip. it uninstalls fine.

but if i install it from pip no errors and all is well. but if i uninstall it from pip. it hangs. I think it has something to do with how pip handles whee the packages are installed. easy_install has that nice .pth file. and pip uses that file for uninstalling the package. but pip doesn't create one during install.

and if i use site it updates all of the paths like it is supposed to but it only works if there is a .pth file.

and pip has a really nice and simple --target for installing modules to places outside of the normal site_packages. this folder of course has to be added to the sys.paths but that is not really a huge issue because the first thing EG pretty much does after creating the dynamic module and processing the CLI args is import the whole Utils file.

rather frustrating.

i am so close to having this thing work properly.

[kdschlosser]

ok this is it. I just have a few spots to polish up and I have to really learn how to do a pull request in github.

but. this is how i have it working.

this only allows for pip to function

eg.Utils.Package.Install(str(name), bool(verbose)) eg.Utils.Package.Uninstall(str(name), bool(verbose)) eg.Utils.Package.Upgrade(str(name), bool(verbose))

and for the people that like a little familiarity

eg.Utils.Package(['install', '--verbose', str(name)])

and so on and so forth

this can only be run from the Shell, the ScriptEditor, and Command

no piece of this can be overridden a traceback will occur if it is tried.

the pip installer has been wrapped so it cannot be overridden as well and can only be called from eg.Utils.Package

I have also made it so that if someone gets crafty and tries to do this at the beginning of a script

try: import testimport except: eg.plugins.EventGhost.PythonScript("eg.Utils.Package.Install('testimport')") import testimport

no bueno

this will not work either

:-D

now this was a very crafty bugger to get running. I had to override the stderr, stdin, and stdout for the instance when pip is originally imported, and for every other time pip is accessed by the Package class. it then will put back in place the EG versions of the stdin, stdout, and stderr. there is one issue which i cannot get around and that's because of windows and a temp file being locked because it is in use. and that is when pip goes to uninstall a package and delete temp files. but since i had to override the stderr anyways i filtered out the error since it is a temp file anyways as not to get an ugly screen of red. I had to do the overrides because of parsing ANSI escape codes as well as the EG defaults do not have flush and isatty. I had to override the stdin for pressing the y for continuing. this may or may not need to be improved upon due to the fact that i do not know what every input kind of question may be. so i just have it set to default to y for everything at the moment.

i was going to make the installer parse a known list of incompatible packages but for sake of not having redundant code because the list that is in the builder is not added into eg at all. and having to maintain 2 separate lists is bonkers.

i was also going to have to check a list of core packages and and not install based on that. but i then discovered that if there is a problem and EG won't start if one of the core packages had been updated then you could manually delete the folder for the package and start eg back up and all was well again.

and you can update a core package of EG if you like. but as soon as you restart EG it's back to normal and same goes for uninstalling a package. I will have to test this once again just to confirm, too many things rolling around in my head over this to exactly remember.

i do not know how to stop EG from popping a Not Responding for a very small period of time while pip is doing it's thing. if you have a slow internet connection this small period may become a longer.

all the files are written into 2 folders under the ProgramData\EventGhost\lib27 folder.

there are 2 directories site-packages scripts

now being that i used site to set the USER_BASE and USER_SITE a cool feature dawned on me. EG could be set up to run a specific tree and load a specific set of packages based on what user logged in. as long as the site.USER_BASE was pointed to the %USERPROFILE% windows environment variable.

if there is anything i missed please let me know so i can finish this up therefor doing specific tasks per user login. and this could be accomplished quite easily.

[blackwind]

Wait, are you saying pip doesn't have an actual API and all the module is doing is running shell commands?

[kdschlosser]

ya lost me there, this imports pip. overrides it to add a "permissions" check if you will to make sure it is being run from only the eg.Utils.Package. it adds this functionality to every piece of pip. not just the main package

and the eg.Utils.Package does a permissions check to make sure it is only being run from the script, command or shell.

and pip has been set up in a way that only pip can modify any of it's attributes

and the packge installer is set so that no one can set any attributes. there for neither one can be overridden in a way that would allow a module to be installed other then using the eg.Utils.Package and only from the 3 mentioned locations.

this is the only way to do this. unless you wanted me to have it do the import via command line by using subprocess. and i don't know if altering the build of EG to allow for the pip.exe script to be added to EG would have been the best route

[blackwind]

So, what I was expecting was that there'd be an API in the module like pip.install("paramiko"), which would do its thing, throwing an error if needed, which could be handled in your function. It sounds, though, like what's happening is that there's no "real" API and that the module is instead just executing shell commands and you're parsing and interacting with them on-the-fly.

Though I hate to put the kibosh on this after you've done all the work, I'm sure you understand why this is a bad thing and why I might not want to accept it into core as is. pip updates from v8.1.2 to v8.1.3, slightly changes its output, and suddenly, I have to deal with a bunch of new GitHub support tickets; the command does something you weren't expecting, and suddenly, I have to deal with a bunch of new GitHub support tickets. Since I'm not getting paid a dime for this, you can probably imagine how much that prospect tickles me.

Go ahead and release your work as a plugin if you're feeling adventurous and don't mind dealing with the inevitable influx of complaints yourself. Should there come a day when pip has a "real" API, we can re-evaluate then. I still like the idea and would still like to have something like this in core, but the current tools at our disposal just don't cut the mustard.

[kdschlosser]

no no no, if pip throws a traceback it passes through. the only one that is caught and dealt with is a traceback.print_exc() that pip does, it doesn't throw a traceback for it just print one out and continues along it's way. and it's for the deletion of a temp file that it is unable to do because it's in use. i had to override the stderr and stdout because the eg.Log versions do not have flush and isatty and also pip outputs a hell of a lot of ansi escape characters in which EG doesn't like so much when printing. so those have to be filtered.

and because EG is gui based and i do not know enough about the inner working of the stdin i just have respond with a y for yes when pip asks if you want to continue on the uninstall. i have tried to do something that would generate a user prompt. but it seems as though the output bits of pip and the input bits are run in 2 different threads. so i cannot use the output to trigger the input dialog because it actually waits for the user input before the prompt even hits. which in turn gets an EOL error. i am sure i can figure that out in the future but at the moment i have only found the one prompt for input so it's not a biggie in my book.

every single module i have tried to install and uninstall works perfect. i even got rid of the need to reboot EG on module uninstall. so it loads the module real time and removes it real time. and gives tracebacks like it should.

i added some functionality to it thinking the exact same way you are about it. pip wouldn't throw a traceback if you tried to upgrade it and it didn't exist. it would just print something to the screen.

the way i set it up is if you try to do an upgrade and it doesn't exist. it prompts asking if you want to install it

or if you install it and it is already installed prompts you if you want to upgrade,

so i have added an API to pip technically speaking. i just have handled that one error about deleting the temp file as not to have question about it because it does it with every single module uninstall.

if you run it in debug mode the traceback for the temp file will show.

the biggest thing was setting up the "permissions" so to speak. and make it so that it cannot be tampered with. and pip is locked to only download from pypi and no where else. and it always writes the packages to the --user which is set using site

that can be changed by the user if they so want to as i do not see any harm in installing a module elsewhere if they want.

they would just have to point the site.USER_BASE and site.USER_SITE to where ever it is they want.

and if they want to upgrade something like wx they can it does not alter the files in the zip. and since the user site_packages is added to the end of sys.path i believe it overrides the core one. but in the event EG does have a problem after doing somehting like that the user couls simply navigate to the user site_packages folder and simply delete the package, and the original with still function properly.

i am sure i could do some magic code and have it so that it would read every package in the user site_packages and import them all and if one fails to remove it from use and post a message about it. and still have EG working properly. and if you would like that done let me know it shouldn't be all that hard.

and there is no way this could be done as a plugin. because of the order in which things need to be done and the plugins load to late in the startup process.

you have to try it and see if it will fit the bill.

i am almost done polishing up the code i have worked out the last few things i needed to. and i will post it up on my fork of EG and you can take a look at the code and i have only been programming for about a year and change. so i know there are ways to do things better i am sure. but for what this does 400 lines is pretty small. and if someone that is much more polished at it could probably easily shrink it another 100-150 i am sure.

[blackwind]

Well, nothing you've said here allays my concerns, but I'll certainly take a look when it's posted. For now, just post a link here in the issue instead of sending a pull request.

[kdschlosser]

yeah i'll do that. i guess i am not fully understanding what your concern is

[blackwind]

In short, exactly what led you to pass on updating the Spotify plugin -- without a proper API, anything can change at any moment and break everything you've done. And here, I'll be the one who ends up having to support it.

[kdschlosser]

ok since it's pretty apparent that you have already made up your mind. i will just keep it for my own personal use. as I know it will make my setup a whole lot better adding the flexibility.

and i passed up the spotify plugin because of developers not caring about anything that may rely on the API and they just up and change it. most times without even notification of a change to the people that make software that depends on that API. they don't depreciate the old variation. I personally believe the people that write plugins or extend the functionality of another piece of software are the ones that make the core software what it is. and a little bit of courtesy keeps those people writing. and if not they will stop adding functionality.

and my thought is why should i help extend someones software when they don't care.

[blackwind]

I agree, and here, it isn't even an API you're working with -- it's the output of a command that has no obligation to remain the same indefinitely or even version-to-version. When -- not if -- the authors of pip change it in some fashion, whether that's...

• Changing "Proceed (y/n)?" to "Proceed (Y/N)?" or "Proceed [y/n]?"
• Adding another proceed prompt somewhere you haven't handled
• Writing something to stderr rather than stdout

...or any number of other things, no matter how hard you try to prepare for the future and no matter how crafty you are, your code is eventually going to break. And it won't even be a matter of not caring on their part -- it's a matter of, this isn't an API, so don't expect it to be stable.

If they had a proper API, we could count on it to be consistent, and I'd have no qualms with leveraging it. As is? I'd prefer not to be holding a ticking timebomb, just waiting to go off at any given release.

As I say, I hate to pull the rug out from under you at the last minute, but it seemed like a reasonable assumption that, if they have a pip module, it would contain a proper API. Yet another lesson in the dangers of assuming, I guess.

[kdschlosser]

i am not relying on Changing "Proceed (y/n)?" to "Proceed (Y/N)?" or "Proceed [y/n]?"

the script doesn't rely on anything that pip outputs. the only thing it does is intercept one message. Should it actually be printed. and if not it happens along it's merry way.

in no way did i state that it relies on anything that is output as information for the module to function.

the only thing that it looks for is if there is a printed traceback from the traceback module which does have an API to handle only one exception that is an irreverent one and doesn't need to be printed

i did state however that i need to find a way to pass user input queries from pip to the user. this is information stuff for you or perhaps a reader that may know a solution and would post it.

you are over analyzing and guessing at how the code is written and how things are done

[blackwind]

And you're cherry-picking singular points made in an attempt to get through to you via specifics and not considering the totality of what I'm saying. If it helps:

I agree, and here, it isn't even an API you're working with -- it's the output of a command that has no obligation to remain the same indefinitely or even version-to-version.

...or any number of other things whether or not they're related to output

It doesn't have to be the output that changes -- it could be extra input requests, it could be changing what it expects from an input request, it could be dumping a different traceback you weren't expecting (or doing so in a way you weren't expecting), it could be changing how a particular parameter works, it could be changing how a particular command works, and I'm still only scratching the surface of possibilities. Yes, you can probably refute one or more of the above points, but you can't refute all possible possibilities without a crystal ball and an aptitude for precognition.

Again, bottom line, without a proper API, anything can change at any moment and break everything you've done. What works for you today in the version you're using might not work tomorrow. For me, that's a problem, and further belaboring the point isn't going to change that -- only a proper API will.

In this and all other matters, if it's fragile, I won't accept it for the good of the project. As CONTRIBUTING.md notes, try not to take it personally -- I'm only doing what I think is best for everyone, and I'd be telling anyone else the same thing.

[kdschlosser]

I'm not taking it personally. I am trying to get a better understanding of what you want for something to be added.

but I will bring up case in point. python WX. there is an API for that. and low and behold they changed some things about and it caused all kinds of problems. so an API doesn't mean that something will never change and it will never break your code. look at Spotify since you brought that up. and that's the reason the original author of the plugin stopped maintaining it. because they kept on changing things in their API and it kept on breaking his code.

so there are no guarantees on how something works. we roll with the punches. and as far as pip goes. the only way it would break is if the version of pip got changed. and that's pretty simple to control. when building EG.

and there is nothing to say that py2exe wont change something and it breaks the builder. so how do you fix that?, you simply don't upgrade it until you are ready to spend the time to repair it.

and I am sure there have been changes made to EG since it's inception to make it work properly with the different windows versions. and Windows has an API as well

[blackwind]

There are certainly exceptions to every rule. Knowing what we know now about the wxPython authors' approach to development, if we were starting from scratch, we might go with something like Qt instead. "We already have one source of future headaches, so what's the big deal about adding another one?" isn't a good reason to proceed. I have enough headaches to deal with as is.

When we add something new to EventGhost, I want it to stand the test of time, regardless of version. If there are red flags indicating that it might not (as there are here), we either approach it a different way, or we don't do it at all; if there aren't red flags going in but it later turns out to be problematic, then we roll with the punches. No matter how you spin this and no matter how long you spend arguing, a nifty feature that four people will use just isn't worth the inevitable maintenance headache, and you'll never convince me otherwise.

My advice for the future: If what you're working on has a dependency on something else, make sure it has a proper API and that the developers have a good track record.

At the end of the day, the more time I spend debating decisions that have already been made, the less time I have to spend on development. Let's get back to what matters, shall we?

[kdschlosser]

@topic2k being that this package installer will not be included into EG at all i have decided you do as you suggested and i thought was a great idea. and as it turns out there is a method for deleting a plugin. so i nested the package installer into the plugins section and added it to the eg.registerplugin. you would add a name or a tuple of names if more then one is needed.

it uses pip to do the legwork and is locked to pypi and all of the classes that do the work including pip are protected. and you can only run the installer from a few places. and one of them being the plugin tree item creation. i also added GUID's to all the tree items to be stored in the xml file. so now all the features that you couldn't run from code. work. if ya want to check it out i hope to have it up on my fork sometime this evening. just trying to iron out one last kink.

[ThomasBott]

What? It's not implemented? I didn't follow the topic in detail, but I assumed seeing that this issue is closed without the wontfix tag that it has been implemented... It's a shame really, I would have loved to install packages from the python shell in EG, that would be so nice for development... @kdschlosser will you create a plugin that allows that? I would so going to use it! :)

[kdschlosser]

unfortunately due to how EG loads it has to be something done at the build level. because there has to be some things that get set during build.

As of now I do not have an unmodified version of EG. but if you give me a day I will take the current beta and outfit it with the package installer.

it runs rather nicely actually. i also added the ability to call it from the eg.RegisterPlugin

so if you have any custom plugins you have made for yourself it's a little easier not having to remember what plugins need what modules. it's a set it and forget it kinda thing.

PM me on the forum and i will chat with you a little more.

[JGuire]

It's 8 years later... So, what happened with this? I'd love to be able to install sounddevice in eg.