kastore not brought in by installing pyslim

[bhaller]

I guess msprime or tskit used to have a dependency on kastore that would pull it in when they got installed, but now they incorporate kastore directly, so that dependency is gone. And I guess pyslim uses kastore, and used to ride along on the coattails of the kastore dependency in msprime/tskit. Now that that dependency has been removed, you can install pyslim and it does not bring in kastore, and then when you try to use pyslim you get a runtime error that kastore is not installed.

[jeromekelleher]

That's odd, it should be pulling kastore in. We have a dependency in setup.py for pip, and it's in the conda recipe

Do you know how pyslim was when you this this @bhaller?

[bhaller]

That's odd, it should be pulling kastore in. We have a dependency in setup.py for pip, and it's in the conda recipe

Do you know how pyslim was when you this this @bhaller?

Sorry, can you reword the question?

[jeromekelleher]

Sorry, lots of missing words there! How was pyslim installed?

[bhaller]

Sorry, lots of missing words there! How was pyslim installed?

Ah! I think I just did:

pip install msprime
pip install tskit
pip install pyslim

When it became clear that pyslim had not pulled in kastore, I tried to do:

pip install kastore

but weirdly, that installed kastore in a different place than the other packages, and then after that install completed (supposedly successfully), pip list still did not show it as installed, and pyslim still didn't load. This kind of thing seems to happen to me all the time with Python on macOS, and I've never understood why. Why would pip install a package in a place that pip itself doesn't even look for packages in?? But something in a warning message that got issued somewhere along the way (sorry for the vagueness) indicated that installing the wheel package might help things along. So then I deleted the incorrectly installed kastore from disk, and did:

pip install wheel
pip install kastore

That installed kastore in the correct location, and then pyslim worked. Not being a "real" Python user, I have no idea what wheel is or why installing it would change the behavior of pip. :->

I am kinda sure that the initial pip install pyslim did not even attempt to pull in kastore (rather than installing it in the wrong place, as pip did later on), but I can't say I'm 100% certain of that, sorry. So it could be that the whole problem here was really that pip was putting kastore in the wrong place, or perhaps that pip thought that kastore was installed even though it wasn't because of some similar install-path screwup. I'm not sure; mysteries abound. Next time I embark upon this path I will try to remember to do it all in one terminal without clearing, so that I have a full record of everything I did and the corresponding output; I have a bad habit of clearing my terminal after each command I execute.

Maybe the real issue here is: why does pip treat kastore differently from the other packages in the first place, and install it in a weird location unless wheel is already installed? Is that the result of something in the configuration of kastore that could be fixed?

[jeromekelleher]

Looks like a classic "use python -m pip install x not pip install x" @bhaller. With macs you tend to get lots of python installations, and if you use pip install x then you're at the mercy of what happens to be in your PATH first. If you use python -m pip install x, then you know it's being installed into that version of Python. Basically, always use python -m pip, where "python" is the version of Python you're using.

It is confusing.

One other bit of advice is to stick with conda as much as possible if you are using conda as your Python installation (which lots of mac folks do).

[bhaller]

@jeromekelleher Interesting! I had not encountered that advice before (IIRC). This is puzzling, though, for several reasons. (1) I only have two python installs, one for 2.7 and one for 3.9; the weird install location for kastore was still a 3.9 location, just a different location than used for the other package installs. (2) I used pip install for all of the packages, consistently, and yet different installs got put into different locations. Anyhow, now I know to use python -m pip install, and that is very useful advice. It's kind of annoying that pip install is even a supported command, if it habitually puts things in the wrong place! If you want to close this issue feel free (although I do still think there is an interesting question here of why kastore's install worked differently from all other packages). If I encounter further problems the next time I am compelled to tilt at this particular windmill, I will open a new issue (and try to keep better records of what happened). Thanks for your help!

[bhaller]

(As for conda, I'm not a conda user at all, since I don't understand it and I have seen all sorts of warnings/difficulties with other people using conda that I don't understand; that's a can of worms I don't want to open...)

[petrelharp]

Wow, what the heck. Whenever I need to figure out where python stuff is installed to, I refer to https://xkcd.com/1987/

[bhaller]

Wow, what the heck. Whenever I need to figure out where python stuff is installed to, I refer to https://xkcd.com/1987/

Wow – as usual, it's like Randall and I are having some kind of mind-meld!

[jeromekelleher]

It seems more likely that there's something odd with your Python install than something specifically wrong with kastore here @bhaller, so I'm going to close. Please do reopen if you encounter the issue again though!