frazer-lab / cluster

Repo for cluster issues.
1 stars 0 forks source link

flh2 python/2.7/pysam/0.8.3 module no longer working #219

Closed djakubosky closed 7 years ago

djakubosky commented 7 years ago

module adding python/2.7/pysam/0.8.3 works on flh1 but not flh2 after CentOS6 update to CentOS7, is there something I should be doing differently?

tatarsky commented 7 years ago

Will have to look at it.

tatarsky commented 7 years ago

OK. So basically that module isn't going to work anymore because it was based on the python27 RPMS that only existed in CentOS 6.

I am looking at the way the newer version of that RPM works in CentOS 7.

djakubosky commented 7 years ago

Thanks! This is an important library for many things I do. Appreciate the help

tatarsky commented 7 years ago

Understood. Its a weird method (the installation). Does this change the behavior? This is in test ONLY on fl-hn2 and note the maintainers have updated the pysam version.

$ module add python/2.7/pysam/0.11.2.2
$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam
>>> 
djakubosky commented 7 years ago

this actually doesn't seem to work djakubosky@fl-hn2:/frazer01/projects/CARDIPS/analysis/cardips-cnv-analysis$ module add python/2.7/pysam/0.11.2.2 ModuleCmd_Load.c(213):ERROR:105: Unable to locate a modulefile for 'python/2.7/pysam/0.11.2.2'

but a problem is it actually has to be 0.8.3 to work with certain things.

djakubosky commented 7 years ago

many packages are very sensitive to pysam version, as I'm sure you've come to find out

tatarsky commented 7 years ago

You have to login/login to get the change in Modulepath I just did. Sorry.

But if 0.8.3 is required I will look into it more shortly.

djakubosky commented 7 years ago

I actually don't know what that means (login/login)

tatarsky commented 7 years ago

typo: logout/login aka exit your shell to fl-hn2 and log back in. But it won't change the 0.8.3 part. Just provide the newer one for now.

djakubosky commented 7 years ago

I can't exactly remember why, but that version is needed for something. I'll look back at my notes, thanks for the help!

tatarsky commented 7 years ago

Yeah, I've seen similar. I'm working on a compat module method for C7. Will be a bit but it should be doable.

tatarsky commented 7 years ago

If you happen to have a short test I can use I'm trying two paths.

  1. See if I can simply use the old python data (the module is just a PYTHONPATH setting)
fl-hn2 [~]$ module add python/2.7/pysam/0.8.3
fl-hn2 [~]$ python
Python 2.7.5 (default, Nov 20 2015, 02:00:19) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pysam
>>> 

I dimly recall some pysam fragments to validate but if you have one handy.

  1. Building a fresh RPM using 0.8.3 which is currently hitting Cython errors so that may be a bit.

I have no idea if that works

tatarsky commented 7 years ago

I will be out for a bit now but I'll keep working on it when I return. Any short validation test so I don't have to bug you would be appreciated.

tatarsky commented 7 years ago

Note to myself: my Cython might need an update.

djakubosky commented 7 years ago

Sorry. Was out of office, I'll try it out thanks!

On Thu, Oct 19, 2017 at 4:19 PM tatarsky notifications@github.com wrote:

Note to myself: my Cython might need an update.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/frazer-lab/cluster/issues/219#issuecomment-338064162, or mute the thread https://github.com/notifications/unsubscribe-auth/AOUvtiSqjasW5PaOeB4ZOLYCEO5BfAq2ks5st9jwgaJpZM4P_8GT .

-- ____

David Jakubosky Biomedical Sciences Graduate Program Laboratory of Kelly A. Frazer, PhD Institute for Genomic Medicine, University of California at San Diego


tatarsky commented 7 years ago

Just advise when you can if the first method works and I will propagate it to the nodes. Its just on fl-hn2 right now. I will look at the generic solutions for multiple pysam versions with the same python again.

tatarsky commented 7 years ago

I have also made some progress with a pure build of pysam 0.8.3 on CentOS 7 compared to the copy method. So when you have time let me know if the "copy" of the old python pysam doesn't work and I can have you test this method.

I believe both methods would work but I'm trying to keep things as exactly "the same" for you as possible given the ways of pysam.

tatarsky commented 7 years ago

Again just to be clear: all this is ONLY on fl-hn2 for now. Not the CentOS7 nodes so test from the head node please.

djakubosky commented 7 years ago

seems that your copy method is functional for me, unsure if it will break when I do other tests but for now it is working.

tatarsky commented 7 years ago

OK. For now then I will propagate the method to the nodes via puppet. I will update this when that is done and the module works the same on all the nodes.

But once the dust clears I would like to try the fresh build when not under time pressure. Just so we don't have too many unknown methods.

tatarsky commented 7 years ago

Oh, and as a plus I believe my newer method would allow us to handle arbitrary pysam versions in a clean module way. I spent some time looking at the RPMs method and understand what they are doing.

tatarsky commented 7 years ago

OK. I believe the nodes are now consistent as well with some tests by me to verify

module add python/2.7/pysam/0.8.3

Results in the proper matching item on them. Please advise (and include node name) if you have issues!

tatarsky commented 7 years ago

And I believe if you did want to validate a newer pysam this also should work

module add python/2.7/pysam/0.11.2.2

No requirement you do so but preparing to support variant pysams.

tatarsky commented 7 years ago

Believed resolved and we also have a way to handle multiple pysams going forward. Re-open if still concerns.