update CI conda envs + reactivate distributed numpy test

vepadulano / PyRDF

Python Library for doing ROOT RDataFrame analysis

https://pyrdf.readthedocs.io/en/latest/

9 stars 7 forks source link

update CI conda envs + reactivate distributed numpy test #93

Closed vepadulano closed 4 years ago

vepadulano commented 4 years ago

Conda environment files have been stripped down to a few required packages instead of listing the whole environment. Openjdk is now set generally to openjdk=8 to maintain compatibility with Spark that works with the Java 8 series. Before it used to be set to a specific Java 8 patch. ROOT is now updated to version 6.20 for both Python versions.

As a consequence, the changes needed to allow distribution of AsNumpy() operation are available and the corresponding test can be reactivated.

vepadulano commented 4 years ago

This is a followup to #78

brianventura commented 4 years ago

Dear Vincenzo,

I am not sure I should post that comment here, but I am so glad to hear that (for the AsNumpy() part at least !). Correct me if I am wrong then, so this means that when ROOT 6.20 will be available in SWAN (LCG97 ?) , one could use the AsNumpy() method with spark without issues ? Anyway I am glad to see people working on that issue :) Have a nice day !

Best regards, Brian Ventura

vepadulano commented 4 years ago

Dear Brian, Yes, with ROOT 6.20 on the LCG releases (unfortunately I don't know which one will be) will also come the possibility of using AsNumpy() operation on the Spark clusters through PyRDF. I have introduced some tests in the package and tested it myself with a few toy analyses, I hope that we will have more robust example to put the feature to the test. Wish you a good day, Vincenzo