Conflicting packages sklearn

[federiconuta]

Hi all,

so I have the following issue when importing some of the packages:

cannot import name 'parse_version' from 'sklearn.utils.fixes' (/Users/federiconutarelli/anaconda3/lib/python3.7/site-packages/sklearn/utils/fixes.py)

E.g. this happened with

from econml.dml import LinearDMLCateEstimator

I also tried to pip install -U sklearn but a part from "requirement already satisfied" the issue seems not to be solved. Is there a specific reason fr that? Any conflicting packages? Or it is just a problem of my environment?

Thank you,

Federico

[vsyrgkanis]

If you are using our econml 0.8.1 then you need to install 0.23.2 sklearn.

you can either: pip install sklearn==0.23.2 To downgrade sklearn

or upgrade to our pre-release 0.9 pip install -U —pre econml Which fixes the incompatibility with the new sklearn

[federiconuta]

@vsyrgkanis thank you for thee reply. Actually condo listing the packages in my env it turns out that I have:

scikit-learn 0.23.2 pypi_0 pypi

pip seems not to recognise sklearn, which, however appears (in version 0, so it is ass if it does not) in my condo list:

sklearn 0.0 pypi_0 pypi

The error is actually still present.

Does the second solution you are proposing generate other conflicts maybe if I use refit_final or other prototypes functionalities?

[vsyrgkanis]

If you’ve upgraded to our 0.9 pre release then most prob you have to upgrade sklearn to 0.24

So try conda installing sklearn 0.24

Sent from my iPhone

On Jan 22, 2021, at 9:28 AM, federiconuta notifications@github.com wrote:

 @vsyrgkanis thank you for thee reply. Actually condo listing the packages in my env it turns out that I have:

scikit-learn 0.23.2 pypi_0 pypi

pip seems not to recognise sklearn, which, however appears (in version 0, so it is ass if it does not) in my condo list:

sklearn 0.0 pypi_0 pypi

The error is actually still present.

Does the second solution you are proposing generate other conflicts maybe if I use refit_final or other prototypes functionalities?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[vsyrgkanis]

I tried our pre-release 0.9b with scikit-learn==0.23.2 and it works with that too. So that might not be the issue

Also correction: you should be doing: pip install -U scikit-learn

not pip install -U sklearn

[vsyrgkanis]

Also another thing: in our 0.9b pre-release, the name LinearDMLCateEstimator has been deprecated as promised in our warnings.

So you need to use LinearDML instead.

[federiconuta]

pip install -U scikit-learn

yes I did it but it raises an error as if it cannot parse properly some dependencies:

ParsingError: Source contains parsing errors: 'setup.cfg'
    [line 49]: '<<<<<<< HEAD\n'
    [line 50]: '=======\n'
    [line 51]: '    graphviz\n'
    [line 53]: "    matplotlib; python_version > '3.5'\n"
    [line 54]: '    pandas\n'
    [line 55]: '>>>>>>> 3606b0bcc7779b78e6df8991dbcd7b72ac3046ef\n'

I am trying to condo installing scikit-learn and it seemed to work fine but is freezes here:

Preparing transaction: done
Verifying transaction: done
Executing transaction: - WARNING conda.gateways.disk.delete:unlink_or_rename_to_trash(140): Could not remove or rename /Users/federiconutarelli/anaconda3/lib/python3.7/site-packages/sklearn/__check_build/__init__.py.  Please remove this file manually (you may need to reboot to free file handles)

[kbattocchi]

That's weird; maybe you could follow the suggestion and reboot your PC and see if you can then uninstall and reinstall successfully?

[federiconuta]

@kbattocchi I tried a reboot but actually nothing happened. Maybe creating a brand new env could be a solution?

[federiconuta]

@kbattocchi actually I saw that there was a problem with conda (conflicts with macOS Catalina). So I started a brand new env (not conda). However when I pip install -U —pre econml the following error appears: Invalid requirement: '—pre'

am I doing something wrong here?

[kbattocchi]

Looks like maybe two hyphen characters (--) have been turned into some other punctuation, like an em-dash? Try to make sure the characters are exactly pip install -U --pre econml.

[federiconuta]

@kbattocchi thanks for the reply. Basically I think the issue lies on the fact that I have a brand new virtual environment. So I have (maybe) to install back econml. To have also the prototypes (with also refit_final and the bootstrap for it) I have just to git clone the latest version right? Just when importing LinearDML I incur in the following error:

No module named 'econml.grf._criterion'

The problem seems to lie in the init.py of grf when importing _criterion.pyx. I also tried to use cython to deal with the .pyx file by doing:

import pyximport
#pyximport.install()

before importing _criterion but it does not seem to work the error being: Building module econml.grf._criterion failed: ["distutils.errors.CompileError: command 'xcrun' failed with exit status 1\n"]

I understood the reason why pip install -U —pre econml did not work and I would like to share for Mac users: yet you do not have to copy-paste it on terminal rather you have to type it word by word on the terminal. I don't know why it is so, but t seems to be a quite common issue for Mac users. To provide you with further details, however, I incurred in the following error:

The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME to enable CUDA

Not being a developer I don't know what it precisely means but it seems that if you do not have a NVIDIA GPU CUDA compatible, this does not work.

Also can you please provide me an example (or a link) of usage of bootstrap inference on refit_final?

Thank you very much

[federiconuta]

@kbattocchi @vsyrgkanis I should have solved by reinstalling conda from scratch. I then made pip install econml and it worked fine. However the problem is that I have to both refit the final stage and then apply bootstrap to DMLIV estimator. This raises an error:

cate = DMLIV(model_Y_X(), model_T_X(), model_T_XZ(),
             dmliv_model_effect(), dmliv_featurizer(),
             n_splits= 20)
cate.fit(Y, T[:,0], XW, Z, store_final=True)

Output:

got an unexpected keyword argument 'store_final'

Can you please help me out on this? Further, where can I find an example of bootstrap application to DMLIV?

[kbattocchi]

@federiconuta In case you haven't seen it, our final 0.9.0 release is now out, so you should be able to run pip install econml -U and get the code with refitting. The correct syntax for fit is cate.fit(Y, ..., cache_values) see the documentation