search

comprna / METEORE

Automatic DNA methylation detection from nanopore tools and their consensus model
MIT License
72 stars 18 forks source link

combination_model_train.py doesn't work ! #12

Closed DelphIONe closed 3 years ago

DelphIONe commented 3 years ago

The requirement.txt with pip install is ok but the script is not ok

python combination_model_train.py -d example_results/deepsignal_results/example_deepsignal-perRead-score.tsv -n example_results/nanopolish_results/example_nanopolish-perRead-score.tsv -g example_results/guppy_results/example_guppy-perRead-score.tsv -c 3 -o output_models /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:30: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations method='lar', copy_X=True, eps=np.finfo(np.float).eps, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:167: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations method='lar', copy_X=True, eps=np.finfo(np.float).eps, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:284: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations eps=np.finfo(np.float).eps, copy_Gram=True, verbose=0, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:862: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations eps=np.finfo(np.float).eps, copy_X=True, fit_path=True, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:1101: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations eps=np.finfo(np.float).eps, copy_X=True, fit_path=True, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:1127: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations eps=np.finfo(np.float).eps, positive=False): /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:1362: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations max_n_alphas=1000, n_jobs=None, eps=np.finfo(np.float).eps, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:1602: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations max_n_alphas=1000, n_jobs=None, eps=np.finfo(np.float).eps, /home/miniconda3/lib/python3.8/site-packages/sklearn/linear_model/least_angle.py:1738: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations eps=np.finfo(np.float).eps, copy_X=True, positive=False): /home/miniconda3/lib/python3.8/site-packages/sklearn/decomposition/online_lda.py:29: DeprecationWarning: np.float is a deprecated alias for the builtin float. To silence this warning, use float by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use np.float64 here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations EPS = np.finfo(np.float).eps /home/miniconda3/lib/python3.8/site-packages/sklearn/ensemble/gradientboosting.py:32: DeprecationWarning: np.bool is a deprecated alias for the builtin bool. To silence this warning, use bool by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.boolhere. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations from ._gradient_boosting import predict_stages /home/miniconda3/lib/python3.8/site-packages/sklearn/ensemble/gradient_boosting.py:32: DeprecationWarning:np.boolis a deprecated alias for the builtinbool. To silence this warning, useboolby itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, usenp.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations from ._gradient_boosting import predict_stages combination_model_train.py:54: SettingWithCopyWarning: A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy data_file["Pos"][mask]=data_file["Pos"][mask]-1 Traceback (most recent call last): File "combination_model_train.py", line 96, in X,y=combine_methods(val) File "combination_model_train.py", line 64, in combine_methods combine_file=reduce(lambda left,right: pd.merge(left, right, how='inner',on=["ID","Pos","Label"]), dfs) File "combination_model_train.py", line 64, in combine_file=reduce(lambda left,right: pd.merge(left, right, how='inner',on=["ID","Pos","Label"]), dfs) File "/home/miniconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 74, in merge op = _MergeOperation( File "/home/miniconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 668, in init ) = self._get_merge_keys() File "/home/miniconda3/lib/python3.8/site-packages/pandas/core/reshape/merge.py", line 1033, in _get_merge_keys right_keys.append(right._get_label_or_level_values(rk)) File "/home/miniconda3/lib/python3.8/site-packages/pandas/core/generic.py", line 1684, in _get_label_or_level_values raise KeyError(key) KeyError: 'Label'

The output folder is not created. Any ideas please ? I would like really to test this pipeline on real data and I stay blocked with your "toy" example :-( With the other script, combine_prediction.py is the same thing.

Thanks for your time,

zakayuen commented 3 years ago

Hi,

Since I have modified the per-read outputs from each pipeline, I need to make changes to the combination_model_train.py and combination_model_prediction.py which I haven't had time to modify yet. I'll update these scripts ASAP and let you know.

Thank you for your patience.

Zaka

DelphIONe commented 3 years ago

Hi,

Have you got any idea of delay to do this modification ? Thanks,

Delph

zakayuen commented 3 years ago

Hi @DelphIONe

Sorry for the delay. combination_model_prediction.py and prediction_to_mod_frequency.py should be working now. The Chr information has been added to the per-read and per-site output.

Cheers, Zaka

DelphIONe commented 3 years ago

Thanks Zaka but I am sorry I have this error (base) X@ngs-05:~/softs/METEORE$ python combination_model_prediction.py -i model_content_all.tsv -m optimized -o cas9_deepsignal_guppy_megalodon_nanopolish_tombo_optimized_perRead.tsv Traceback (most recent call last): File "combination_model_prediction.py", line 10, in import joblib File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/init.py", line 119, in from .parallel import Parallel File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/parallel.py", line 28, in from ._parallel_backends import (FallbackToBackend, MultiprocessingBackend, File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/_parallel_backends.py", line 22, in from .executor import get_memmapping_executor File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/executor.py", line 14, in from .externals.loky.reusable_executor import get_reusable_executor File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/externals/loky/init.py", line 12, in from .backend.reduction import set_loky_pickler File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/externals/loky/backend/reduction.py", line 125, in from joblib.externals import cloudpickle # noqa: F401 File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/externals/cloudpickle/init.py", line 3, in from .cloudpickle import * File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 152, in _cell_set_template_code = _make_cell_set_template_code() File "/home/X/miniconda3/lib/python3.8/site-packages/joblib/externals/cloudpickle/cloudpickle.py", line 133, in _make_cell_set_template_code return types.CodeType( TypeError: an integer is required (got type bytes)

Do you think I have need a particular (previous) conda environment ?

Johnsonzcode commented 3 years ago

Same error, fixed by https://blog.csdn.net/congcong_i/article/details/116500921