ValueError: cannot convert float NaN to integer ???

vblackwell commented 5 years ago

Describe the bug The error that is coming up is:

Traceback (most recent call last):
  File "kmapperex3.py", line 38, in <module>
    clusterer=sklearn.cluster.AffinityPropagation().fit(x))
  File "/Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/kmapper/kmapper.py", line 373, in map
    cluster_id = "cube{}_cluster{}".format(i, int(a[1]))
ValueError: cannot convert float NaN to integer

I am somewhat new to coding so I am not sure if in the kmapper.py file cluster_id needs to be initialized as a float instead of an integer but this what's popping up. This is also the first time I have reported a bug in github so please ask any questions if any of this does not make sense.

To Reproduce Steps to reproduce the behavior:

Go to line 38 in the python screenshot where it states "clusterer=sklearn.cluster.AffinityPropagation().fit(x))"
Then go to line 373 in the kmapper.py file from the KeplerMapper github where it states "cluster_id = "cube{}_cluster{}".format(i, int(a[1]))"
See error

Expected behavior I expected to have an html file saved that represented my data topologically.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

OS: Mojave
Browser: chrome
Version: 10.14

Additional context Add any other context about the problem here.

sauln commented 5 years ago

Hi @vblackwell, sorry you're having this problem. I'm not really sure what is going on. Could you please provide a little more information for me to help fix there issue.

Please include the script you use rather just a screenshot of the script. You can copy and paste it into a commend, surrounded by ``` marks.
Are you able to provide the data you use? It would help me to replicate the error.

vblackwell commented 5 years ago

Yes, here is the script and the data I am using:

'''Veronicas-MacBook-Pro-2:thesis_stuff veronicablackwell$ python kmapperex3.py /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/_libs/init.py:4: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .tslib import iNaT, NaT, Timestamp, Timedelta, OutOfBoundsDatetime /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/init.py:26: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import (hashtable as _hashtable, /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/dtypes/common.py:6: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import algos, lib /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/util/hashing.py:7: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import hashing, tslib /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/indexes/base.py:7: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import (lib, index as libindex, tslib as libts, /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/tseries/offsets.py:21: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 import pandas._libs.tslibs.offsets as liboffsets /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/ops.py:16: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import algos as libalgos, ops as libops /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/indexes/interval.py:32: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs.interval import ( /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/internals.py:14: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import internals as libinternals /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/sparse/array.py:33: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 import pandas._libs.sparse as splib /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/window.py:36: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 import pandas._libs.window as _window /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/groupby/groupby.py:68: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import (lib, reduction, /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/core/reshape/reshape.py:30: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import algos as _algos, reshape as _reshape /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/io/parsers.py:45: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 import pandas._libs.parsers as parsers /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/pandas/io/pytables.py:50: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from pandas._libs import algos, lib, writers as libwriters /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/sparse/csgraph/init.py:148: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._shortest_path import shortest_path, floyd_warshall, dijkstra,\ /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/sparse/csgraph/_validation.py:5: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._tools import csgraph_to_dense, csgraph_from_dense,\ /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/sparse/csgraph/init.py:150: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._traversal import breadth_first_order, depth_first_order, \ /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/sparse/csgraph/init.py:152: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._min_spanning_tree import minimum_spanning_tree /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/utils/init.py:9: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .murmurhash import murmurhash3_32 /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/special/init.py:531: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._ufuncs import /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/stats/distributions.py:34: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from . import vonmises_cython /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/stats/stats.py:252: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from ._rank import rankdata, tiecorrect /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/utils/extmath.py:24: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._logistic_sigmoid import _log_logistic_sigmoid /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/utils/extmath.py:26: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .sparsefuncs_fast import csr_row_norms /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/metrics/cluster/supervised.py:23: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .expected_mutual_info_fast import expected_mutual_information /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/spatial/init.py:90: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from .ckdtree import /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/spatial/init.py:91: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from .qhull import /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/metrics/pairwise.py:30: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .pairwise_fast import _chi2_kernel_fast, _sparse_manhattan /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/neighbors/init.py:6: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .ball_tree import BallTree /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/neighbors/init.py:7: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .kd_tree import KDTree /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/utils/random.py:10: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._random import sample_without_replacement /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/utils/graph.py:15: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .graph_shortest_path import graph_shortest_path # noqa /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/linear_model/base.py:35: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ..utils.seq_dataset import ArrayDataset, CSRDataset /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/linear_model/least_angle.py:23: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ..utils import arrayfuncs, as_float_array, check_X_y, deprecated /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/linear_model/coordinate_descent.py:29: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import cd_fast /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/linear_model/init.py:22: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .sgd_fast import Hinge, Log, ModifiedHuber, SquaredLoss, Huber /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/linear_model/sag.py:12: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from .sag_fast import sag /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/svm/base.py:8: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import libsvm, liblinear /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/svm/base.py:9: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import libsvm_sparse /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/decomposition/online_lda.py:28: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._online_lda import (mean_change, _dirichlet_expectation_1d, /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/isotonic.py:12: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._isotonic import _inplace_contiguous_isotonic_regression, _make_unique /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/manifold/t_sne.py:24: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _utils /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/manifold/t_sne.py:25: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _barnes_hut_tsne /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/tree/tree.py:40: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._criterion import Criterion /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/kmeans.py:35: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import _k_means /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/kmeans.py:36: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._k_means_elkan import k_means_elkan /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/hierarchical.py:23: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from . import hierarchical /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/dbscan.py:19: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._dbscan_inner import dbscan_inner /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release. from numpy.core.umath_tests import inner1d /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/ensemble/gradient_boosting.py:34: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88 from ._gradient_boosting import predict_stages /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/sparse/compressed.py:130: VisibleDeprecationWarning: rank is deprecated; use the ndim attribute or function instead. To find the rank of a matrix see numpy.linalg.matrix_rank. if np.rank(self.data) != 1 or np.rank(self.indices) != 1 or np.rank(self.indptr) != 1: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/scipy/stats/stats.py:1581: FutureWarning: Using a non-tuple sequence for multidimensional indexing is deprecated; use arr[tuple(seq)] instead of arr[seq]. In the future this will be interpreted as an array index, arr[np.array(seq)], which will result either in an error or a different result. return np.add.reduce(sorted[indexer] weights, axis=axis) / sumval ..Composing projection pipeline length 1: Projections: l2norm

Distance matrices: False

Scalers: MinMaxScaler(copy=True, feature_range=(0, 1))

..Projecting on data shaped (62, 5)

..Projecting data using: l2norm

..Scaling with: MinMaxScaler(copy=True, feature_range=(0, 1))

Mapping on data shaped (62, 5) using lens shaped (62, 2)

Minimal points in hypercube before clustering: 1 Creating 100 hypercubes. There are 0 points in cube_0 / 100 Cube_0 is empty.

There are 0 points in cube_1 / 100 Cube_1 is empty.

There are 0 points in cube_2 / 100 Cube_2 is empty.

There are 0 points in cube_3 / 100 Cube_3 is empty.

There are 0 points in cube_4 / 100 Cube_4 is empty.

There are 1 points in cube_5 / 100 /Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/sklearn/cluster/affinitypropagation.py:134: RuntimeWarning: invalid value encountered in subtract tmp -= np.sum(tmp, axis=0) Found 0 clusters in cube_5

Traceback (most recent call last): File "kmapperex3.py", line 38, in clusterer=sklearn.cluster.AffinityPropagation().fit(x)) File "/Users/veronicablackwell/Library/Python/2.7/lib/python/site-packages/kmapper/kmapper.py", line 373, in map cluster_id = "cube{}_cluster{}".format(i, int(a[1])) ValueError: cannot convert float NaN to integer''' alldata.xlsx

sauln commented 5 years ago

Hi @vblackwell, sorry for the delay in looking at this. I have been very busy with my own deadlines and now will be traveling for the next few weeks. I'll try to look deeper into this first thing in the new year. Thanks for your patience.

sauln commented 5 years ago

In your script, pass the pre-fit clusterer object:

graph = mapper.map(lens, data, coverer=km.Cover(overlap_perc=0.7),
                   clusterer=sklearn.cluster.AffinityPropagation())

notice how .fit(x) is not called here yet.

scikit-tda / kepler-mapper

ValueError: cannot convert float NaN to integer ??? #123