Open baligoyem opened 2 years ago
I'm not sure but it could be something about the data you are passing again. Is there a way for you to come up with a trivial data set that causes problems so we can test it with a self-contained test?
Hi @baligoyem,
I see two errors in your stacktrace : "ValueError: cannot convert float NaN to integer" and "ValueError: need at least one array to concatenate". They seem to be related to your training set. If you can provide us a google colab notebook or any other sharable notebook, we could better investigate it.
Another option would be to run the lightgbm notebook from this repo and to check if you have the same issue. Maybe you could understand from where is the issue.
Hi @tlapusan and @parrt ,
Sorry for my late reply. I will share the data and my notebook soon. First thing I need to do is masking the data and then waiting for getting the same error again :)
You could also run the lightgbm notebook from this repo to check if you have the same issue or not
Actually, I've already used this repo. Thanks for your suggestion.
Hi @parrt @tlapusan
I meet same problem!
File "D:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\matplotlib\path.py", line 947, in arc
n = int(2 ** np.ceil((eta2 - eta1) / halfpi))
ValueError: cannot convert float NaN to integer
I debug it, in line 1097 in trees.py _class_leaf_viz
counts = node.class_counts()
the counts is [0, 0]
, so get the error.
my data is too big, sorry i can not upload it. but you can test it by this.
counts = node.class_counts()
counts[:] = 0
Thank you very much!
Thanks @wukan1986 could you try cutting your data down? Also please upgrade to dtreeviz 2.0. thanks!
Hi @parrt
I install dtreeviz yesterday, is 2.0 I cut the data from 1GB to 44KB. delete a lot of rows and features.
please download code and data from this train.zip
d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\matplotlib\axes\_axes.py:3056: RuntimeWarning: invalid value encountered in divide
ValueError Traceback (most recent call last)
d:\GitHub\my_quant\tests\train.py in line 2
79 # %%
----> 80 v = viz_model.view(fancy=True)
81 v.show()
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\dtreeviz\trees.py:555, in DTreeVizAPI.view(self, precision, orientation, instance_orientation, show_root_edge_labels, show_node_labels, show_just_path, fancy, histtype, highlight_path, x, max_X_features_LR, max_X_features_TD, depth_range_to_display, label_fontsize, ticks_fontsize, fontname, title, title_fontsize, colors, scale)
553 continue
554 if self.shadow_tree.is_classifier():
--> 555 _class_leaf_viz(node, colors=color_values,
556 filename=f"{tmp}/leaf{node.id}_{os.getpid()}.svg",
557 graph_colors=colors,
558 fontname=fontname)
559 leaves.append(class_leaf_node(node))
560 else:
561 # for now, always gen leaf
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\dtreeviz\trees.py:1099, in _class_leaf_viz(node, colors, filename, graph_colors, fontname)
1097 counts = node.class_counts()
1098 prediction = node.prediction_name()
-> 1099 _draw_piechart(counts, size=size, colors=colors, filename=filename, label=f"n={nsamples}\n{prediction}",
1100 graph_colors=graph_colors, fontname=fontname)
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\dtreeviz\trees.py:1318, in _draw_piechart(counts, size, colors, filename, label, fontname, graph_colors)
...
--> 947 n = int(2 ** np.ceil((eta2 - eta1) / halfpi))
948 if n < 1:
949 raise ValueError("n must be >= 1 or None")
ValueError: cannot convert float NaN to integer
ValueError Traceback (most recent call last)
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\IPython\core\formatters.py:338, in BaseFormatter.__call__(self, obj)
336 pass
337 else:
--> 338 return printer(obj)
339 # Finally look for special method names
340 method = get_real_method(obj, self.print_method)
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\IPython\core\pylabtools.py:152, in print_figure(fig, fmt, bbox_inches, base64, **kwargs)
149 from matplotlib.backend_bases import FigureCanvasBase
150 FigureCanvasBase(fig)
--> 152 fig.canvas.print_figure(bytes_io, **kw)
153 data = bytes_io.getvalue()
154 if fmt == 'svg':
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\matplotlib\backend_bases.py:2299, in FigureCanvasBase.print_figure(self, filename, dpi, facecolor, edgecolor, orientation, format, bbox_inches, pad_inches, bbox_extra_artists, backend, **kwargs)
2297 if bbox_inches:
2298 if bbox_inches == "tight":
-> 2299 bbox_inches = self.figure.get_tightbbox(
2300 renderer, bbox_extra_artists=bbox_extra_artists)
2301 if pad_inches is None:
2302 pad_inches = rcParams['savefig.pad_inches']
File d:\Users\Kan\miniconda3\envs\torch1\lib\site-packages\matplotlib\figure.py:1684, in FigureBase.get_tightbbox(self, renderer, bbox_extra_artists)
...
633 return Bbox([xys.min(axis=0), xys.max(axis=0)])
File <__array_function__ internals>:180, in concatenate(*args, **kwargs)
ValueError: need at least one array to concatenate
<Figure size 37x37 with 1 Axes>
Thank you very much
Hi @wukan1986, thanks for sending the code and data. It helps for debugging.
I tried you code and it works for me
dtreeviz 2.0 lightgbm==3.3.3
I see the error is generated somewhere inside matplotlib ... I'm having matplotlib==3.4.2
my matplotlib is 3.5.3
I will try it in google colab notebook
Hi @tlapusan @parrt
I update matplotlib to 3.6.2, get same error
https://colab.research.google.com/drive/1EmihQj9WpkY3dLHH9CSq5h7OGtlDQ0dk?usp=sharing
I wonder if this is an issue of a variety of different package versions on colab vs jupyter lab.
@tlapusan his version is lightgbm-3.3.3; mine is 3.2.1.
Running his .py i get:
beast:~/Downloads/train $ python train.py
Traceback (most recent call last):
File "/Users/parrt/Downloads/train/train.py", line 57, in <module>
lgb.log_evaluation(10),
AttributeError: module 'lightgbm' has no attribute 'log_evaluation'
damn and upgrading doesn't work because I get an M1 silicon error on my mac.
@tlapusan his version is lightgbm-3.3.3; mine is 3.2.1.
Running his .py i get:
beast:~/Downloads/train $ python train.py Traceback (most recent call last): File "/Users/parrt/Downloads/train/train.py", line 57, in <module> lgb.log_evaluation(10), AttributeError: module 'lightgbm' has no attribute 'log_evaluation'
damn and upgrading doesn't work because I get an M1 silicon error on my mac.
yes, I needed also to upgrade the lightgbm to solve that error.
@tlapusan his version is lightgbm-3.3.3; mine is 3.2.1.
Running his .py i get:
beast:~/Downloads/train $ python train.py Traceback (most recent call last): File "/Users/parrt/Downloads/train/train.py", line 57, in <module> lgb.log_evaluation(10), AttributeError: module 'lightgbm' has no attribute 'log_evaluation'
damn and upgrading doesn't work because I get an M1 silicon error on my mac.
log_evaluation can remove, it just print log every 10 rounds
When I try to plot any trees of my lgbm model, sometimes an error occurs and it doesn't show the tree. For instance, I can't see the tree, which has "0" index number. But when I type the index as 1 for this model, there is no problem and it shows the tree which has "1" index number.
Moreover, "try-except pass" structure doesn't work in this case and I can't ignore the error message below.