search

scottrogowski / code2flow

Pretty good call graphs for dynamic languages
MIT License
4k stars 295 forks source link

subprocess.CalledProcessError: Command '['dot', '-Tpng', 'out.gv']' returned non-zero exit status 3221225477 #29

Closed sgbaird closed 3 years ago

sgbaird commented 3 years ago 
(base) PS C:\Users\sterg\anaconda3\envs\elm2d-crabnet\Lib\site-packages\sklearn\preprocessing> code2flow _data.py
Code2Flow: Found 1 files from sources argument.
Code2Flow: Implicitly detected language as 'py'.
Code2Flow: Processing 1 source file(s).
Code2Flow:   _data.py
Code2Flow: Found groups ['File: _data', 'Class: MinMaxScaler', 'Class: StandardScaler', 'Class: MaxAbsScaler', 'Class: RobustScaler', 'Class: PolynomialFeatures', 'Class: Normalizer', 'Class: Binarizer', 'Class: KernelCenterer', 'Class: QuantileTransformer', 'Class: PowerTransformer'].
Code2Flow: Found nodes ['(global)', 'Binarizer.__init__', 'Binarizer._more_tags', 'Binarizer.fit', 'Binarizer.transform', 'KernelCenterer.__init__', 'KernelCenterer._more_tags', 'KernelCenterer._pairwise', 'KernelCenterer.fit', 'KernelCenterer.transform', 'MaxAbsScaler.__init__', 'MaxAbsScaler._more_tags', 'MaxAbsScaler._reset', 'MaxAbsScaler.fit', 'MaxAbsScaler.inverse_transform', 'MaxAbsScaler.partial_fit', 'MaxAbsScaler.transform', 'MinMaxScaler.__init__', 'MinMaxScaler._more_tags', 'MinMaxScaler._reset', 'MinMaxScaler.fit', 'MinMaxScaler.inverse_transform', 'MinMaxScaler.partial_fit', 'MinMaxScaler.transform', 'Normalizer.__init__', 'Normalizer._more_tags', 'Normalizer.fit', 'Normalizer.transform', 'PolynomialFeatures.__init__', 'PolynomialFeatures._combinations', 'PolynomialFeatures.fit', 'PolynomialFeatures.get_feature_names', 'PolynomialFeatures.powers_', 'PolynomialFeatures.transform', 'PowerTransformer.__init__', 'PowerTransformer._box_cox_inverse_tranform', 'PowerTransformer._box_cox_optimize', 'PowerTransformer._check_input', 'PowerTransformer._fit', 'PowerTransformer._more_tags', 'PowerTransformer._yeo_johnson_inverse_transform', 'PowerTransformer._yeo_johnson_optimize', 'PowerTransformer._yeo_johnson_transform', 'PowerTransformer.fit', 'PowerTransformer.fit_transform', 'PowerTransformer.inverse_transform', 'PowerTransformer.transform', 'QuantileTransformer.__init__', 'QuantileTransformer._check_inputs', 'QuantileTransformer._dense_fit', 'QuantileTransformer._more_tags', 'QuantileTransformer._sparse_fit', 'QuantileTransformer._transform', 'QuantileTransformer._transform_col', 'QuantileTransformer.fit', 'QuantileTransformer.inverse_transform', 'QuantileTransformer.transform', 'RobustScaler.__init__', 'RobustScaler._more_tags', 'RobustScaler.fit', 'RobustScaler.inverse_transform', 'RobustScaler.transform', 'StandardScaler.__init__', 'StandardScaler._more_tags', 'StandardScaler._reset', 'StandardScaler.fit', 'StandardScaler.inverse_transform', 'StandardScaler.partial_fit', 'StandardScaler.transform', '_handle_zeros_in_scale', 'add_dummy_feature', 'binarize', 'maxabs_scale', 'minmax_scale', 'normalize', 'power_transform', 'quantile_transform', 'robust_scale', 'scale'].
Code2Flow: Found calls ['MaxAbsScaler()', 'MinMaxScaler()', 'NotImplementedError()', 'PowerTransformer()', 'QuantileTransformer()', 'RobustScaler()', 'StandardScaler()', 'TypeError()', 'UNKNOWN_VAR.astype()', 'UNKNOWN_VAR.format()', 'UNKNOWN_VAR.join()', 'UNKNOWN_VAR.sum()', 'UNKNOWN_VAR.tocsc()', 'UNKNOWN_VAR.tocsr()', 'ValueError()', 'X.copy()', 'X.eliminate_zeros()', 'X.multiply()', 'X.ravel()', 'X.reshape()', 'X.tocoo()', 'X.tocsc()', 'X.tocsr()', '_check_sample_weight()', '_csr_polynomial_expansion()', '_handle_zeros_in_scale()', '_incremental_mean_and_var()', '_incremental_weighted_mean_and_var()', 'abs()', 'add_dummy_feature()', 'binarize()', 'chain.from_iterable()', 'check_array()', 'check_is_fitted()', 'check_random_state()', 'col.take()', 'columns.append()', 'comb()', 'enumerate()', 'feature_names.append()', 'hasattr()', 'incr_mean_variance_axis()', 'index.append()', 'inplace_column_scale()', 'inplace_csr_row_normalize_l1()', 'inplace_csr_row_normalize_l2()', 'int()', 'inv_fun()', 'isinstance()', 'klass()', 'len()', 'list()', 'max()', 'mean_variance_axis()', 'min()', 'min_max_axis()', 'n.fit_transform()', 'new_index.append()', 'normalize()', 'norms.repeat()', 'np.abs()', 'np.accumulate()', 'np.allclose()', 'np.any()', 'np.arange()', 'np.array()', 'np.asarray()', 'np.bincount()', 'np.catch_warnings()', 'np.clip()', 'np.concatenate()', 'np.diff()', 'np.empty()', 'np.errstate()', 'np.exp()', 'np.filterwarnings()', 'np.full()', 'np.hstack()', 'np.interp()', 'np.isnan()', 'np.isscalar()', 'np.linspace()', 'np.log()', 'np.log1p()', 'np.logical_not()', 'np.max()', 'np.maximum()', 'np.minimum()', 'np.multiply()', 'np.nanmax()', 'np.nanmean()', 'np.nanmedian()', 'np.nanmin()', 'np.nanpercentile()', 'np.nanstd()', 'np.ones()', 'np.power()', 'np.ptp()', 'np.repeat()', 'np.rollaxis()', 'np.sign()', 'np.size()', 'np.spacing()', 'np.sqrt()', 'np.sum()', 'np.transpose()', 'np.vstack()', 'np.where()', 'np.zeros()', 'np.zeros_like()', 'optim_function()', 'optimize.brent()', 'pt.fit_transform()', 'quantiles.append()', 'random_state.choice()', 'range()', 'row_norms()', 's.fit_transform()', 'scale.copy()', 'self._check_input()', 'self._check_inputs()', 'self._combinations()', 'self._dense_fit()', 'self._fit()', 'self._reset()', 'self._sparse_fit()', 'self._transform()', 'self._transform_col()', 'self._validate_data()', 'self._yeo_johnson_transform()', 'self.append()', 'self.astype()', 'self.fit()', 'self.fit_transform()', 'self.inverse_transform()', 'self.partial_fit()', 'self.sum()', 'self.transform()', 'slice()', 'sparse.coo_matrix()', 'sparse.csc_matrix()', 'sparse.hstack()', 'sparse.issparse()', 'sparse.isspmatrix()', 'sparse.isspmatrix_coo()', 'sparse.isspmatrix_csc()', 'sparse.isspmatrix_csr()', 'sparse_constructor()', 'stats.boxcox()', 'stats.cdf()', 'stats.ppf()', 'str()', 'sum()', 'to_stack.append()', 'transform_function()', 'warnings.warn()', 'x_trans.var()', 'zip()'].
Code2Flow: Found variables ['BaseEstimator->UNKNOWN_MODULE', 'FLOAT_DTYPES->UNKNOWN_MODULE', 'K-><Call owner_token=self token=_validate_data>', 'OneHotEncoder->UNKNOWN_MODULE', 'TransformerMixin->UNKNOWN_MODULE', 'X-><Call owner_token=None token=check_array>', 'X-><Call owner_token=X token=copy>', 'X-><Call owner_token=X token=ravel>', 'X-><Call owner_token=X token=reshape>', 'X-><Call owner_token=X token=tocsr>', 'X-><Call owner_token=np token=asarray>', 'X-><Call owner_token=s token=fit_transform>', 'X-><Call owner_token=self token=_check_input>', 'X-><Call owner_token=self token=_check_inputs>', 'X-><Call owner_token=self token=_validate_data>', 'X-><Call owner_token=self token=fit_transform>', 'X-><Call owner_token=self token=inverse_transform>', 'X-><Call owner_token=self token=transform>', 'XP-><Call owner_token=UNKNOWN_VAR token=tocsc>', 'XP-><Call owner_token=np token=empty>', 'XP-><Call owner_token=sparse token=hstack>', 'X_col-><Call owner_token=np token=clip>', 'X_col-><Call owner_token=stats token=cdf>', 'X_col-><Call owner_token=stats token=ppf>', 'Xp_next-><Call owner_token=None token=_csr_polynomial_expansion>', 'Xr-><Call owner_token=np token=rollaxis>', '_check_sample_weight->UNKNOWN_MODULE', '_csr_polynomial_expansion->UNKNOWN_MODULE', '_deprecate_positional_args->UNKNOWN_MODULE', '_incremental_mean_and_var->UNKNOWN_MODULE', '_incremental_weighted_mean_and_var->UNKNOWN_MODULE', 'bias-><Call owner_token=sparse token=csc_matrix>', 'boxcox->UNKNOWN_MODULE', 'chain->UNKNOWN_MODULE', 'check_array->UNKNOWN_MODULE', 'check_is_fitted->UNKNOWN_MODULE', 'check_random_state->UNKNOWN_MODULE', 'clip_max-><Call owner_token=stats token=ppf>', 'clip_min-><Call owner_token=stats token=ppf>', 'col-><Call owner_token=col token=take>', 'col-><Call owner_token=np token=concatenate>', 'column_data-><Call owner_token=np token=zeros>', 'column_slice-><Call owner_token=None token=slice>', 'combinations-><Call owner_token=self token=_combinations>', 'combinations->UNKNOWN_MODULE', 'combinations_w_r->UNKNOWN_MODULE', 'data-><Call owner_token=np token=concatenate>', 'data_max-><Call owner_token=np token=maximum>', 'data_max-><Call owner_token=np token=nanmax>', 'data_min-><Call owner_token=np token=minimum>', 'data_min-><Call owner_token=np token=nanmin>', 'deprecated->UNKNOWN_MODULE', 'incr_mean_variance_axis->UNKNOWN_MODULE', 'index-><Call owner_token=None token=list>', 'indices-><Call owner_token=np token=concatenate>', 'indptr-><Call owner_token=np token=concatenate>', 'inplace_column_scale->UNKNOWN_MODULE', 'inplace_csr_row_normalize_l1->UNKNOWN_MODULE', 'inplace_csr_row_normalize_l2->UNKNOWN_MODULE', 'max_abs-><Call owner_token=np token=maximum>', 'max_abs-><Call owner_token=np token=nanmax>', 'mean_-><Call owner_token=np token=nanmean>', 'mean_1-><Call owner_token=np token=nanmean>', 'mean_2-><Call owner_token=np token=nanmean>', 'mean_variance_axis->UNKNOWN_MODULE', 'min_max_axis->UNKNOWN_MODULE', 'n->QuantileTransformer', 'name-><Call owner_token=UNKNOWN_VAR token=join>', 'norms-><Call owner_token=None token=_handle_zeros_in_scale>', 'norms-><Call owner_token=None token=row_norms>', 'norms-><Call owner_token=UNKNOWN_VAR token=sum>', 'norms-><Call owner_token=np token=max>', 'norms-><Call owner_token=np token=maximum>', 'norms_elementwise-><Call owner_token=norms token=repeat>', 'not_cond-><Call owner_token=np token=logical_not>', 'np->UNKNOWN_MODULE', 'optimize->UNKNOWN_MODULE', 'out-><Call owner_token=np token=zeros_like>', 'out_col-><Call owner_token=X token=multiply>', 'pt->PowerTransformer', 'quantiles-><Call owner_token=np token=transpose>', 'rng-><Call owner_token=None token=check_random_state>', 'row-><Call owner_token=np token=concatenate>', 'row_norms->UNKNOWN_MODULE', 's->MaxAbsScaler', 's->MinMaxScaler', 's->RobustScaler', 'sample_weight-><Call owner_token=None token=_check_sample_weight>', 'scale-><Call owner_token=scale token=copy>', 'scale_-><Call owner_token=None token=_handle_zeros_in_scale>', 'scale_-><Call owner_token=np token=nanstd>', 'self->Binarizer', 'self->KernelCenterer', 'self->MaxAbsScaler', 'self->MinMaxScaler', 'self->Normalizer', 'self->PolynomialFeatures', 'self->PowerTransformer', 'self->QuantileTransformer', 'self->RobustScaler', 'self->StandardScaler', 'sparse->UNKNOWN_MODULE', 'start-><Call owner_token=None token=int>', 'stats->UNKNOWN_MODULE', 'subsample_idx-><Call owner_token=random_state token=choice>', 'var-><Call owner_token=None token=_handle_zeros_in_scale>', 'warnings->UNKNOWN_MODULE', 'weights-><Call owner_token=None token=_check_sample_weight>', 'x_inv-><Call owner_token=np token=exp>', 'x_inv-><Call owner_token=np token=zeros_like>', 'x_trans-><Call owner_token=self token=_yeo_johnson_transform>'].
Code2Flow: Generating output file...
Code2Flow: Wrote output file 'out.gv' with 48 nodes and 55 edges.
Code2Flow: For better machine readability, you can also try outputting in a json format.
Code2Flow: Code2flow finished processing in 0.04 seconds.
Code2Flow: Running graphviz to make the image...
Traceback (most recent call last):
  File "C:\Users\sterg\AppData\Local\Programs\Python\Python39\Scripts\code2flow-script.py", line 33, in <module>
    sys.exit(load_entry_point('code2flow==2.2.0', 'console_scripts', 'code2flow')())
  File "c:\users\sterg\appdata\local\programs\python\python39\lib\site-packages\code2flow\engine.py", line 625, in main
    code2flow(
  File "c:\users\sterg\appdata\local\programs\python\python39\lib\site-packages\code2flow\engine.py", line 556, in code2flow
    _generate_final_img(output_file, extension, final_img_filename, len(edges))
  File "c:\users\sterg\appdata\local\programs\python\python39\lib\site-packages\code2flow\engine.py", line 473, in _generate_final_img
    _generate_graphviz(output_file, extension, final_img_filename)
  File "c:\users\sterg\appdata\local\programs\python\python39\lib\site-packages\code2flow\engine.py", line 461, in _generate_graphviz
    subprocess.run(command, stdout=f, check=True)
  File "c:\users\sterg\appdata\local\programs\python\python39\lib\subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['dot', '-Tpng', 'out.gv']' returned non-zero exit status 3221225477.

Based on https://github.com/pachterlab/kb_python/issues/67, maybe this is also a Windows issue.

Probably more related: https://github.com/xflr6/graphviz/issues/69

Here's the out.gv.zip (zipped because of GitHub file requirements)

Not very time-sensitive for me.

scottrogowski commented 3 years ago

Verified that dot fails for me on the file you passed over - so it's not a windows issue. Segmentation faults on dot happens for very large graphs but your graph is rather small so I don't know off the top of my head why there is a segmentation fault.

It's probably not a code2flow issue but regardless it's a bad user experience that at least needs a more helpful error message. I'll keep this issue open until I can investigate.

scottrogowski commented 3 years ago

I added a little more debugging around that statement and made a release.

Unfortunately, I can't figure out your issue. Removing one seemingly random edge, node_44c65099 -> node_fb9ca97e [color="#cf142b" penwidth="2"];, makes the segfault go away though there is nothing unusual about this edge or the nodes to think they are in any way special. So unfortunately I'm going to close this issue as there is not much I can do. As far as I can tell, the only really solution would be to use different graphing software of which I'm not aware of any