kingfengji / gcForest

This is the official implementation for the paper 'Deep forest: Towards an alternative to deep neural networks'
http://lamda.nju.edu.cn/code_gcForest.ashx
1.31k stars 425 forks source link

Afer I modified the file "demo_mnist-gc.json"and trying to save the model ,then the problem occurs like this #35

Closed zhaopanpan73 closed 6 years ago

zhaopanpan73 commented 6 years ago

[ 2018-03-06 14:31:54,717][fgnet.fit_transform] X_train.shape=(60000, 1, 28, 28), y_train.shape=(60000,), X_test.shape=(10000, 1, 28, 28), y_test.shape=(10000,) [ 2018-03-06 14:31:54,719][fg_win_layer.fit_transform] [data][win1/7x7], bottoms=[u'X', u'y'], tops=[u'win1/7x7/ets', u'win1/7x7/rf'] [ 2018-03-06 14:31:54,719][fg_win_layer.fit_transform] [progress][win1/7x7] ti=0/2, top_name=win1/7x7/ets [ 2018-03-06 14:31:54,721][fg_win_layer.fit_transform] [data][win1/7x7,train] bottoms.shape=60000x1x28x28,60000 [ 2018-03-06 14:31:54,753][win_utils.get_windows] get_windows_start: X.shape=(60000, 1, 28, 28), X_win.shape=(49, 7260000), nw=11, nh=11, c=1, win_x=7, win_y=7, stride_x=2, stride_y=2 [ 2018-03-06 14:31:54,994][win_utils.get_windows] get_windows_end [ 2018-03-06 14:31:55,043][fg_win_layer.fit_transform] [data][win1/7x7,test] bottoms.shape=10000x1x28x28,10000 [ 2018-03-06 14:31:55,049][win_utils.get_windows] get_windows_start: X.shape=(10000, 1, 28, 28), X_win.shape=(49, 1210000), nw=11, nh=11, c=1, win_x=7, win_y=7, stride_x=2, stride_y=2 [ 2018-03-06 14:31:55,164][win_utils.get_windows] get_windows_end [ 2018-03-06 14:32:54,977][base_estimator.fit] Save estimator to /home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/models/win1-7x7-ets-3_folds/win1-7x7-ets-3_folds-0.pkl ... [ 2018-03-06 14:32:55,601][base_estimator.predict_proba] Load estimator from /home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/models/win1-7x7-ets-3_folds/win1-7x7-ets-3_folds-0.pkl ... [ 2018-03-06 14:32:55,603][base_estimator.predict_proba] done ... Traceback (most recent call last): File "examples/demo_mnist.py", line 65, in X_All_enc = gc.fit_transform(X_train, y_train, X_test=X_test, y_test=y_test) File "lib/gcforest/gcforest.py", line 31, in fit_transform self.fg.fit_transform(X_train, y_train, X_test, y_test, train_config) File "lib/gcforest/fgnet.py", line 54, in fit_transform layer.fit_transform(train_config) File "lib/gcforest/layers/fg_win_layer.py", line 107, in fit_transform keep_model_in_mem=train_config.keep_model_in_mem) File "lib/gcforest/estimators/kfold_wrapper.py", line 101, in fit_transform y_proba = est.predict_proba(X[val_idx].reshape((-1, n_dims)), cache_dir=cache_dir) File "lib/gcforest/estimators/base_estimator.py", line 77, in predict_proba batch_size = batch_size or self._default_predict_batch_size(est, X) File "lib/gcforest/estimators/sklearn_estimators.py", line 44, in _default_predict_batch_size return forest_predict_batch_size(clf, X) File "lib/gcforest/estimators/sklearn_estimators.py", line 23, in forest_predict_batch_size mem_size_1 = clf.nclasses clf.n_estimators 16 AttributeError: 'NoneType' object has no attribute 'nclasses'

the modified demo_mnist-gc.json content is as follow: "dataset":{ "train": {"type": "mnist", "data_set": "train", "layout_x": "tensor"}, "test": {"type": "mnist", "data_set": "test", "layout_x": "tensor"} }, "train":{ "keep_model_in_mem":1, "model_cache_dir":"/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/models", "random_state":0, "data_cache":{ "cache_in_disk":{ "default":1 }, "keep_in_mem":{ "default":0 }, "cache_dir":"/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/datas" } }, "dataset": { "train": { "type": "ds_pickle2", "data_path": "/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/datas/train/outputs.pkl", "X_keys": ["pool1/7x7/ets", "pool1/7x7/rf", "pool1/10x10/ets", "pool1/10x10/rf", "pool1/13x13/ets", "pool1/13x13/rf"] }, "test": { "type": "ds_pickle2", "data_path": "/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/datas/test/outputs.pkl", "X_keys": ["pool1/7x7/ets", "pool1/7x7/rf", "pool1/10x10/ets", "pool1/10x10/rf", "pool1/13x13/ets", "pool1/13x13/rf"] } }, and the rest is same as the original in the github. When the problem occurs, I have check the code for severl times, and no errors were found. .Hopeing someone can help me .............

kingfengji commented 6 years ago

delete the contents in /home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/datas then try again, should work.

zhaopanpan73 commented 6 years ago

@kingfengji Hi when the problems occurs , the "cache_dir":"/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/datas" was not created ,only the "model_cache_dir":"/home/zhaopanpan/gcforest/mnist/fg-tree50-depth10-3folds/models" was created and content is the picture below 455

zhaopanpan73 commented 6 years ago

@kingfengji I found the model is None when saved into the cache_path through the methold called "save_model_to_disk” ,,but the model is really exists before it saved.

kingfengji commented 6 years ago

@zhaopanpan73 You are right. I have modified this to make life easier. Next time you run it, it won't produce this pkl that is not useful. for this particular problem, just update the code, delete this pkl and run again. thanks.