Open ViacheslavDanilov opened 2 years ago
@ViacheslavDanilov thank you for reporting the issue. The original path should be split based on \
and /
and then joined with os.path.join
- what do you think?
@pplonski thanks for the prompt reply.
When I replace the double slash in learner_subpath
, the learner_path
variable is getting readable by l = AlgorithmFactory.load()
and the code works well. As you see, os.path.join
works only when a model is trained and tested on the same operating system i.e. trained on Windows and tested on Windows. However, when there are cases such as I encountered, the error is thrown. I would suggest normalizing the path somehow. The most straightforward solution I see is to use something like the following learner_subpath = learner_subpath.replace("\\", "/") if '\\' in learner_subpath else learner_subpath
and call it right before learner_path = os.path.join(results_path, learner_subpath)
. At least, this resolves the problem with cross-platform usage.
Here is what I get using the source code:
results_path = 'experiments/LMN_compete_mae_Raw_Power_Source_0408_1405'
learner_subpath = '100_LightGBM\\learner_fold_0.lightgbm'
learner_path = 'experiments/LMN_compete_mae_Raw_Power_Source_0408_1405/100_LightGBM\\learner_fold_0.lightgbm'
Having the same issue when deploying a model from windows to linux, @ViacheslavDanilov were you able to solve it?
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/ubuntu/./mastermind/AI/ai.py", line 148, in predict
return self.model.predict(features)[0]
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/automl.py", line 387, in predict
return self._predict(X)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/base_automl.py", line 1361, in _predict
predictions = self._base_predict(X)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/base_automl.py", line 1318, in _base_predict
X_stacked = self.get_stacked_data(X, mode="predict")
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/base_automl.py", line 421, in get_stacked_data
oof = m.predict(X)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/ensemble.py", line 308, in predict
y_predicted_from_model = model.predict(X)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/model_framework.py", line 429, in predict
y_p = learner.predict(X_data)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/algorithms/lightgbm.py", line 270, in predict
self.reload()
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/algorithms/algorithm.py", line 36, in reload
self.load(self.model_file_path)
File "/home/ubuntu/.local/lib/python3.8/site-packages/supervised/algorithms/lightgbm.py", line 285, in load
self.model = lgb.Booster(model_file=model_file_path)
File "/home/ubuntu/.local/lib/python3.8/site-packages/lightgbm/basic.py", line 2639, in __init__
_safe_call(_LIB.LGBM_BoosterCreateFromModelfile(
File "/home/ubuntu/.local/lib/python3.8/site-packages/lightgbm/basic.py", line 125, in _safe_call
raise LightGBMError(_LIB.LGBM_GetLastError().decode('utf-8'))
lightgbm.basic.LightGBMError: Could not open AutoML\4_Default_LightGBM_KMeansFeatures\learner_fold_0.lightgbm
I trained models on Windows, then I tried to use them on Linux, however, I could not load them due to an incorrect path joining. During model loading, I got
learner_path
in the following formatexperiments_dir/model_1/100_LightGBM\\learner_fold_0.lightgbm
. The last two slashes were incorrectly concatenated with the rest part of the path. In this regard, I would suggest adding something likelearner_subpath = learner_subpath.replace("\\", "/")
before this code line. Though, there is a need to think about opposite cases: when a model is trained on Linux and then is used on Windows.