search

jpmml / jpmml-lightgbm

Java library and command-line application for converting LightGBM models to PMML
GNU Affero General Public License v3.0
174 stars 58 forks source link

Fail to convert lightgbm to pmml （IndexOutOfBoundsException） #22

Closed sp910 closed 5 years ago

sp910 commented 5 years ago

when I run java -jar jpmml-lightgbm-executable-1.2-SNAPSHOT.jar --lgbm-input aa.txt --pmml-output lightgbm.pmml to convert lightgbm model to pmml format, but encountered,

Exception in thread "main" java.lang.IndexOutOfBoundsException: Index: 2, Size: 2 at java.util.ArrayList.rangeCheck(ArrayList.java:657) at java.util.ArrayList.get(ArrayList.java:433) at org.jpmml.lightgbm.Tree.selectValues(Tree.java:244) at org.jpmml.lightgbm.Tree.encodeNode(Tree.java:155) at org.jpmml.lightgbm.Tree.encodeNode(Tree.java:191) at org.jpmml.lightgbm.Tree.encodeNode(Tree.java:191) at org.jpmml.lightgbm.Tree.encodeNode(Tree.java:191) at org.jpmml.lightgbm.Tree.encodeTreeModel(Tree.java:94) at org.jpmml.lightgbm.ObjectiveFunction.createMiningModel(ObjectiveFunction.java:66) at org.jpmml.lightgbm.BinomialLogisticRegression.encodeMiningModel(BinomialLogisticRegression.java:48) at org.jpmml.lightgbm.GBDT.encodeMiningModel(GBDT.java:285) at org.jpmml.lightgbm.GBDT.encodePMML(GBDT.java:274) at org.jpmml.lightgbm.Main.run(Main.java:131) at org.jpmml.lightgbm.Main.main(Main.java:117)

Attached is the lightgbm model file. aa.txt

lightgbm version is 2.2.3

Can someone please kindly help? Thanks much!

vruusmann commented 5 years ago

The model file (aa.txt) appears to contain a pandas_categorical section, which confuses the current parser code: https://github.com/jpmml/jpmml-lightgbm/blob/master/src/main/java/org/jpmml/lightgbm/GBDT.java#L422-L466

Does the conversion succeed if you simplify category levels?

sp910 commented 5 years ago

I found it will be success when I use more data traing，but with not simplify category levels.

Attached file can be convert succeed, but I didn't simplify category levels. gbm_3yr.txt

77QingLiu commented 5 years ago

I also encounter this error, could you please explain how to simplify category levels?

yairdata commented 4 years ago

happened to me on 1.3.0 version i use the pandas_categorical because i want the categorical values to be interpreted as string and not be converted to Integers. i can't work with the integers because the performance is worse when it works with integers , and because of inference issues regarding conversion in runtime to integers. (sklearn pipeline had other issues)

etveritas commented 4 years ago

I also encounter this error, could you please explain how to simplify category levels?

Me too

etveritas commented 4 years ago

@vruusmann I have category and continuouse feature both in lightgbm model, and I encounter this error.Could you please explain how to simplify category levels?

jean-jm commented 9 months ago

me too

vruusmann commented 9 months ago

me too

Too bad, but I can't help you.

You have full access to your LightGBM model file, and my JPMML-LightGBM source code. Feel free to fix the issue, and submit a corrective patch back to the project.

jean-jm commented 9 months ago

您好，邮件已收到，谢谢啦~

jean-jm commented 9 months ago

this code has some problems ,so that it will throw error like "outofsize"

------------------ 原始邮件 ------------------ 发件人: "jpmml/jpmml-lightgbm" @.>; 发送时间: 2023年11月23日(星期四) 下午4:44 @.>; @.**@.>; 主题: Re: [jpmml/jpmml-lightgbm] Fail to convert lightgbm to pmml （IndexOutOfBoundsException） (#22)

me too

Too bad, but I can't help you.

You have full access to your LightGBM model file, and my JPMML-LightGBM source code. Feel free to fix the issue, and submit a corrective patch back to the project.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>

vruusmann commented 9 months ago

this code has some problems ..

@jean-jm This is not an actionable bug report.

Please provide a reproducible example (a LightGBM model object, which fails to convert using the latest JPMML-LightGBM version 1.4.6), or work on this on your own.

Please note that I don't need your actual proprietary LightGBM model object. You can train one from scratch, using some toy dataset (eg. "iris").

jean-jm commented 9 months ago

3q! this txt file can be reproducible, looking forward to your reply!

------------------ 原始邮件 ------------------ 发件人: "jpmml/jpmml-lightgbm" @.>; 发送时间: 2023年11月23日(星期四) 晚上7:54 @.>; @.**@.>; 主题: Re: [jpmml/jpmml-lightgbm] Fail to convert lightgbm to pmml （IndexOutOfBoundsException） (#22)

this code has some problems ..

@jean-jm This is not an actionable bug report.

Please provide a reproducible example (a LightGBM model object, which fails to convert using the latest JPMML-LightGBM version 1.4.6), or work on this on your own.

Please note that I don't need your actual proprietary LightGBM model object. You can train one from scratch, using some toy dataset (eg. "iris").

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

jean-jm commented 9 months ago

test2jm_light_dataset1_20231117_v1.txt.zip this file can be used, thank you so much, looking forward to your reply.

vruusmann commented 9 months ago

@jean-jm Let's continue here: https://github.com/jpmml/jpmml-lightgbm/issues/62

Your issue looks new/different from what was discussed here originally.