keras-team / autokeras

AutoML library for deep learning
http://autokeras.com/
Apache License 2.0
9.1k stars 1.4k forks source link

switch bert to KerasNLP implementation #1816

Closed haifeng-jin closed 1 year ago

haifeng-jin commented 1 year ago

AutoKeras has switched to KerasNLP for BERT implementation without any problem. Successfully removed 1500+ lines of code! This PR is pending on KerasNLP 0.4.0 release before merging.

Due to the limitation of AutoKeras, we are using the BertPreprocessor as part of the main Keras model. In the future, this should be in the tf.data pipeline instead.

cc: @mattdangerw, @jbischof, @chenmoneygithub

jbischof commented 1 year ago

KerasNLP 0.4 is released. Merge away!

mattdangerw commented 1 year ago

Yeah, this is awesome!! Keep us posted on how it goes!

codecov[bot] commented 1 year ago

Codecov Report

Base: 100.00% // Head: 100.00% // No change to project coverage :thumbsup:

Coverage data is based on head (d3fe967) compared to base (4dd289a). Patch coverage: 100.00% of modified lines in pull request are covered.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #1816 +/- ## ========================================== Coverage 100.00% 100.00% ========================================== Files 51 50 -1 Lines 3422 2920 -502 ========================================== - Hits 3422 2920 -502 ``` | [Impacted Files](https://codecov.io/gh/keras-team/autokeras/pull/1816?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team) | Coverage Δ | | |---|---|---| | [autokeras/keras\_layers.py](https://codecov.io/gh/keras-team/autokeras/pull/1816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-YXV0b2tlcmFzL2tlcmFzX2xheWVycy5weQ==) | `100.00% <ø> (ø)` | | | [autokeras/\_\_init\_\_.py](https://codecov.io/gh/keras-team/autokeras/pull/1816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-YXV0b2tlcmFzL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | | | [autokeras/blocks/basic.py](https://codecov.io/gh/keras-team/autokeras/pull/1816/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team#diff-YXV0b2tlcmFzL2Jsb2Nrcy9iYXNpYy5weQ==) | `100.00% <100.00%> (ø)` | | Help us with your feedback. Take ten seconds to tell us [how you rate us](https://about.codecov.io/nps?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team). Have a feature suggestion? [Share it here.](https://app.codecov.io/gh/feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=keras-team)

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.