Closed C-K-Loan closed 3 years ago
maziyarpanahi
commented
3 years ago I fixed this issue before the 2.7.0 release, however, the model was already trained and uploaded with the wrong parameter. I'll try to reupload the models to see if this issue is resolved.
maziyarpanahi
commented
3 years ago @vankov Could you please have a look at the bad tasks exception? It should crash and just output irrelevant content like the original model. This also can be a good time to export T5 models with the new Encoder/Decoder method we used for MarianMT.
maziyarpanahi
commented
3 years ago @C-K-Loan Please make sure you removed the T5 models from ~/cache_pretrained and try them again. I think the issue with the T5 type is resolved now. (I re-uploaded the correct models with the same name/timestamp so you need to clean up the previous ones first)
C-K-Loan
commented
3 years ago @maziyarpanahi perfect the first bug is fixed now
maziyarpanahi
commented
3 years ago @C-K-Loan @vankov The second issue (unknown tasks) is not related to the T5 model(s) and is caused by SentencePiece. The model will output PieceIDs that are outside of the range of the SP model. I made sure we filter out those IDs when we are decoding them back to String and the issue seems to be resolved.
maziyarpanahi
commented
3 years ago This should be resolved in the 2.7.1 release https://github.com/JohnSnowLabs/spark-nlp/releases/tag/2.7.1 Please re-open if the issue persists.
List of bugs encountered in 2.7.0
T5.__str() method returns wrong value
T5Transformerinstance, i.e.t5 = T5Transformer.pretrained()and you doprint(t5)it displaysSentenceDetectorDL. The actualtype(t5)isT5Transformer. The problem comes from the fact that for some reason thet5.__str__()method returnsSentenceDetectorDL(Fixed In https://github.com/JohnSnowLabs/spark-nlp/tree/271-release-candidate )T5 wierd Behaivour/Crashes for bad task parameters
t5_smallT5 model throws C++ Errors when configuring task as something wierd/very dissimilar to standard tasks like 'make funny' or 'make sad' or 'make angry'. These errors don't are not occuring forgoogle_t5_small_ssm_nqmodel at all.google_t5_small_ssm_nqoften outputs 'palindrome' in these cases, but its also quite random what you get for undefined/weird tasks. (See Screenshot)This does not happen for tasks that don't exist/work, i.e. 'translate english to spanish' is actually translating everything to German and prepending the German word for 'Spanish' to the original input String. This is not throwing a C++ error .
See screenshot for the error message :
The following are the contents of the error file (Sensitive Data Removed in the end) :