search

mlcommons / training

Reference implementations of MLPerf™ training benchmarks
https://mlcommons.org/en/groups/training
Apache License 2.0
1.6k stars 553 forks source link

[sentiment analysis] run time error #543

Closed dimanzt closed 1 year ago

dimanzt commented 2 years ago

Hi all, I'm trying to run sentiment analysis on a system with a T4 GPU but I'm getting the following error, any idea how to fix that?

/paddle/training/sentiment_analysis/paddle# ./run_and_time.sh 1 STARTING TIMING RUN AT 2022-03-29 11:17:41 PM Running sentiment benchmark with seed 1

Traceback (most recent call last): File "train.py", line 214, in save_dirname="understand_sentiment_conv.inference.model") File "train.py", line 188, in main save_dirname=save_dirname) File "train.py", line 173, in train train_loop(fluid.default_main_program()) File "train.py", line 137, in train_loop exe.run(fluid.default_startup_program()) File "/usr/local/lib/python2.7/dist-packages/paddle/fluid/executor.py", line 336, in run self.executor.run(program.desc, scope, 0, True, True) RuntimeError: function_attributes(): after cudaFuncGetAttributes: invalid device function ENDING TIMING RUN AT 2022-03-29 11:21:30 PM RESULT,sentiment,1,229,,2022-03-29 11:17:41 PM

matthew-frank commented 1 year ago

The sentiment analysis program was never adopted as a supported benchmark, but that was not made clear in the source repo until recently. Unfortunately I don't know of anyone who knows how to get this code running anymore.

@johntran-nv can you close this as "will-not-fix"?