search

nod-ai / SHARK-TestSuite

Temporary home of a test suite we are evaluating
Apache License 2.0
5 stars 35 forks source link

removing model which are not valid and duplicates #379

Closed pdhirajkumarprasad closed 1 month ago

pdhirajkumarprasad commented 1 month ago

removing models from list which are not valid based on https://github.com/nod-ai/SHARK-ModelDev/issues/812 and remove duplicates using tool introduced here: https://github.com/nod-ai/SHARK-TestSuite/pull/376

saienduri commented 1 month ago

LGTM, let's wait for @zjgarvey before landing

pdhirajkumarprasad commented 1 month ago

I'd like to retain the original shard lists somewhere. I'm not sure if Kumar or others want to still have numbers reported based on the full list with redundant tests.

--I can keep the original list with extension _old but our run should be with the unique model list and it's fine if that lead to lesser no. of model but that will be unique list

Do we also need to redistribute across shards?

--Yes, I can do that

pdhirajkumarprasad commented 1 month ago

closing this pull request as the way DUP model was found is not correct and it will reduce the coverage of ops. Once we have correct list, will create the pull request