search

Azure / counterfit

a CLI that provides a generic automation layer for assessing the security of ML models
MIT License
800 stars 128 forks source link

TextAttack API changed #13

Closed einderyl closed 3 years ago

einderyl commented 3 years ago

Hello, I've just started using this library and things were working for a few days, but I think there's been an API change in TextAttack which broke it (ART works OK).

When I try to load textattack, it can't find some embeddings files:

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/alzotolalgo.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/bae.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/bert.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/camrwr.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/deepwordbug.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/improved_ga.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/InputReduction.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/Kuleshov2017.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/pwws.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/textbugger.py: Could not find word_embeddings\paragramcf on server.

textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings\paragramcf.

[!] Failed to load counterfit/frameworks/textattack/textfooler.py: Could not find word_embeddings\paragramcf on server.

[+] Framework loaded successfully!

It finishes by saying the library loaded successfully, but list attacks gives

[!] No frameworks have been loaded. Try 'load '.

It was working at least 24 hours ago, so it looks like it might be the most recent TextAttack commit.

moohax commented 3 years ago

Will take a look and get a fix.

What OS are you on? It looks to be Windows. TextAttack can't find the path to download because the path that gets passed is incorrect, note the end word_embeddings\paragramcf. It should be https://textattack.s3.amazonaws.com/word_embeddings/paragramcf.

For Windows you need to manually download the paragramcf file and extract it into c:\users\user\.cache.

einderyl commented 3 years ago

Yup, I'm on Windows, thanks for the embeddings fix!

moohax commented 3 years ago

Did that work for you? If so, I'll close the issue.

einderyl commented 3 years ago

Yes it worked for me, but the other TextAttack issue (calling it from Counterfit using the moviereviews example) isn't fixed. I'll edit and open a new issue since they don't seem to be related.