trailofbits / fickling

A Python pickling decompiler and static analyzer
GNU Lesser General Public License v3.0
421 stars 48 forks source link

Errors when scanning Stable Diffusion/Textual Inversion embeddings pickle file #22

Closed BeanCounterTop closed 2 years ago

BeanCounterTop commented 2 years ago

I'm trying to give the stable diffusion community the ability to trade Textual Inversion embeddings (basically, fine-tuning the model) between each other. When I run fickle against one my embeddings, I see this:

(base) berble@berbletron:~/Downloads/archive$ fickling -t data.pkl

PROTO EMPTY_DICT Pushed {} BINPUT Memoized 0 -> {} MARK Pushed MARK BINUNICODE Pushed 'string_to_token' BINPUT Memoized 1 -> 'string_to_token' EMPTY_DICT Pushed {} BINPUT Memoized 2 -> {} BINUNICODE Pushed '' BINPUT Memoized 3 -> '' GLOBAL Traceback (most recent call last): File "/home/berble/.local/bin/fickling", line 8, in sys.exit(main()) File "/home/berble/.local/lib/python3.8/site-packages/fickling/cli.py", line 82, in main print(unparse(trace.run())) File "/home/berble/.local/lib/python3.8/site-packages/fickling/tracing.py", line 54, in run self.on_statement(added) File "/home/berble/.local/lib/python3.8/site-packages/fickling/tracing.py", line 38, in on_statement print(f"\t{unparse(statement).strip()}") File "/home/berble/.local/lib/python3.8/site-packages/astunparse/init.py", line 13, in unparse Unparser(tree, file=v) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 38, in init self.dispatch(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch meth(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 113, in _ImportFrom interleave(lambda: self.write(", "), self.dispatch, t.names) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 19, in interleave f(next(seq)) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 66, in dispatch meth(tree) File "/home/berble/.local/lib/python3.8/site-packages/astunparse/unparser.py", line 856, in _alias if t.asname: AttributeError: 'alias' object has no attribute 'asname'

Any idea where I could start looking? We'd really like to be able to share embeddings safely!

Here's a base64-encoded copy of my data.pkl:

gAJ9cQAoWA8AAABzdHJpbmdfdG9fdG9rZW5xAX1xAlgBAAAAKnEDY3RvcmNoLl91dGlscwpfcmVi dWlsZF90ZW5zb3JfdjIKcQQoKFgHAAAAc3RvcmFnZXEFY3RvcmNoCkxvbmdTdG9yYWdlCnEGWAEA AAAwcQdYAwAAAGNwdXEIS010cQlRSwEpKYljY29sbGVjdGlvbnMKT3JkZXJlZERpY3QKcQopUnEL dHEMUnENc1gPAAAAc3RyaW5nX3RvX3BhcmFtcQ5jdG9yY2gubm4ubW9kdWxlcy5jb250YWluZXIK UGFyYW1ldGVyRGljdApxDymBcRB9cREoWAgAAAB0cmFpbmluZ3ESiFgLAAAAX3BhcmFtZXRlcnNx E2gKKVJxFGgDY3RvcmNoLl91dGlscwpfcmVidWlsZF9wYXJhbWV0ZXIKcRVoBCgoaAVjdG9yY2gK RmxvYXRTdG9yYWdlCnEWWAEAAAAxcRdYBgAAAGN1ZGE6MHEYTQADdHEZUUsASwFNAAOGcRpNAANL AYZxG4loCilScRx0cR1ScR6IaAopUnEfh3EgUnEhc1gIAAAAX2J1ZmZlcnNxImgKKVJxI1gbAAAA X25vbl9wZXJzaXN0ZW50X2J1ZmZlcnNfc2V0cSRjX19idWlsdGluX18Kc2V0CnElXXEmhXEnUnEo WA8AAABfYmFja3dhcmRfaG9va3NxKWgKKVJxKlgWAAAAX2lzX2Z1bGxfYmFja3dhcmRfaG9va3Er TlgOAAAAX2ZvcndhcmRfaG9va3NxLGgKKVJxLVgSAAAAX2ZvcndhcmRfcHJlX2hvb2tzcS5oCilS cS9YEQAAAF9zdGF0ZV9kaWN0X2hvb2tzcTBoCilScTFYGgAAAF9sb2FkX3N0YXRlX2RpY3RfcHJl X2hvb2tzcTJoCilScTNYGwAAAF9sb2FkX3N0YXRlX2RpY3RfcG9zdF9ob29rc3E0aAopUnE1WAgA AABfbW9kdWxlc3E2aAopUnE3WAUAAABfa2V5c3E4fXE5aANOc3VidS4=

ESultanik commented 2 years ago

This appears to be due to a bug in the astunparse module that wasn't fixed until Python 3.9. I am adding a workaround in Fickling that fixes this problem, but in the mean time the issue should disappear if you upgrade to a newer version of Python.