dvmazur / mixtral-offloading

Run Mixtral-8x7B models in Colab or consumer desktops
MIT License
2.28k stars 223 forks source link

Utilized pop for meta keys cleanup #9

Closed vivekmaru36 closed 6 months ago

vivekmaru36 commented 6 months ago

Instead of manually iterating through each key in del_keys to delete them from the meta dictionary, use the pop() method to remove these keys if they exist

The pop() method removes the key if it exists and does nothing if the key is not found

dvmazur commented 6 months ago

Hey!

Replacing the del statement with calling dict's pop method doesn't change the logic of the code.

Besides that, calling pop runs more instructions than calling del:

Python 3.10.12 (main, Nov 20 2023, 15:14:05) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import dis
>>> some_dict = dict(a=1)
>>> def del_from_dict(): del some_dict["a"]
...
>>> def pop_from_dict(): some_dict.pop("a")
...
>>> dis.dis(del_from_dict)
  1           0 LOAD_GLOBAL              0 (some_dict)
              2 LOAD_CONST               1 ('a')
              4 DELETE_SUBSCR
              6 LOAD_CONST               0 (None)
              8 RETURN_VALUE
>>> dis.dis(pop_from_dict)
  1           0 LOAD_GLOBAL              0 (some_dict)
              2 LOAD_METHOD              1 (pop)
              4 LOAD_CONST               1 ('a')
              6 CALL_METHOD              1
              8 POP_TOP
             10 LOAD_CONST               0 (None)
             12 RETURN_VALUE