laixintao / problems

❓See issues.
0 stars 0 forks source link

Python @lru_cache 的问题 #14

Closed laixintao closed 5 years ago

laixintao commented 5 years ago

1 为什么要用两把锁?

https://github.com/python/cpython/blob/2fb2bc81c3f40d73945c6102569495140e1182c7/Lib/functools.py#L580

这里我感觉直接用一把锁,将第一把锁锁住的内容放到第二把锁的 pass 那个地方就可以了。为什么要 lock 两次?

猜想是减少锁的粒度。

2 为什么要强行赋值来阻止引用计数变零?

https://github.com/python/cpython/blob/2fb2bc81c3f40d73945c6102569495140e1182c7/Lib/functools.py#L613

没太看懂注释,猜想是为了不在持有锁的时候开始垃圾回收,来减少锁时间,提高性能。

laixintao commented 5 years ago
 def wrapper(*args, **kwds):
            # Size limited caching that tracks accesses by recency
            nonlocal root, hits, misses, full
            key = make_key(args, kwds, typed)
            with lock:
                link = cache_get(key)
                if link is not None:
                    # Move the link to the front of the circular queue
                    link_prev, link_next, _key, result = link
                    link_prev[NEXT] = link_next
                    link_next[PREV] = link_prev
                    last = root[PREV]
                    last[NEXT] = root[PREV] = link
                    link[PREV] = last
                    link[NEXT] = root
                    hits += 1
                    return result
                elif full:
                    # Use the old root to store the new key and result.
                    misses += 1
                    result = user_function(*args, **kwds)
                    oldroot = root
                    oldroot[KEY] = key
                    oldroot[RESULT] = result
laixintao commented 5 years ago

第二个问题是为了避免key或result被销毁,不是关于性能。比如这里的result如果是一个很特殊的对象,它的类定义了del,要知道这里是可以写任意代码的,我可能又去调用了被lru_cache装饰了的函数,那么这里是一个RLOCK,同一个线程我又可以进来,我又处于更新环形双向链表的过程中,会造成这个双向链表的结构被破坏

laixintao commented 5 years ago

你这样user_function不就有锁了(如果将第一个lock放到 pass那个地方),如果我user_function里面起了一个线程然后又执行了一个lru_cache的函数呢,不死锁了