8.11. "weakref" — Weak references
*********************************

New in version 2.1.

**Source code:** Lib/weakref.py

======================================================================

The "weakref" module allows the Python programmer to create *weak
references* to objects.

In the following, the term *referent* means the object which is
referred to by a weak reference.

A weak reference to an object is not enough to keep the object alive:
when the only remaining references to a referent are weak references,
*garbage collection* is free to destroy the referent and reuse its
memory for something else.  A primary use for weak references is to
implement caches or mappings holding large objects, where it’s desired
that a large object not be kept alive solely because it appears in a
cache or mapping.

For example, if you have a number of large binary image objects, you
may wish to associate a name with each.  If you used a Python
dictionary to map names to images, or images to names, the image
objects would remain alive just because they appeared as values or
keys in the dictionaries.  The "WeakKeyDictionary" and
"WeakValueDictionary" classes supplied by the "weakref" module are an
alternative, using weak references to construct mappings that don’t
keep objects alive solely because they appear in the mapping objects.
If, for example, an image object is a value in a
"WeakValueDictionary", then when the last remaining references to that
image object are the weak references held by weak mappings, garbage
collection can reclaim the object, and its corresponding entries in
weak mappings are simply deleted.

"WeakKeyDictionary" and "WeakValueDictionary" use weak references in
their implementation, setting up callback functions on the weak
references that notify the weak dictionaries when a key or value has
been reclaimed by garbage collection.  Most programs should find that
using one of these weak dictionary types is all they need – it’s not
usually necessary to create your own weak references directly.  The
low-level machinery used by the weak dictionary implementations is
exposed by the "weakref" module for the benefit of advanced uses.

Not all objects can be weakly referenced; those objects which can
include class instances, functions written in Python (but not in C),
methods (both bound and unbound), sets, frozensets, file objects,
*generator*s, type objects, "DBcursor" objects from the "bsddb"
module, sockets, arrays, deques, regular expression pattern objects,
and code objects.

Changed in version 2.4: Added support for files, sockets, arrays, and
patterns.

Changed in version 2.7: Added support for thread.lock, threading.Lock,
and code objects.

Several built-in types such as "list" and "dict" do not directly
support weak references but can add support through subclassing:

   class Dict(dict):
       pass

   obj = Dict(red=1, green=2, blue=3)   # this object is weak referenceable

**CPython implementation detail:** Other built-in types such as
"tuple" and "long" do not support weak references even when
subclassed.

Extension types can easily be made to support weak references; see
Weak Reference Support.

class weakref.ref(object[, callback])

   Return a weak reference to *object*.  The original object can be
   retrieved by calling the reference object if the referent is still
   alive; if the referent is no longer alive, calling the reference
   object will cause "None" to be returned.  If *callback* is provided
   and not "None", and the returned weakref object is still alive, the
   callback will be called when the object is about to be finalized;
   the weak reference object will be passed as the only parameter to
   the callback; the referent will no longer be available.

   It is allowable for many weak references to be constructed for the
   same object. Callbacks registered for each weak reference will be
   called from the most recently registered callback to the oldest
   registered callback.

   Exceptions raised by the callback will be noted on the standard
   error output, but cannot be propagated; they are handled in exactly
   the same way as exceptions raised from an object’s "__del__()"
   method.

   Weak references are *hashable* if the *object* is hashable.  They
   will maintain their hash value even after the *object* was deleted.
   If "hash()" is called the first time only after the *object* was
   deleted, the call will raise "TypeError".

   Weak references support tests for equality, but not ordering.  If
   the referents are still alive, two references have the same
   equality relationship as their referents (regardless of the
   *callback*).  If either referent has been deleted, the references
   are equal only if the reference objects are the same object.

   Changed in version 2.4: This is now a subclassable type rather than
   a factory function; it derives from "object".

weakref.proxy(object[, callback])

   Return a proxy to *object* which uses a weak reference.  This
   supports use of the proxy in most contexts instead of requiring the
   explicit dereferencing used with weak reference objects.  The
   returned object will have a type of either "ProxyType" or
   "CallableProxyType", depending on whether *object* is callable.
   Proxy objects are not *hashable* regardless of the referent; this
   avoids a number of problems related to their fundamentally mutable
   nature, and prevent their use as dictionary keys.  *callback* is
   the same as the parameter of the same name to the "ref()" function.

weakref.getweakrefcount(object)

   Return the number of weak references and proxies which refer to
   *object*.

weakref.getweakrefs(object)

   Return a list of all weak reference and proxy objects which refer
   to *object*.

class weakref.WeakKeyDictionary([dict])

   Mapping class that references keys weakly.  Entries in the
   dictionary will be discarded when there is no longer a strong
   reference to the key.  This can be used to associate additional
   data with an object owned by other parts of an application without
   adding attributes to those objects.  This can be especially useful
   with objects that override attribute accesses.

   Note: Caution: Because a "WeakKeyDictionary" is built on top of a
     Python dictionary, it must not change size when iterating over
     it.  This can be difficult to ensure for a "WeakKeyDictionary"
     because actions performed by the program during iteration may
     cause items in the dictionary to vanish “by magic” (as a side
     effect of garbage collection).

"WeakKeyDictionary" objects have the following additional methods.
These expose the internal references directly.  The references are not
guaranteed to be “live” at the time they are used, so the result of
calling the references needs to be checked before being used.  This
can be used to avoid creating references that will cause the garbage
collector to keep the keys around longer than needed.

WeakKeyDictionary.iterkeyrefs()

   Return an iterable of the weak references to the keys.

   New in version 2.5.

WeakKeyDictionary.keyrefs()

   Return a list of weak references to the keys.

   New in version 2.5.

class weakref.WeakValueDictionary([dict])

   Mapping class that references values weakly.  Entries in the
   dictionary will be discarded when no strong reference to the value
   exists any more.

   Note: Caution:  Because a "WeakValueDictionary" is built on top
     of a Python dictionary, it must not change size when iterating
     over it.  This can be difficult to ensure for a
     "WeakValueDictionary" because actions performed by the program
     during iteration may cause items in the dictionary to vanish “by
     magic” (as a side effect of garbage collection).

"WeakValueDictionary" objects have the following additional methods.
These methods have the same issues as the "iterkeyrefs()" and
"keyrefs()" methods of "WeakKeyDictionary" objects.

WeakValueDictionary.itervaluerefs()

   Return an iterable of the weak references to the values.

   New in version 2.5.

WeakValueDictionary.valuerefs()

   Return a list of weak references to the values.

   New in version 2.5.

class weakref.WeakSet([elements])

   Set class that keeps weak references to its elements.  An element
   will be discarded when no strong reference to it exists any more.

   New in version 2.7.

weakref.ReferenceType

   The type object for weak references objects.

weakref.ProxyType

   The type object for proxies of objects which are not callable.

weakref.CallableProxyType

   The type object for proxies of callable objects.

weakref.ProxyTypes

   Sequence containing all the type objects for proxies.  This can
   make it simpler to test if an object is a proxy without being
   dependent on naming both proxy types.

exception weakref.ReferenceError

   Exception raised when a proxy object is used but the underlying
   object has been collected.  This is the same as the standard
   "ReferenceError" exception.

See also:

  **PEP 205** - Weak References
     The proposal and rationale for this feature, including links to
     earlier implementations and information about similar features in
     other languages.


8.11.1. Weak Reference Objects
==============================

Weak reference objects have no attributes or methods, but do allow the
referent to be obtained, if it still exists, by calling it:

>>> import weakref
>>> class Object:
...     pass
...
>>> o = Object()
>>> r = weakref.ref(o)
>>> o2 = r()
>>> o is o2
True

If the referent no longer exists, calling the reference object returns
"None":

>>> del o, o2
>>> print r()
None

Testing that a weak reference object is still live should be done
using the expression "ref() is not None".  Normally, application code
that needs to use a reference object should follow this pattern:

   # r is a weak reference object
   o = r()
   if o is None:
       # referent has been garbage collected
       print "Object has been deallocated; can't frobnicate."
   else:
       print "Object is still live!"
       o.do_something_useful()

Using a separate test for “liveness” creates race conditions in
threaded applications; another thread can cause a weak reference to
become invalidated before the weak reference is called; the idiom
shown above is safe in threaded applications as well as single-
threaded applications.

Specialized versions of "ref" objects can be created through
subclassing. This is used in the implementation of the
"WeakValueDictionary" to reduce the memory overhead for each entry in
the mapping.  This may be most useful to associate additional
information with a reference, but could also be used to insert
additional processing on calls to retrieve the referent.

This example shows how a subclass of "ref" can be used to store
additional information about an object and affect the value that’s
returned when the referent is accessed:

   import weakref

   class ExtendedRef(weakref.ref):
       def __init__(self, ob, callback=None, **annotations):
           super(ExtendedRef, self).__init__(ob, callback)
           self.__counter = 0
           for k, v in annotations.iteritems():
               setattr(self, k, v)

       def __call__(self):
           """Return a pair containing the referent and the number of
           times the reference has been called.
           """
           ob = super(ExtendedRef, self).__call__()
           if ob is not None:
               self.__counter += 1
               ob = (ob, self.__counter)
           return ob


8.11.2. Example
===============

This simple example shows how an application can use object IDs to
retrieve objects that it has seen before.  The IDs of the objects can
then be used in other data structures without forcing the objects to
remain alive, but the objects can still be retrieved by ID if they do.

   import weakref

   _id2obj_dict = weakref.WeakValueDictionary()

   def remember(obj):
       oid = id(obj)
       _id2obj_dict[oid] = obj
       return oid

   def id2obj(oid):
       return _id2obj_dict[oid]
