Python Descriptors

5 min readMay 9, 2020

Any python class which implement the descriptor protocol can be considered a descriptor class. And any class containing a descriptor class (either in its class dictionary or in the dictionary in one of its parents) will be referred in this article as an owner class.

Python descriptors are objects that implement the descriptor protocol. It gives you the ability to create objects that have special behaviour when they’re accessed as attributes of other objects. Python descriptor protocol is the mechanism that powers the properties, methods, static methods, and class methods.

The dunder methods below are the definition of the descriptor protocol:

__get__(self, obj, type=None) -> object
__set__(self, obj, value) -> None
__delete__(self, obj) -> None
__set_name__(self, owner, name)

Note that it is not necessary to implement all the protocol methods above to create a descriptor class.

There are two types of descriptors:

non-data descriptor which only implements the .__get__()
data descriptor if it implements .__set__() or .__delete__()

Data descriptors have precedence in the python lookup chain.

The code-block below is an example of a descriptor:

#python descriptorsclass Protein():
    def __get__(self, obj, type=None) -> object:
        print("get protein")
        return "Chicken"    def __set__(self, obj, value) -> None:
        print("setting the value")
        raise AttributeError("Cannot change the value")class Breakfast():
    protein = Protein()my_breakfast = Breakfast()
print(my_breakfast.protein)

In the example above, Protein() implements the descriptor protocol. Once it’s instantiated as an attribute of Breakfast, it can be considered a descriptor.

As a descriptor, it has a binding behaviour when it’s accessed using the dot notation. In this case, the descriptor logs a message on the console every time it’s accessed to get or set the value.

Binding behaviour, in this case, means that the descriptor binds itself to the owner class.

Now, let’s run the script to see the output.

>> get protein
>> Chicken

How Descriptors Work in Python’s Internals

The example above can be achieved through the @properties decorator. This is true, and the python properties are actually just an implementation of descriptors.

Python Descriptors in Properties

If you want to get the same result as the previous example without explicitly using a python descriptor, then the most straightforward approach is to use a property. The example below uses a property decorator to achieve the same result as example 1.

class Protein():    @property
    def protein(self) -> object:
       print("get protein")       return "Chicken"    @protein.setter
    def protein(self, value) -> None:
        print("setting the value")
        raise AttributeError("Cannot change the value")my_breakfast = Breakfast()
print(my_breakfast.protein)

The example above makes use of decorators to define a property, but as you may know, decorators are just syntactic sugar. The example before, in fact, can be written as follows:

class Protein():
    def getter(self) -> object:
        print("get protein")
        return "Chicken"    def setter(self, value) -> None:
        print("setting the value")        raise AttributeError("Cannot change the value")
# Create a protein property using the property function
# The sginature of the property function is:
# property(fget=None, fset=None, fdel=None, doc=None) -> objectprotein = property(getter, setter)#The function above returns an object that implements a descriptor protocol.
my_breakfast = Breakfast()
print(my_breakfast.protein)

Python Descriptors in Methods and Functions

If you’ve ever written an object-oriented program in Python, then you’ve certainly used methods. These are regular functions that have the first argument reserved for the object instance. When you access a method using dot notation, you’re calling the corresponding function and passing the object instance as the first parameter.

The magic that transforms your obj.method(*args) into method(obj, *args) is inside the .__get__() implementation of the function object that is, in fact, a non-data descriptor. In particular, the function object implements .__get__() so that it returns a bound method when you access it with dot notation. The (*args) that follow invoke the functions by passing all the extra arguments needed.

This works for regular instance methods just like it does for class methods or static methods. So, if you call a static method with obj.method(*args), then it’s automatically transformed into method(*args). Similarly, if you call a class method with obj.method(type(obj), *args), then it’s automatically transformed into method(type(obj), *args).

In the official docs, you can find some examples of how static methods and class methods would be implemented if they were written in pure Python instead of the actual C implementation.

Note that, in Python, a class method is just a static method that takes the class reference as a first argument in the argument list.

How Attributes Are Accessed With the Lookup Chain

To understand a little more about Python descriptors and Python internals, you need to understand what happens in Python when an attribute is accessed. In Python, every object has a built-in __dict__ attribute. This is a dictionary that contains all the attributes defined in the object itself. To see this in action, consider the following example:

class Breakfast(object):
    protein = "Chicken"
    price = 10class BreakfastSet(Breakfast):
    price = 12
    
    def __init__(self, promo=True):
        self.promo = promomy_breakfast = BreakfastSet()print(my_breakfast.protein)
print(my_breakfast.price)
print(my_breakfast.promo)print(my_breakfast.__dict__)
print(type(my_breakfast).__dict__)
print(type(my_breakfast).__base__.__dict__)===================
Result:>> Chicken
>> 12
>> True>> {'promo': True}
>> {'__module__': '__main__', 'price': 12, '__init__': <function BreakfastSet.__init__ at 0x10eda6e18>, '__doc__': None}
>>{'__module__': '__main__', 'protein': 'Chicken', 'price': 10, '__dict__': <attribute '__dict__' of 'Breakfast' objects>, '__weakref__': <attribute '__weakref__' of 'Breakfast' objects>, '__doc__': None}

https://docs.python.org/3/reference/datamodel.html#invoking-descriptors

This code creates a new object and prints the contents of the __dict__ attribute for both the object and the class. Now, run the script and analyze the output to see the __dict__ attributes set:

The __dict__ attributes are set as expected. Note that, in Python, everythin is an object. A class is actually an object as well, so it will also have a __dict__ attribute that contains all the attributes and methods of the class…..

How to Use Python Descriptors Properly

If you want to use Python descriptors in your code, then you just need to implement the descriptor protocol. The most important methods of this protocol are .__get__() and .__set__(), which have the following signature

When you implement the protocol, keep these things in mind:

self is the instance of the descriptor you’re writing.

obj is the instance of the object your descriptor is attached to.

type is the type of the object the descriptor is attached to.

In .__set__(), you don’t have the type variable, because you can only call .__set__() on the object. In contrast, you can call .__get__() on both the object and the class.

Another important thing to know is that Python descriptors are instantiated just once per class. That means that every single instance of a class containing a descriptor shares that descriptor instance. This is something that you might not expect and can lead to a classic pitfall, like this:

class Protein():
    def __init__(self):
        self.value = "Chicken"    def __get__(self, obj, type=None) -> object:
        return self.value    def __set__(self, obj, value) -> None:
        self.value = valueclass Breakfast():
    protein = Protein()first_breakfast = Breakfast()
second_breakfast = Breakfast()
print(first_breakfast.protein)
print(second_breakfast.protein)print("------assign new value 'Beef' to first_breakfast.protein------")first_breakfast.protein = "Beef"
print(first_breakfast.protein)
print(second_breakfast.protein)third_breakfast = Breakfast()
print(third_breakfast.protein)

Result:

Chicken
Chicken
------assign new value 'Beef' to first_breakfast.protein------
Beef
Beef
Beef
Beef
Beef

In the example above, each breakfast instance points to the same instance of the Protein descriptor.

Solution use __set_name__ for Pyhton > 3.6

#use set_nameclass Protein():
    def __set_name__(self, owner, name):
        self.name = name    def __get__(self, obj, type=None) -> object:
        return obj.__dict__.get(self.name) or None    def __set__(self, obj, value) -> None:
        obj.__dict__[self.name] = value
class Breakfast():
    protein = Protein()
first_breakfast = Breakfast()
second_breakfast = Breakfast()print(first_breakfast.protein)
print(second_breakfast.protein)print("------assign new value to <object>.protein------")first_breakfast.protein = "Beef"
seconf_breakfast.protein = "Tofu"print(first_breakfast.protein)
print(second_breakfast.protein)third_breakfast = Breakfast()
print(third_breakfast.protein)
print(first_breakfast.protein)
print(second_breakfast.protein)