Lesson 11 - Magic Methods in Python

In the previous lesson, Properties in Python, we mentioned properties. In today's Python tutorial, we're going to look at magic methods of objects.

Magic methods

Objects' magic methods are methods that start and end with two underscores. We've already encountered several of these methods, such as the __init__() magic method for object initialization or the __str__() method to get an object as a human readable text representation.

Creating objects

__new__(cls, *args, **kwargs)

We call the __new__() method when we need control over creating an object. Especially if we have a custom class that inherits from built-in classes like int (number) or str (string). Sometimes it's better to use the descriptors or the Factory design pattern for a given situation.

The __new__() method returns either the created object or nothing. If it returns the object, the __init__() method is called, if not, the __init__() method is not called.

Example

class Test:

    def __new__(cls, fail=False):
        print("__new__ method called")
        if not fail:
            return super().__new__(cls)

    def __init__(self):
        print("__init__ method called")

test_1 = Test()
test_2 = Test(fail=True)

The __new__() method takes the object's class as the first parameter and then other arguments passed in the constructor. The class parameter is passed to the __new__() method automatically. If the creation of the object is successful, the __init__() method is called with the parameters from the constructor.

In the __new__() method, we can even assign attributes to the object.

Example

class Point:

    def __new__(cls, x, y):
        self = super().__new__(cls)
        self.x = x
        self.y = y
        return self


point = Point(10, 5)
print(point.x, point.y)

We don't return the object immediately but save it to a variable and assign attributes to it.

__init__(self, *args, **kwargs)

The __init__() method is called when objects are initialized. As the first parameter (self), it takes the object which is passed automatically. The __init__() method should return only None, which Python itself returns if the method doesn't have a return type specified. If the __init__() method returns something other than None, TypeError is thrown.

An example in an interactive console

>>> class Test:
...     def __self__(self): return 1
...
>>> test = Test()
Traceback (most recent call last):
    ...
TypeError: __init__() should return None, not 'int'

__del__(self)

The __del__() method is also called the object's destructor and is called when then object is destroyed, but its behavior depends on the particular Python implementation. In CPython, it's called when the number of references to the object drops to zero. The del command does not call the __del__() method directly, it only decreases the number of references by one. There's also no guarantee that the __del__() method is called when the program is closed. It's better to use a try-finally block or a context manager to free resources.

Object representation

__repr__(self)

The method should return the source-code representation of the object as text so that the following applies:

x = eval(repr(x))

__str__(self)

This magic method should return the human readable representation of the object as a string just like the __repr__() method.

For example:

class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __str__(self):
        return "x: {0.x}, y: {0.y}".format(self)

    def __repr__(self):
        return "Point({0.x}, {0.y})".format(self)


point = Point(10, 5)
print(point)
new_point = eval(repr(point))

__bytes__(self)

This method should return an object representation using bytes, which means that it should return a bytes object.

__format__(self, format_spec)

The method is used to format the text representation of the object. It's called by the format() string method (str.format())

We can use the built-in format() function, which is syntactic sugar for:

def format(value, format_spec):
    return value.__format__(format_spec)

More about string formatting: https://www.python.org/…ps/pep-3101/

A method example:

class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    ... # we omit the previous methods

    def __format__(self, format_spec):
        value = "x: {0.x}, y: {0.y}".format(self)
        return format(value, format_spec)


point = Point(10, 5)
# Trims the string to 12 characters, aligns the text right and fills the empty space with spaces
print("{:>12}".format(point))

If we don't override the method, using the format() method causes TypeError (since CPython 3.4)

Comparison methods

Python pass references to the objects being compared to comparison methods automatically.

__lt__(self, other)

Less than:

x < y

__le__(self, other)

Less or equal:

x <= y

__eq__(self, other)

Equal:

x == y

__ne__(self, other)

Not equal:

x != y

__gt__(self, other)

Greater than:

x > y

__ge__(self, other)

Greater or equal:

x >= y

All comparison methods return True or False or may throw an exception if comparison with the other object isn't supported.

An example:

from math import hypot

class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    def __lt__(self, other):
        if isinstance(other, Point):
            return hypot(self.x, self.y) < hypot(other.x, other.y)
        raise TypeError("unordable types: {}() < {}()".format(self.__class__.__name__,
                                                            other.__class__.__name__))

    def __le__(self, other):
        if isinstance(other, Point):
            return hypot(self.x, self.y) <= hypot(other.x, other.y)
        raise TypeError("unordable types: {}() <= {}()".format(self.__class__.__name__,
                                                            other.__class__.__name__))

    ...

The special __class__() method returns a reference to the object's class. Only the implementations of the __lt__() and __le__() methods are shown in the example, the rest is similar. We compare two Point objects according to the distance from the origin of the coordinates (i.e. the point [0, 0]).

The functools module allows to generate other methods automatically by using one of the __lt__(), __lg__(), __gt__(), or __ge__() methods, and the class should also have __eq__(). However, the use of the total_ordering decorator can have a negative impact on the program speed.

Because the __class__() method returns a link to the class, this method can be used to "clone" the object.

class Point:

    def __init__(self, x, y):
        self.x = x
        self.y = y

    ...

    def clone(self):
        return self.__class__(self.x, self.y)


point = Point(10, 5)
point_clone = point.clone()

Other methods

__hash__(self)

The __hash__() method should be reimplemented if the __eq__() method is defined in order to use the object in some collections.

Its implementation:

class Point:

    ...

    def __hash__(self):
        return hash(id(self))

The id() function returns the address of an object in memory that does not change for the object.

__bool__(self)

The method returns True or False, depending on how the object is evaluated. Numeric objects give False for zero values and containers (list, tuples, ...) give False if they are empty.

class Point:

    ...

    def __bool__(self):
        return bool(self.x and self.y)

Calling an object

__call__(self, *args, **kwargs)

This method can call an object as if it was a function. For example, we'll get an "improved" function that can store status information.

Let's code a factorial that stores the calculated values:

class Factorial:

    def __init__(self):
        self.cache = {}

    def fact(self, number):
        if number == 0:
            return 1
        else:
            return number * self.fact(number-1)

    def __call__(self, number):
        if number in self.cache:
            return self.cache[number]
        else:
            result = self.fact(number)
            self.cache[number] = result
            return result

factorial = Factorial()
print(factorial(200))
print(factorial(2))
print(factorial(200))
print(factorial.cache)

Context manager

The __enter__() and __exit__() methods are used to create custom context managers. The context manager calls the __enter__() method before entering the with block, and the __exit__() method is called when it's been left.

__enter__(self)

The method is called when the context manager enters the context. If the method returns a value, it's stored in a variable following as:

with smt_ctx() as value:
    do_sth() # we put our commands here

__exit__(self, type, value, traceback)

This method is called when leaving the with block. The type variable contains an exception if it has occurred in the with block, if not, it contains None.

In the exception lesson, we'll look at the context manager in more detail and create one.

In the next lesson, Magic Methods in Python - Math methods, we look at mathematical functions and operators.

 

Did you have a problem with anything? Download the sample application below and compare it with your project, you will find the error easily.