You have likely seen context managers before without evening knowing it. In this article we will show how to make context managers from scratch in Python. There is also a library called contextlib that is part of the Python built-in. It is certainly advisable to check the contextlib library out, however, in this article we will make them using the context manager protocol in a class to demonstrate how they work.
Contents
- __enter__ and __exit__methods to implement context manager protocol
- Some simple examples
- Numpy matrix to Mathjax mini project.
Let's start with the context manager that most people will have seen before. You may have been wondering what exactly the with keyword does in Python. Well, its jobs is to indicate to Python that we are entering a context. Compare the two statements below, one is a context manager and the other just a vanilla file open. I have created a myfile.txt and saved it on my desktop to illustrate this concept.
###########CONTEXT MANAGER###################
with open('myfile.txt', 'r') as f:
for row in f:
print(row)
#'line 1'
#'line 2'
#'line 3'
#'line 4'
#'line 5'
print('with open file (f) is now closed:', f.closed)
# with open file (f) is now closed: True
################VANILLA OPEN###################
vanilla_open = open('myfile.txt', 'r')
for row in vanilla_open:
print(row)
print('Vanilla open file is now closed:' , vanilla_open.closed )
#Vanilla open file is now closed: False
vanilla_open.close()
print('Vanilla open file is now closed:' , vanilla_open.closed )
#Vanilla open file is now closed: True
The reader has almost surely seen the following error and been slightly stumped on why it is happening. Well the context manager version cleans up after it has finished and exits the operation. Notice we didn't close the file.
ValueError: I/O operation on closed file.
Let's recreate the context manager we used above to show how it works.
__enter__ and __exit__ Methods
The __enter__ and __exit__ methods are known as the context manager protocol. They work as you would expect in that on entering the context, the __enter__ method is called, and when exiting the __exit__ method is called.
class Example:
def __init__(self):
pass
def __enter__(self):
print('__enter__ called we are now in the context')
return self
def __exit__(self, exc_type, value, traceback):
print('__exit__ called , exiting the context')
return False
with Example() as ex:
print('inside the context we can do stuff')
#__enter__ called we are now in the context
#inside the context we can do stuff
#__exit__ called , exiting the context
Now that we are comfortable with the protocol let's create the file opener we used at the beginning of the document.
class MyFileOpener:
def __init__(self, filename, mode):
self._filename = filename
self.mode = mode
def __enter__(self):
print(f'Entering context going to open {self._filename}')
self._f = open(self._filename, self.mode)
return self._f
def __exit__(self, exc_type, value, traceback):
print('context is finsihed closing the file')
self._f.close()
return False # dont suppress exceptions
with MyFileOpener('myfile.txt', 'a') as file:
for i in range(5):
file.write(f'\nThis is another line number {i} ')
So that works exactly the same as the with open context manager we discussed at the beginning of this document. Opening the 'myfile.txt' document the contents now looks as follows:
line 1
line 2
line 3
line 4
line 5
This is another line number 0
This is another line number 1
This is another line number 2
This is another line number 3
This is another line number 4
Let's take one more simple example before we move on to a mini-project. We will create a context manager that redirects the print statements in the context managers body 'myfile.txt' as opposed to the console / command prompt.
import sys
class PrintRedirect:
def __init__(self, file, mode='a'):
self.file = file
self.mode = mode
#save this here so we can set it back to default later
self.sys_default_stdout = sys.stdout
def __enter__(self):
self._f = open(self.file, self.mode)
print(f'All further print statements will be saved to {self.file}')
sys.stdout = self._f
return self
def __exit__(self, exc_type, value, traceback):
sys.stdout = self.sys_default_stdout
print('you can see print statements in console / prompt again',
'we have exited the context')
return True # supress exceptions
with PrintRedirect('myfile.txt', 'a') as Printer:
print("\n You can't see me in the console")
print("Adding some more lines to my file")
print("Time to end the context this is the last line to save to file")
The output to the console in this example is as follows:
All further print statements will be saved to myfile.txt
you can see print statements in console / prompt again we have exited the context
We can open myfile.txt to view the updated contents.
line 1
line 2
line 3
line 4
line 5
This is another line number 0
This is another line number 1
This is another line number 2
This is another line number 3
This is another line number 4
You can't see me in the console
Adding some more lines to my file
Time to end the context this is the last line to save to file
We can open myfile.txt to view the updated contents.
line 1
line 2
line 3
line 4
line 5
This is another line number 0
This is another line number 1
This is another line number 2
This is another line number 3
This is another line number 4
You can't see me in the console
Adding some more lines to my file
Time to end the context this is the last line to save to file
Matrix to Mathjax Mini-Project
Below is an example of a matrix for readers that aren't familiar with the term or need a refresher. The matrix below is written in MathJax, this is a mark-up language to show mathematical symbols on the internet. You can right click > Show maths as > Tex commands , to see what this looks like.
\(\begin{pmatrix} 1.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 1.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 1.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 1.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 1.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 1.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 1.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 1.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 1.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 0.0 & 1.0 \end{pmatrix} \)
This is a rather very boring and error prone task to write in Mathjax, which really isn't good for the soul. So in this mini project we will create a context manager to convert numpy and scipy matrices to Mathjax to save ourselves some time. We will combine the simple examples we used above to implement this.
So basically we want to be able to convert the following:
import numpy as np
A = np.arange(9).reshape(3,3)
print(A)
'''
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
'''
To Mathjax which will look like :
\begin{pmatrix}
0.0 & 1.0 & 2.0 \\
3.0 & 4.0 & 5.0 \\
6.0 & 7.0 & 8.0 \end{pmatrix}
\(\begin{pmatrix} 0.0 & 1.0 & 2.0 \\ 3.0 & 4.0 & 5.0 \\ 6.0 & 7.0 & 8.0 \end{pmatrix} \)
We want to be able to save this directly to a file along with a message so we can remember what the matrix represents. And allow for easy access when copying from the file to the browser.
If you want to follow along with this section you need numpy and scipy installed, these packages are easy to install with pip install if you haven't got them already.
We will use the LU Decomposition as an example. If you have ever typed this in Mathjax or Latex you will know this is incredibly boring and annoying.
import sys
import numpy as np
import scipy.linalg as la
class MatrixMathJaxWriter:
def __init__(self, file_out):
self.matrix = None
self.rows = None
self.cols = None
self.default_stdout = sys.stdout
self.filename = file_out
def matrix_writer(self, matrix, message):
self.matrix = matrix
self.rows = matrix.shape[0]
self.cols = matrix.shape[1]
print('\n\n')
print('\n\n',message)
print('\\begin{pmatrix}')
for row in range(self.rows):
if row > 0:
print('\\\\')
for col in range(self.cols):
if col != self.cols-1:
print(float(self.matrix[row:row+1, col]), '&', end = ' ')
else:
print(float(self.matrix[row:row+1, col]), end = ' ')
print('\n\end{pmatrix}', end= ' ')
def __enter__(self):
print('entering context',
f'all further prints written to {self.filename}')
self._myfile = open(self.filename, 'a')
sys.stdout = self._myfile
return self
def __exit__(self, exc_type, value, traceback):
sys.stdout = self.default_stdout
self._myfile.close()
print(f'exiting and closing {self.filename} \n\n and setting sys.stdout',
' back to the default')
return False
with MatrixMathJaxWriter('matrixfile.txt') as mat:
A = np.random.randint(-50, 50, size=(5,5))
P, L, U = la.lu(A)
mat.matrix_writer(np.round(P,3), 'This is P')
mat.matrix_writer(np.round(L,3), 'This is L')
mat.matrix_writer(np.round(U,3), 'This is U')
mat.matrix_writer(np.round(A,3), 'This is A')
Notice that all we see when running the script above is:
entering context all further prints written to matrixfile.txt
exiting and closing matrixfile.txt
and setting sys.stdout back to the default
But when we open the matrixfile.txt in our current working directory we will see the following
This is P
\begin{pmatrix}
0.0 & 1.0 & 0.0 & 0.0 & 0.0 \\
0.0 & 0.0 & 0.0 & 0.0 & 1.0 \\
0.0 & 0.0 & 0.0 & 1.0 & 0.0 \\
1.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
0.0 & 0.0 & 1.0 & 0.0 & 0.0
\end{pmatrix}
This is L
\begin{pmatrix}
1.0 & 0.0 & 0.0 & 0.0 & 0.0 \\
-0.816 & 1.0 & 0.0 & 0.0 & 0.0 \\
0.421 & -0.738 & 1.0 & 0.0 & 0.0 \\
0.026 & -0.298 & -0.746 & 1.0 & 0.0 \\
-0.763 & 0.527 & -0.662 & -0.018 & 1.0
\end{pmatrix}
This is U
\begin{pmatrix}
-38.0 & -31.0 & -42.0 & 24.0 & 12.0 \\
0.0 & -73.289 & -40.263 & 58.579 & -22.211 \\
0.0 & 0.0 & -35.011 & 76.098 & -26.433 \\
0.0 & 0.0 & 0.0 & 64.579 & -26.648 \\
0.0 & 0.0 & 0.0 & 0.0 & 22.878
\end{pmatrix}
This is A
\begin{pmatrix}
31.0 & -48.0 & -6.0 & 39.0 & -32.0 \\
29.0 & -15.0 & 34.0 & -39.0 & 20.0 \\
-1.0 & 21.0 & 37.0 & -9.0 & 0.0 \\
-38.0 & -31.0 & -42.0 & 24.0 & 12.0 \\
-16.0 & 41.0 & -23.0 & 43.0 & -5.0
\end{pmatrix}
\(A = PLU\)
\(\begin{pmatrix} 31.0 & -48.0 & -6.0 & 39.0 & -32.0 \\ 29.0 & -15.0 & 34.0 & -39.0 & 20.0 \\ -1.0 & 21.0 & 37.0 & -9.0 & 0.0 \\ -38.0 & -31.0 & -42.0 & 24.0 & 12.0 \\ -16.0 & 41.0 & -23.0 & 43.0 & -5.0 \end{pmatrix} = \begin{pmatrix} 0.0 & 1.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 0.0 & 0.0 & 1.0 \\ 0.0 & 0.0 & 0.0 & 1.0 & 0.0 \\ 1.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ 0.0 & 0.0 & 1.0 & 0.0 & 0.0 \end{pmatrix} \begin{pmatrix} 1.0 & 0.0 & 0.0 & 0.0 & 0.0 \\ -0.816 & 1.0 & 0.0 & 0.0 & 0.0 \\ 0.421 & -0.738 & 1.0 & 0.0 & 0.0 \\ 0.026 & -0.298 & -0.746 & 1.0 & 0.0 \\ -0.763 & 0.527 & -0.662 & -0.018 & 1.0 \end{pmatrix} \begin{pmatrix} -38.0 & -31.0 & -42.0 & 24.0 & 12.0 \\ 0.0 & -73.289 & -40.263 & 58.579 & -22.211 \\ 0.0 & 0.0 & -35.011 & 76.098 & -26.433 \\ 0.0 & 0.0 & 0.0 & 64.579 & -26.648 \\ 0.0 & 0.0 & 0.0 & 0.0 & 22.878 \end{pmatrix} \)
That saved us a lot of time! It would have taken a long tedious 20 minutes + to write that out by hand. With the context manager we could do it very quickly indeed!
Summary
- The __enter__ and __exit__ special methods known as the context manager protocol indicate we are dealing with a context manager.
- Context managers are useless if we want to do something and then immediately clean up after we are finished.
Got an example of how you have used a context manager? Want to share it here? Feel free to email me! It is difficult to think of good examples of using context managers other than to read and write files so some suggestions / real examples would be appreciated.