7  Files and Exceptions

7.1 Reading from a File

Now that you’ve mastered the basic skills you need to write organized programs that are easy to use, it’s time to think about making your programs even more relevant and usable.

An incredible amount of data is available in text files. Text files can contain weather data, traffic data, socioeconomic data, literary works, and more. Reading from a file is particularly useful in data analysis applications, but it’s also applicable to any situation in which you want to analyze or modify information stored in a file.

7.1.1 Reading the Contents of a File

To begin, we need a file with a few lines of text in it. Let’s start with a file that contains pi to 30 decimal places:

%%writefile pi_digits.txt
3.1415926535
  8979323846
  2643383279

Here’s a program that opens this file, reads it, and prints the contents of the file to the screen:

file_object = open('pi_digits.txt')
print(file_object.read())
file_object.close()

3.1415926535
  8979323846
  2643383279

To do any work with a file, even just printing its contents, you first need to open the file to access it. The open() function needs one argument: the name of the file you want to open. Python looks for this file in the directory where the program that’s currently being executed is stored. The open() function returns an object representing the file. Here, open('pi_digits.txt') returns an object representing pi_digits.txt. Python assigns this object to file_object, which we’ll work with later in the program.

# A recommended way:
with open('pi_digits.txt') as file_object: #  file_object = open('pi_digits.txt')
    contents = file_object.read()          #  We do not have to call file_object.close()
print(contents.strip())

3.1415926535
  8979323846
  2643383279

The keyword with closes the file once access to it is no longer needed. Notice how we call open() in this program but not close(). You could open and close the file by calling open() and close(), but if a bug in your program prevents the close() method from being executed, the file may never close! This may seem trivial, but improperly closed files can cause data to be lost or corrupted.

Once we have a file object representing pi_digits.txt, we use the read() method in the second line of our program to read the entire contents of the file and store it as one long string in contents. Since read() returns an empty string when it reaches the end of the file; this empty string shows up as a blank line. If you want to remove the extra blank line, you can use strip() in the call to print().

It’s not always easy to know exactly when you should close a file, but with the structure shown here, Python will figure that out for you. All you have to do is open the file and work with it as desired, trusting that Python will close it automatically when the with block finishes execution.

7.1.2 File Paths

Sometimes, depending on how you organize your work, the file you want to open won’t be in the same directory as your program file. To get Python to open files from a directory other than the one where your program file is stored, you need to provide a file path , which tells Python to look in a specific location on your system. A relative file path will tell Python to look for a given location relative to the directory where the currently running program file is stored. For example, you’d write:

with open('text_files/filename.txt') as file_object:

This line tells Python to look for the desired .txt file in the folder text_files and assumes that text_files is located in the current directory.

Windows systems use a backslash (\) instead of a forward slash (/) when displaying file paths, but you can still use forward slashes in your code.

!mkdir text_files

%%writefile text_files/pi_digits2.txt
3.1415926535
  8979323846
  2643383279

Writing text_files/pi_digits2.txt

with open('text_files/pi_digits2.txt') as file_object:
    contents = file_object.read()
print(contents.strip())

3.1415926535
  8979323846
  2643383279

You can also tell Python exactly where the file is on your computer regardless of where the program that’s being executed is stored. This is called an absolute file path. Absolute paths are usually longer than relative paths, so it’s helpful to assign them to a variable and then pass that variable to open():

file_path = '/home/phonchi/other_files/text_files/filename.txt'
with open(file_path) as file_object:

7.1.3 Reading Line by Line

When you’re reading a file, you’ll often want to examine each line of the file. You might be looking for certain information in the file, or you might want to modify the text in the file in some way. You can use a for loop on the file object to examine each line from a file one at a time:

filename = 'pi_digits.txt'
with open(filename) as file_object: # file_object is also iterable!
    for line in file_object:
        print(line.strip())

3.1415926535
8979323846
2643383279

To examine the file’s contents, we work through each line in the file by looping over the file object. Since there is a newline in each line of file and the print function adds its own newline each time we call it, so we will end up with two newline characters at the end of each line: one from the file and one from print(). Using strip() on each line in the print() call eliminates these extra blank lines.

7.1.4 Making a List of Lines from a File

When you use with, the file object returned by open() is only available inside the with block that contains it. If you want to retain access to a file’s contents outside the with block, you can store the file’s lines in a list inside the block and then work with that list!

filename = 'pi_digits.txt'
with open(filename) as file_object:
    lines = file_object.readlines()

print(lines) # List of strings
pi_string = ''
for line in lines:
    pi_string += line.strip()
    
print(pi_string)
print(len(pi_string)) # The string is 32 characters long because it also includes the leading 3 and a decimal point

['3.1415926535\n', '  8979323846\n', '  2643383279\n']
3.141592653589793238462643383279
32

The readlines() method takes each line from the file and stores it in a list. This list is then assigned to lines, which we can continue to work with after the with block ends. Here, we create a variable, pi_string, to hold the digits of pi. We then create a loop that adds each line of digits to pi_string and removes the newline character from each line.

When Python reads from a text file, it interprets all text in the file as a string. If you read in a number and want to work with that value in a numerical context, you’ll have to convert it to an integer using the int() function or convert it to a float using the float() function.

7.1.5 Writing to a File

One of the simplest ways to save data is to write it to a file. When you write text to a file, the output will still be available after you close the terminal containing your program’s output. You can examine output after a program finishes running, and you can share the output files with others as well. You can also write programs that read the text back into memory and work with it again later!

7.1.5.1 Writing to an Empty File

To write text to a file, you need to call open() with a second argument telling Python that you want to write to the file

filename = 'programming.txt'

with open(filename, 'w') as file_object:
    file_object.write("We love programming!")

The call to open() in this example has two arguments. The first argument is still the name of the file we want to open. The second argument, ‘w’, tells Python that we want to open the file in write mode. You can open a file in read mode (‘r’), write mode (‘w’), append mode (‘a’), or a mode that allows you to read and write to the file ( ‘r+’). If you omit the mode argument, Python opens the file in read-only mode by default. Here, we use the write() method on the file object to write a string to the file.

Python can only write strings to a text file. If you want to store numerical data in a text file, you’ll have to convert the data to string format first using the str() function.

The open() function automatically creates the file you’re writing to if it doesn’t already exist. However, be careful opening a file in write mode (‘w’) because if the file does exist, Python will erase the contents of the file before returning the file object.

with open(filename, 'w') as file_object:
    print("12")

12

While reading through a file, the system maintains a file-position pointer(index) representing the location of the next character to read. Therefore, the following code snippet will allow you to append it to the end of the file.

year = 2023

with open(filename, 'r+') as file_object:
    spam = file_object.readlines()
    print(file_object.tell())
    print(len("We love programming!"))
    file_object.write(str(year))

20
20

The tell() function will return the current position of the file-position pointer. We can also use seek() to change the position. Checkout more details about file-position pointer here.

7.1.6 Appending to a File

If you want to add content to a file instead of writing over existing content, you can also open the file in append mode. When you open a file in append mode, Python doesn’t erase the contents of the file before returning the file object. Any lines you write to the file will be added at the end of the file. If the file doesn’t exist yet, Python will create an empty file for you.

filename = 'programming.txt'

with open(filename, 'a') as file_object:
    file_object.write("\nWe also love finding meaning in large datasets.\n")
    file_object.write("We love creating apps that can run in a browser.\n")

The write() function doesn’t add any newlines to the text you write. So we need to add newline characters if we would like to. There is also a writelines() function that can write list of strings into files.

7.2 Storing Data

Many of your programs will ask users to input certain kinds of information. You might allow users to store preferences in a game or provide data for visualization. Whatever the focus of your program is, you’ll store the information users provide in data structures such as lists and dictionaries. When users close a program, you’ll almost always want to save the information they entered. A simple way to do this involves storing your data using the json module.

The json module allows you to dump simple Python data structures into a file and load the data from that file the next time the program runs. You can also use json to share data between different programming languages. It’s a useful and portable format.

7.2.0.1 Using json.dump() and json.load()

The json.dump() function takes two arguments: a piece of data to store and a file object it can use to store the data.

import json

numbers = [2, 3, 5, 7, 11, 13]
filename = 'numbers.json'

with open(filename, 'w') as f:
    json.dump(numbers, f)

Now we’ll write a program that uses json.load() to read the list back into memory

filename = 'numbers.json'
with open(filename) as f:
    numbers = json.load(f)
    
print(numbers)

[2, 3, 5, 7, 11, 13]

7.3 Exceptions

Python uses special objects called exceptions to manage errors that arise during a program’s execution. Whenever an error makes Python unsure of what to do next, it creates an exception object. If we write code that handles the exception, the program will continue running. If you don’t handle the exception, the program will halt and show a traceback, which includes a report of the exception that was raised.

Exceptions are handled with try-except blocks. A try-except block tells Python what to do if an exception is raised. When we use try-except blocks, our programs will continue running even if things go wrong. Instead of tracebacks, which can be confusing for users to read, users will see friendly error messages that we write!

7.3.1 Handling the ZeroDivisionError Exception

print(5/0)

ZeroDivisionError: division by zero

The error reported at the first line in the traceback, ZeroDivisionError, is an exception object. Python creates this kind of object in response to a situation where it can’t do what we ask. When this happens, Python stops the program and tells us the kind of exception that was raised. We can use this information to modify our program.

When we think an error may occur, you can write a try-except block to handle the exception that might be raised. We tell Python to try running some code and tell it what to do if the code results in a particular kind of exception.

try:
    print(5/0)
except ZeroDivisionError:
    print("You can't divide by zero!")

You can't divide by zero!

We put print(5/0), the line that caused the error, inside a try block. If the code in a try block works, Python skips over the except block. If the code in the try block causes an error, Python looks for an except block whose error matches the one that was raised and ran the code in that block. In this example, the user sees a friendly error message instead of a traceback.

try:
    print(5/0)
except:
    print("Exceptions occur!")

Exceptions occur!

If you do not add any exception type, it will capture all exceptions!

7.3.2 Using Exceptions to Prevent Crashes

Handling errors correctly is especially important when the program has more work to do after the error occurs. Let’s create a simple calculator that does only division:

%%writefile division_calculator.py
print("Give me two numbers, and I'll divide them.")
print("Enter 'q' to quit.")

while True:
    first_number = input("\nFirst number: ")
    if first_number == 'q':
        break
    second_number = input("Second number: ")
    if second_number == 'q':
        break
    try:
        answer = int(first_number) / int(second_number)
    except ZeroDivisionError:
        print("You can't divide by 0!")
    else: # Only executed if try block is succeed
        print(answer)
    finally: # Always executed
        print("\nGive me two numbers, and I'll divide them.")
        print("Enter 'q' to quit.")

Overwriting division_calculator.py

Here, the error may occur on the line that performs the division, so that’s where we’ll put the try-except block. This example also includes an else block. Any code that depends on the try block executing successfully goes into the else block. In addition, the finally clause is guaranteed to execute, regardless of whether its try suite executes successfully or an exception occurs.

We ask Python to try to complete the division operation in a try block, which includes only the code that might cause an error. The program continues to run, and the user never sees a traceback.

7.3.3 Handling the FileNotFoundError Exception

One common issue when working with files is handling missing files. The file you’re looking for might be in a different location, the filename may be misspelled, or the file may not exist at all

filename = 'alice.txt'
with open(filename) as f: # Note it is in read mode
    contents = f.read()

FileNotFoundError: [Errno 2] No such file or directory: 'alice.txt'

filename = 'alice.txt'
try:
    with open(filename) as f:
        contents = f.read()
except FileNotFoundError:
    print(f"Sorry, the file {filename} does not exist.")

Sorry, the file alice.txt does not exist.

In this example, the code in the try block produces a FileNotFoundError, so Python looks for an except block that matches that error. Python then runs the code in that block, and the result is a friendly error message instead of a traceback.

7.3.4 Exercise 1: Assuming we are designing a word game called “The Mysterious Island” and we need to load the statistics of the player and enemies each time the game begins. Try to complete the following functions load_data() and save_data() so that you can load the JSON file if it does not exist using the exception handling techniques you just learned.

import random
import time

def slow_print(text, delay=0.05):
    for char in text:
        print(char, end='', flush=True)
        time.sleep(delay)
    print()

def battle(player, enemy):
    slow_print(f"You encounter a {enemy['name']} with {enemy['hp']} HP!")
    while player['hp'] > 0 and enemy['hp'] > 0:
        choice = input("Do you want to attack or escape? (a/e): ")
        if choice.lower() == 'a':
            player_damage = max(random.randint(player['attack']//2, player['attack']), 1)
            enemy_damage = max(random.randint(enemy['attack']//2, enemy['attack']), 1)
            slow_print(f"You deal {player_damage} damage to the {enemy['name']}!")
            enemy['hp'] -= player_damage
            if enemy['hp'] <= 0:
                break
            slow_print(f"The {enemy['name']} deals {enemy_damage} damage to you!")
            player['hp'] -= enemy_damage
        elif choice.lower() == 'e':
            escape_chance = random.randint(1, 10)
            if escape_chance <= 2:
                slow_print("You successfully escape from the battle!")
                return
            else:
                slow_print("You failed to escape!")
                enemy_damage = max(random.randint(enemy['attack']//2, enemy['attack']), 1)
                slow_print(f"The {enemy['name']} deals {enemy_damage} damage to you!")
                player['hp'] -= enemy_damage
        else:
            slow_print("Invalid choice! Try again.")
    if player['hp'] <= 0:
        slow_print("You were defeated!")
        print("Game over!")
        return False
    else:
        slow_print(f"You defeated the {enemy['name']}!")

def load_data():
    # Perform exception handling, if this is the first time you load the game
    _____:
        with open('game_data.json', 'r') as f:
            data = ____________________ # Read the data here using load()
    __________________________:
        data = {
            "player": {
                "name": "Player",
                "hp": 50,
                "attack": 10
            },
            "enemies": [
                {
                    "name": "Slime",
                    "hp": 10,
                    "attack": 5
                },
                {
                    "name": "Goblin",
                    "hp": 25,
                    "attack": 8
                },
                {
                    "name": "Dragon",
                    "hp": 80,
                    "attack": 15
                }
            ]
        }
        save_data(data)
    return data

def save_data(data):
    with open('game_data.json', 'w') as f:
        ______________# Save the data so that you can play again using dump()

data = load_data()
player = data['player']
enemies = data['enemies']
flag = True

for enemy in enemies:
    flag = battle(player, enemy)

if flag != False: 
    print("Congratulations!")

You encounter a Slime with 10 HP!
You deal 7 damage to the Slime!
The Slime deals 2 damage to you!
Invalid choice! Try again.
You deal 9 damage to the Slime!
You defeated the Slime!
You encounter a Goblin with 25 HP!
You deal 6 damage to the Goblin!
The Goblin deals 8 damage to you!
You deal 7 damage to the Goblin!
The Goblin deals 5 damage to you!
You deal 7 damage to the Goblin!
The Goblin deals 4 damage to you!
You deal 10 damage to the Goblin!
You defeated the Goblin!
You encounter a Dragon with 80 HP!
You failed to escape!
The Dragon deals 13 damage to you!
You failed to escape!
The Dragon deals 9 damage to you!
You failed to escape!
The Dragon deals 15 damage to you!
You were defeated!
Game over!

In this chapter, you learned how to work with files. You learned to read an entire file at once and read through a file’s contents one line at a time. You learned to write to a file and append text onto the end of a file. You learned how to store Python data structures so you can save information your users provide, preventing them from having to start over each time they run a program. Finally, You read about exceptions and how to handle the exceptions you’re likely to see in your programs.