Flow Control with Exceptions in Python

My programming education in college was very computer engineery, and not at all computer sciency. Instead of "Hello, World", we started with logic gates and MOSFETs. We learned how to make single purpose circuits, and then how to design an instruction set to combine those circuits to make a programmable circuit. After writing simple programs in machine code in a simplified Instruction Set Architecture, assembly language was remarkably readable. When we started learning C, a good portion of the education was compiling simple functions to the representative assembly.

While this curriculum was very well suited to the embedded systems, digital logic design, and microprocessor design courses that followed, it wasn't the best preparation for building modern applications. I had a difficult time ignoring the layers of abstraction between the Java code I was writing and the resulting machine code. I spent so much time trying to rationalize the abstraction of simple built-in methods like String.toUpper() that I couldn't code productively. In fact, it took years for me to understand that I had fundamental misunderstandings about some features of higher level languages.

I vividly remember the lightbulb moment I had when I realized the utility of exceptions. Until that moment, there wasn't any difference between an exception and a segfault in my mind. When an exception isn't handled, it causes the program to halt, just like a segfault. I spent a lot of time and CPU cycles coding around exceptions. During that lightbulb moment, I was writing a function in Powershell to make sure a given file was available to edit. I spent a couple hours trying to figure out what flag or attribute existed to tell me if I could use it or not, and finally found a StackOverflow post that told me to just try to edit it and deal with the exception if I couldn't. Struggling through that function finally taught me that exceptions are not errors; exceptions are flow control.

Learning How and When

Abraham Maslow popularized the phrase "if all you have is a hammer, everything looks like a nail." His argument, when using that phrase, was for methodological innovation in psychology research. Novel problems must be researched with novel approaches. But he acknowledges that the learning process to find the correct approach is inelegant. "What one mostly learns from such first efforts is how it should be done better the next time."

In the learning process, most of the work is learning to recognize a situation that a particular approach is well suited for, and I think an excellent way to do that is to treat that approach as the only option. In rock climbing, I refined my heel hook technique by using that for every single move until I understood when it would help and when it would hinder. In programming, I challenged myself to write a functional command line tool without a single if statement.

Exceptions in Python

Exceptions in Python and other higher level languages are really just the language trying to tell you that something unexpected happened, and they give you the opportunity to choose how to manage that unexpected thing. For example, something as simple as opening a file carries a bunch of underlying assumptions. The file must exist, the program must have the proper permissions to read the file, and the program must be able to access the directory the file is in (it's tough to get to the corporate network drive when you aren't on the VPN). Depending on which of those requirements is satisfied, you may want to handle things differently. If the program can't access the directory, maybe you want to doublecheck that there aren't any spelling errors in the file path. If the file doesn't exist, maybe you want to create it. Exceptions gives you, the programmer, options to deal with those problems rather than allowing the program to just quit.

There are really only 3 elements of Python I relied on to replace if statements:

try

A try block defines a section of code where an exception might occur. It's common practice to use these only at the edges of a program, or where it interacts with something potentially unreliable like a file, a network connection, or a user.

try:
    do_thing_that_might_fail_weirdly()

except

An except block is the complement to a try block, like an else is to an if.

try:
    do_thing_that_might_fail_weirdly()
except:
    print("The thing failed weirdly")

Excepts can also look for specific exceptions. These can be defined by the programmer or just built in.

except WeirdFailure:
    print("The thing failed weirdly")

Excepts can also be chained together off a single try block, searching for different errors.

try:
    do_thing_that_might_fail_weirdly()
except WeirdFailure:
    print("The thing did the weird thing")
except OtherWeirdFailure:
    print("The thing did the other weird thing")
except:
    print("This block catches every other exception")

assert

This was my big cheating move. Assert statements are usually used in tests to make sure things are as you expect. In this coding challenge, I heavily relied on assert statements to get around conditional statements.

try:
    assert "show" == user_input # Hammer, meet scrambled eggs
except AssertionError:
    pass

TODO List Manager Structure

The user experience I wanted for this todo list manager was a captive command line interface. I wanted to launch the program by choosing one of many different lists and then interacting with only that chosen list. I wanted to make sure I could check items off the list, give myself notes, and support sub-tasks.

Launching the Program

I used the argparse library in Python to automatically define command line flags and a help dialog:

def initiate():
    description = "Manage all your TODOs with this overcomplicated application!"
    parser = argparse.ArgumentParser(description=description)
    parser.add_argument('todo_path', help="File path to the todo file you want to interact with.")

    args = parser.parse_args()
    return args.todo_path

Opening that todo file and reading in the JSON gave me the first opportunity to use the hammer I chose for this exercise. I needed to check two things: that the file exists, and that it is a valid JSON file. The function below will do one of three things: return the dictionary representation of the JSON file passed into it, return a blank dictionary, or close the program.

def read_todo_file(file_path):
    try:
        with open(file_path) as todo_file:
            todo = json.load(todo_file)
    except json.JSONDecodeError:
        # We don't want to corrupt a file that is in the wrong format.
        print(f"{file_path} exists, but is not a valid JSON file. Exiting")
        sys.exit()
    except:
        # If the file doesn't exist, then we create a new file
        print(f"{file_path} does not exist, starting a new todo list.")
        todo = {}
    finally:
        return todo

Once the todo file is read into memory, we enter the user interface.

User Interface

I chose a very simple user interface; it's just >. Once a user enters a command, it runs through a set of expected commands, and has a catch-all clause to handle gibberish or misspellings. If you look carefully, you'll see that it's recursive, but with no way to get back up the layers of recursion. It's a dumb memory leak, but that wasn't the point of this challenge.

def user_interface(todo, file_path):

    user_input = input("> ")

    try:
        try:
            assert "show" == user_input.lower()
            show(todo, file_path)
        except AssertionError:
            pass
        try:
            assert "show closed" == user_input.lower()
            show_closed(todo, file_path)
        except AssertionError:
            pass
        try:
            cmd = "show task "
            assert cmd in user_input.lower()
            show_task(todo, user_input.lower().replace(cmd,''), file_path)
        except AssertionError:
            pass
        try:
            cmd = "add notes "
            assert cmd in user_input.lower()
            add_notes(todo, user_input.lower().replace(cmd,''), file_path)
        except AssertionError:
            pass
        try:
            cmd = "add subtask "
            assert cmd in user_input.lower()
            add_subtask(todo,user_input.lower().replace(cmd,''), file_path)
        except AssertionError:
            pass
        try:
            cmd = "add "
            assert cmd in user_input.lower()
            add_task(todo, user_input.lower().replace(cmd,''), file_path)
        except AssertionError:
            pass
        try:
            cmd = "close "
            assert cmd in user_input.lower()
            close_task(todo, user_input.lower().replace(cmd,''), file_path)
        except AssertionError:
            pass
        try:
            assert "exit" == user_input.lower()
            exit(todo, file_path)
        except AssertionError:
            pass
        try:
            assert "help" == user_input.lower()
            help(todo, file_path)
        except AssertionError:
            pass
        raise SyntaxError
    except SyntaxError:
        print("Syntax Error")
        user_interface(todo, file_path)

One interesting thing I want to point out is that at the end of the upper level try block, I put in raise SyntaxError. SyntaxError is a built-in exception, and I'm manually throwing that exception because it's the best way for me to express that the user didn't type what I wanted them to. The except block at the end will deal with that by means of my horrible recursion.

Using Dictionary Exceptions

I defined the "show" command to list all open tasks. In the JSON schema, an open task is defined as a task that does not have a "completed_timestamp" element. Therefore, I can try to read that timestamp and put the code I care about in the except block.

def show(todo, file_path):
    for task in todo.keys():
        try:
            complete = todo[task]["completed_timestamp"]
        except KeyError:
            print(task)

    user_interface(todo, file_path)

The traditional way to write that would be something like the function below. I mostly turned that not in conditional phrase into what I'll call an "intentionally subverted expectation."

def show(todo):
    for task in todo.keys():
        if "completed_timestamp" not in task.keys():
            print(task)

I am particularly proud of the way I close tasks. Just like when reading in the original todo file, there were three cases to deal with: the task is open, the task is already closed, and the task doesn't exist. First, I try to get the completion timestamp. My expectation is that does not exist and the program will throw a KeyError. However, the program will also throw a KeyError if task_name doesn't exist, so I need another try/except block inside the upper except block. Inside that, I write a line that will throw a KeyError only if the task does not exist. And once we get through all those potential exceptions, we take the useful action of creating the completed_timestamp key and giving it a value.

If the original statement executes without any exception, I know that the task is already closed. Therefore, I raise a ValueError to handle that case.

def close_task(todo, task_name, file_path):
    try:
        complete = todo[task_name]["completed_timestamp"]
        raise ValueError
    except ValueError:
        # Ironicaly, the error is that there is a value
        print("Task already closed. Nothing else to do")
    except KeyError:
        try:
            task = todo[task_name]
            todo[task_name]["completed_timestamp"] = str(datetime.now())
        except KeyError:
            print("No task found")

    user_interface(todo, file_path)

Conclusion

I will never use this program for any of my todo lists. I will never use most of these techniques for any useful code. But I did treat exceptions as the hammer that would solve all of my problems and found the truly useful cases. The best case for using exceptions in this program is reading in the original todo file. I can't think of a more efficient or more readable way to make sure that todo file exists and is a valid JSON file.

As usual, the complete code for this post is available on Github.