I've Gone Mad for Mad Libs

It all started so innocently. I had heard so much about the Jinja2 templating engine for network automation, and I decided to come up with a quick presentation to show it off while giving myself an excuse to learn it. In short, Jinja is a templating engine originally used for dynamically rendering HTML in Python that has been adopted by the network automation community to spit out customized configuration flat files.

Due to the diverse background of my audience, I didn't want to have the caucus of DBAs or the gaggle of Unix sysadmins suffer through a Cisco IOS configuration just because that's the format I'm more comfortable with. I came up with the idea of templating Mad Libs as a fun way to level the playing field.

Basics of Jinja2 for Mad Libs

When using Jinja, you have a template file along with your script. The template (saved as template.txt in this example) might look like the below. Variables to be substituted reside within double-curly brackets.

Hello {{ name }}, I'm the Jinja Ninja!

Your code to render and print the text to console would be:

from Jinja2 import Environment, FileSystemLoader

env = Environment(loader=FilesystemLoader(searchpath=/path/to/templates))
jinja_ninja_template = env.get_template('template.txt')

# Two Ways to Format the Rendering. I'll Show Both.
my_name = "Josh"
named_arg_render = jinja_ninja_template.render(name = my_name)

variable_dict = {name: my_name}
dictionary_render = jinja_ninja_template.render(variable_dict)

print(named_arg_render)
# Would print "Hello Josh, I'm the Jinja Ninja!"
print(dictionary_render)
# Would also print "Hello Josh, I'm the Jinja Ninja!"

Advanced Fancy Templating for Mad Libs

I adapted some classic literature to a mad libs format because I didn't want to write things from scratch. Poems especially make some pretty weird and funny mad libs. And my template for Stopping by Woods on a Snowy Evening by Robert Frost allowed me to demonstrate some really interesting bits of Jinja's templating.

Stopping by {{ plural_noun_1|capitalize }} on a {{ noun_3|capitalize ~ 'y' }} Evening

by Robert Frost and {{ your_name }}

Whose {{ plural_noun_1 }} these are I think I know.   
His {{ noun_1 }} is in the {{ noun_2 }} though;   
He will not see me stopping here   
To watch his {{ plural_noun_1 }} fill up with {{ noun_3 }}.   

My little {{ noun_4 }} must think it queer   
To stop without a {{ noun_5 }} near   
Between the {{ plural_noun_1 }} and {{ adjective_1 }} lake   
The {{ superlative_adjective_1 }} evening of the year.   

He gives his {{ noun_6 }} a shake   
To {{ verb_1 }} if there is some mistake.   
The only other sound’s the sweep   
Of {{ adjective_2 }} wind and downy {{ noun_7 }}.   

The {{ plural_noun_1 }} are {{ adjective_3 }}, dark and deep,   
But I have {{ plural_noun_2 }} to keep,
{% for i in range(2) %}   
And {{ plural_noun_3 }} to {{ verb_2 }} before I {{ verb_3 }}{% if loop.last %}.{% else %},{% endif %}
{% endfor %}
  • Whereas you might do something like string_variable_name.capitalize() in Python to capitalize a string, you use the pipe operator to perform "filtering" in the template.
  • String concatenation is done via the tilde ~
  • Looping is absolutely allowed using typical Python for loops. Code blocks in the template use the {% %} format. There is a dictionary called loop that is exposed in a for loop (documentation here) to summarize useful information about the loop.
  • You can use part of that loop dictionary as a condition in an if statement. Notice in the last line of the template how I have an if/else block nested inside the for loop. Its entire purpose is to determine whether to put a comma or period at the end of the line that repeats.
  • The documentation and error handling for Jinja is incredible. The official documentation website and a little experimentation were completely sufficient to work through everything in this project.

What if You Don't Know What Variables Are In the Template?

To create a generic script to interpret the Jinja templates, it needs to be able to handle a varying number of verbs, nouns, and other parts of speech, as well as more unusual stuff like proper nouns, collections of words that rhyme, or whatever else is there. So, we need to make our script discover what's in the template. Here's how:

from Jinja2 import Environment, FileSystemLoader, meta

env = Environment(loader = FileSystemLoader(searchpath = '/path/to/templates'))

# This is the part where we figure out what we need
template_src = env.loader.get_source(template)
template_parsed = env.parse(template_src)
parts_of_speech = meta.find_undeclared_variables(template_parsed) # Returns a set
user_prompts = list(parts_of_speech) # Lists iterate faster than sets
try:
    # If you use range() to make a for loop,
    # Jinja2 will return 'range' as an
    # undeclared variable. You have to ignore it.
    user_prompts.remove("range")
except ValueError:
    pass

# This is where we prompt the user
dictionary_for_rendering = {}
for prompt in user_prompts:
    dictionary_for_rendering[prompt] = input(f'{prompt}: ')

# And now we render
template = env.get_template(template)
print(template.render(dictionary_for_rendering)

This Was The Initial Presentation

I successfully presented a command line script that would discover variables in a Jinja template, prompt the user to fill them in, and display the result. There was wild applause in my own head.

How Did This Get Out Of Hand?

A few things happened to transform this effort from a fun and silly presentation to a truly wacky side project. First, I brought my laptop to a coffee shop one Saturday morning and everyone around me got really excited to screw up classic literature via random word choices. Second, I completed my fireplace project and learned how truly easy Flask is to use. Third, I began planning a presentation that would require some sample network data (working title is Did the Infrastructure or the Application Mess Up: Fault Domain Isolation Through Packet Capture Timing Analysis).

Let's Adapt This For A Browser

This was the perfect opportunity to use Jinja for what it was actually designed for: dynamically rendered HTML!

I knew I wanted three webpages: one to present the story options, one to prompt the user for the parts of speech, and one to show the result. This was plenty to get the stub going:

from flask import Flask, request
from jinja2 import Environment, FileSystemLoader

app = Flask(__name__) # the application needs to be a global statement
web_env = Environment(
            loader = FileSystemLoader(searchpath = r'./html_templates')
        )
story_env = Environment(
            loader = FileSystemLoader(searchpath = r'./story_templates')
        )

@app.route('/')
def home_page():
    pass

@app.route('/show_form')
def madlib_form():
    pass

@app.route('/show_output', methods=['POST'])
def present_madlib():
    pass

if __name__ == '__main__':
    app.run(debug = True) # debug mode dynamically reloads the web server upon code changes

Because I wanted this to become a little more appropriate for the real world, I decided to change the undeclared variable search to just a list in a file. On an actual web server, it would save a decent amount of processing, comparatively. So, I offloaded all the story attributes to a JSON file.

{
    "stories": [
        {
            "id": 1,
            "name": "The Woods",
            "template": "woods.txt",
            "portrait": "/portraits/robert_frost.jpg",
            "attributes": [
                {"type": "Your Name", "var_name": "your_name"},
                {"type": "Plural Noun", "var_name": "plural_noun_1"},
                {"type": "Plural Noun", "var_name": "plural_noun_2"},
                {"type": "Plural Noun", "var_name": "plural_noun_3"},
                {"type": "Noun", "var_name": "noun_1"},
                {"type": "Noun", "var_name": "noun_2"},
                {"type": "Noun", "var_name": "noun_3"},
                {"type": "Noun", "var_name": "noun_4"},
                {"type": "Noun", "var_name": "noun_5"},
                {"type": "Noun", "var_name": "noun_6"},
                {"type": "Noun", "var_name": "noun_7"},
                {"type": "Adjective", "var_name": "adjective_1"},
                {"type": "Adjective", "var_name": "adjective_2"},
                {"type": "Adjective", "var_name": "adjective_3"},
                {"type": "Superlative Adjective", "var_name": "superlative_adjective_1"},
                {"type": "Verb", "var_name": "verb_1"},
                {"type": "Verb", "var_name": "verb_2"},
                {"type": "Verb", "var_name": "verb_3"}
            ]
        }
    ]
}

Don't worry, we'll get to that portraits value a little further in the descent into madness. For now, this gives us plenty of information to start rendering some web pages.

def home_page():
    home_page_template = web_env.get_template('home.html')
    stories = madlibs_helpers.get_story_list() 
    # I offloaded local file operations to a helper file.
    # If in the future I take this from JSON files to a
    # database, there's no need to mess with the Flask code.
    return home_page_template.render(story_list = stories)

This then renders the following HTML templates:

home.html

{% extends "base.html" %}
{% block body %}
    <h2>Choose from the following mad libs!</h2>
    {% for story in story_list %}
    <p><a href="/show_form?id={{ story.id }}">{{ story.name }}</a></p>
    {% endfor %}
{% endblock %}

base.html - Notice how I fill in the body block in home.html

<!DOCTYPE html>
<head>
    {% block head %}
    <title>Josh's Mad Libs!</title>
    {% endblock %}
</head>
<body>
    {% block body %}
    {% endblock %}
</body>

The other two pages went just as easily, and I was able to start capturing data for my other presentation.

Not Enough Data

The problem with making a really easy, lightweight web application is that when you need to use it to demonstrate issues that can happen with slightly heavier web applications, it doesn't generate enough traffic. I didn't even fill full packets with those HTML responses!

So, what takes a decent amount of data? Pictures! Why not put a picture of the original author of these selections of classic literature along with the filled-in mad lib? Why not find pictures where the author looks truly disappointed at how the user has ruined their life's work?

thoreau_disappointed.PNG

That's where that portrait attribute comes into play.

Where Are We Now?

So now, I have a web application that generates a decent amount of traffic that I can use to model network issues, server issues, backend access issues, and other things for my timing analysis presentation. And since I put so much effort into it, I decided I should just release it to the world!

I stuck it on a lightweight Droplet from DigitalOcean. You can check it out at madlibs.je-clark.com. All the code (minus some config work to get things going on the Droplet) is on Github.

Feel free to let me know if there are any other selections from classic literature that would make good (or truly horrible) Mad Libs. I've discovered that things between 100 and 150 words work pretty well.

Have fun! I know I have.