Session 05


MVC Applications

Wherin we learn about the Model View Controller approach to app design and explore data persistence in Python.

By Alan Evangelista (Own work) [CC0], via Wikimedia Commons

Separation of Concerns

In the first part of this course, you were introduced to the concept of Object Oriented Programming

OOP was first formalized in the 1970s in Smalltalk, invented by Alan Kay at Xerox PARC

Smalltalk was also the first language which utilized the Model View Controller design pattern.

This pattern (like all design patterns) seeks to provide a way of thinking that helps to make software design easier.

In this case, the goal is to help clarify the high-level separation of concerns in a system.

Three Components

The pattern divides the elements of a system into three parts:

This component represents the data that comprises the system, and the logic used to manipulate that data.

This component can be any representation of the data to the outside world: a chart, diagram, table, user interface, etc.

It also includes representations of the actions available in the system.


This component coordinates the Model and the View in a system.

It accepts input from a user and channels that input into the Model.

It accepts information about the current state of the Model and transmits that information to the View.

On the Web

This pattern has proven useful for thinking about the applications we build for the web.

A web browser provides a convenient container for views of data.

These views are created by controller software hosted on a server.

This controller software accepts input from users via HTTP requests, channeling it into a data model, often stored in some database.

The controller returns information about the state of the data model to the user via HTTP responses

This approach is so common, that it has been formalized into any number of web frameworks

Web frameworks abstract away the specifics of the HTTP request/response cycle, leaving simple MVC components for the developer to use.

Web frameworks exist in nearly all modern languages.

Python has scores of them.

Over the weeks to come, we’ll learn about two of them, Pyramid and Django.

A Word About Terminology

Although the MVC pattern is a useful abstraction, there are a few differences in how things are named in Python web frameworks

model <–> model

controller <–> view

view <–> template (or even HTTP response)

For more on this difference, you can read this from the Pyramid design documentation.

Our First Application

But enough abstract blabbering.

There’s no better way to make concepts like these concrete than to build something using them.

Let’s make an application!

We’re going to build a Learning Journal.

When we’re done, you’ll have a live, online application you can use to keep note of the things you are learning about Python development.

We’ll use one of our Python web framework to do this: Pyramid


First published in 2010, Pyramid is a powerful, flexible web framework.

You can create compelling one-page applications, much like in microframeworks like Flask

You can also create powerful, scalable applications using the full power of Python

Created by the combined powers of the teams behind Pylons and Zope

It represents the first true second-generation web framework in existence.

Starting the Project

The first step is to prepare for the project.

Begin by creating a location where you’ll do your work.

I generally put all my work in a folder called projects in my home directory:

$ cd
$ mkdir projects
$ cd projects
$ mkdir learning-journal
$ cd learning-journal
$ pwd

We continue our preparations by creating the virtual environment we will use for our project.

Again, this will help us to keep our work here isolated from anything else we do.

Remember how to make a new venv?

$ pyvenv ljenv
c:\Temp>python -m venv myenv

And then, how to activate it?

$ source ljenv/bin/activate
C:> ljenv/Scripts/activate.bat

Next, we install the Pyramid web framework into our new virtualenv.

We can do this with the pip in our active ljenv:

(ljenv)$ pip install pyramid
Collecting pyramid
  Downloading pyramid-1.5.2-py2.py3-none-any.whl (545kB)
    100% |################################| 548kB 172kB/s
Successfully installed PasteDeploy-1.5.2 WebOb-1.4
pyramid-1.5.2 repoze.lru-0.6 translationstring-1.3
venusian-1.0 zope.deprecation-4.1.1 zope.interface-4.1.2

Once that is complete, we are ready to create a scaffold for our project.

Working with Pyramid

Many web frameworks require at least a bit of boilerplate code to get started.

Pyramid does not.

However, our application will require a database and handling that does require some.

Pyramid provides a system for creating boilerplate called pcreate.

You use it to generate the skeleton for a project based on some pattern:

(ljenv)$ pcreate -s alchemy learning_journal
Creating directory /Users/cewing/projects/learning-journal/learning_journal
Welcome to Pyramid.  Sorry for the convenience.

Let’s take a quick look at what that did

├── development.ini
├── learning_journal
│   ├──
│   ├──
│   ├── scripts
│   │   ├──
│   │   └──
│   ├── static
│   ├── templates
│   │   └──
│   ├──
│   └──
├── production.ini

You’ve now created something worth saving.

Start by initializing a new git repository in the learning_journal folder you just created:

(ljenv)$ cd learning_journal
(ljenv)$ git init
Initialized empty Git repository in

Check git status to see where things stand:

(ljenv)$ git status
On branch master

Initial commit

Untracked files:
  (use "git add <file>..." to include in what will be committed)


Add your work to this new repository:

(ljenv)$ git add .
(ljenv)$ git status
Changes to be committed:
  (use "git rm --cached <file>..." to unstage)

    new file:   CHANGES.txt
    new file:
    new file:   production.ini
    new file:

Python creates .pyc files when it executes your code.

There are many other files you don’t want or need in your repository

You can ignore this in git with the .gitignore file.

Create one now, in this same directory, and add the following basic lines:


Finally, add this new file to your repository, too.

(ljenv)$ git add .gitignore

To preserve all these changes, you’ll need to commit what you’ve done:

(ljenv)$ git commit -m "initial commit of the Pyramid learning journal"

This will make a first commit here in this local repository.

For homework, you’ll put this into GitHub, but this is enough for now.

Let’s move on to learning about what we’ve built so far.

When you ran the pcreate command, a new folder was created: learning_journal.

This folder contains your project.

At the top level, you have configuration (.ini files)

You also have a file called

This file turns this collection of Python code and configuration into an installable Python distribution

Let’s take a moment to look over the code in that file

from setuptools import setup, find_packages
requires = [
    ... # packages on which this software depends (dependencies)
      ... # package metadata (used by PyPI)
      # Entry points are ways that we can run our code once installed
      main = learning_journal:main
      initialize_learning_journal_db = learning_journal.scripts.initializedb:main

Pyramid is Python

In the file of your app package, you’ll find a main function:

def main(global_config, **settings):
    """ This function returns a Pyramid WSGI application.
    engine = engine_from_config(settings, 'sqlalchemy.')
    Base.metadata.bind = engine
    config = Configurator(settings=settings)
    config.add_static_view('static', 'static', cache_max_age=3600)
    config.add_route('home', '/')
    return config.make_wsgi_app()

Let’s take a closer look at this, line by line.

def main(global_config, **settings):

Configuration is passed in to an application after being read from the .ini file we saw above.

These files contain sections ([app:main]) containing name = value pairs of configuration data

This data is parsed with the Python ConfigParser module.

The result is a dict of values:

{'app:main': {'pyramid.reload_templates': True, ...}, ...}

The default section of the file is passed in as global_config, the section for this app as settings.

from sqlalchemy import engine_from_config
from .models import DBSession, Base
engine = engine_from_config(settings, 'sqlalchemy.')
Base.metadata.bind = engine

We will use a package called SQLAlchemy to interact with our database.

Our connection is set up using settings read from the .ini file.

Can you find the settings for the database?

The DBSession ensures that each database transaction is tied to HTTP requests.

The Base provides a parent class that will hook our models to the database.

config = Configurator(settings=settings)
config.add_static_view('static', 'static', cache_max_age=3600)
config.add_route('home', '/')

Pyramid controlls application-level configuration using a Configurator class.

It uses app-specific settings passed in from the .ini file

We can also include configuration from other add-on packages

Additionally, we can configure routes and views needed to connect our application to the outside world here (more on this next week).

Finally, the Configurator instance performs a scan to ensure there are no problems with what we’ve created.

We will return to the configuration of our application repeatedly over the next sessions.

Pyramid configuration is powerful and flexible.

We’ll use a few of its features

But there’s a lot more you could (and should) learn.

Read about it in the configuration chapter of the Pyramid documentation.

Let’s take a moment to rest up and absorb what we’ve learned.

When we return, we’ll see how we can create models that will embody the data for our Learning Journal application.

Pyramid Models

Models in Pyramid

The central component of MVC, the model, captures the behavior of the application in terms of its problem domain, independent of the user interface. The model directly manages the data, logic and rules of the application

– from the Wikipedia article on Model-view-controller

Models and ORMs

In an MVC application, we define the problem domain by creating one or more Models.

These capture relevant details about the information we want to preserve and how we want to interact with it.

In Python-based MVC applications, these Models are implemented as Python classes.

The individual bits of data we want to know about are attributes of our classes.

The actions we want to take using that data are methods of our classes.

Together, we can refer to this as the API of our system.

It’s all well and good to have a set of Python classes that represent your system.

But what happens when you want to save information.

What happens to a instance of a Python class when you quit the interprer?

When your script stops running?

The code in a website runs when an HTTP request comes in from a client.

It stops running when an HTTP response goes back out to the client.

So what happens to the data in your system in-between these moments?

The data must be persisted

In the last class from part one of this series, you explored a number of alternatives for persistence

  • Python Literals
  • Pickle/Shelf
  • Interchange Files (CSV, XML, INI)
  • Object Stores (ZODB, Durus)
  • NoSQL Databases (MongoDB, CouchDB)
  • SQL Databases (sqlite, MySQL, PostgreSQL, Oracle, SQLServer)

Any of these might be useful for certain types of applications.

On the web, you tend to see two used the most:

  • NoSQL
  • SQL

How do you choose one over the other?

In general, the telling factor is going to be how you intend to use your data.

In systems where the dominant feature is viewing/interacting with individual objects, a NoSQL storage solution might be the best way to go.

In systems with objects that are related to eachother, SQL-based Relational Databases are a better choice.

Our system is more like this latter type (trust me on that one for now).

We’ll be using SQL (sqlite to start with).

So we have a system where our data is captured in Python objects

And a storage system where our data must be rendered as database tables

Python provides a specification for interacting directly with databases: dbapi2

And there are multiple Python packages that implement this specification for various databases:

  • sqlite3
  • python-mysql
  • psycopg2
  • ...

With these, you can write SQL to save your Python objects into your database.

But that’s a pain.

SQL, while not impossible, is yet another language to learn.

And there is a viable alternative in using an Object Relational Manager (ORM)

An ORM provides a layer of abstraction between you and SQL

You instantiate Python objects and set attributes on them

The ORM handles converting data from these objects into SQL statements (and back)


In our project we will be using the SQLAlchemy ORM.

You can find SQLAlchemy among the packages in requires in in our new learning_journal package.

However, we don’t yet have that code installed.

To do so, we will need to “install” our own package

Make sure your ljenv virtualenv is active and then type the following:

(ljenv)$ python develop
running develop
running egg_info
creating learning_journal.egg-info
Finished processing dependencies for learning-journal==0.0

Once that is complete, all the dependencies listed in our will be installed.

You can also install the package using python install

But using develop allows us to continue developing our package without needing to re-install it every time we change something.

It is very similar to using the -e option to pip

Now, we’ll only need to re-run this command if we change itself.

We also need to adjust our .gitignore file:

(ljenv)$ git status
Untracked files:
  (use "git add <file>..." to include in what will be committed)


The egg-info directory that was just created is an artifact of installing a Python egg.

It should never be committed to a repository.

Let’s add *.egg-info to our .gitignore file and then commit that change

Remember how?

Our project skeleton contains up a first, basic model created for us:

# in
Base = declarative_base()

class MyModel(Base):
    __tablename__ = 'models'
    id = Column(Integer, primary_key=True)
    name = Column(Text)
    value = Column(Integer)
Index('my_index',, unique=True, mysql_length=255)

Our class inherits from Base

We ran into Base earlier when discussing configuration.

We were binding it to the database we wanted to use (the engine)

Any class we create that inherits from this Base becomes a model

It will be connected through the ORM to a table in our database.

The name of the table is determined by the __tablename__ special attribute.

Other aspects of table configuration can also be controlled through special attributes

Instances of the class, once saved, will become rows in the table.

Attributes of the model that are instances of Column will become columns in the table.

You can learn much more in the Declarative chapter of the SQLAlchemy docs

Each attribute of your model that will be persisted must be an instance of Column.

Each instance requires at least a specific data type (such as Integer).

Additionally, you can control other aspects of the column such as it being a primary key.

In the declarative style we are using, the name of the column in the database will default to the attribute name you assigned.

If you wish, you may provide a name specifically. It must be the first argument and must be a string.

Creating The Database

We have a model which allows us to persist Python objects to an SQL database.

But we’re still missing one ingredient here.

We need to create our database, or there will be nowhere for our data to go.

Luckily, our pcreate scaffold also gave us a convenient way to handle this:

# in
main = learning_journal:main
initialize_learning_journal_db = learning_journal.scripts.initializedb:main

The console_script set up as an entry point will help us.

Let’s look at that code for a moment.

# in scripts/
from ..models import DBSession, MyModel, Base
# ...
def main(argv=sys.argv):
    if len(argv) < 2:
    config_uri = argv[1]
    options = parse_vars(argv[2:])
    settings = get_appsettings(config_uri, options=options)
    engine = engine_from_config(settings, 'sqlalchemy.')
    with transaction.manager:
        model = MyModel(name='one', value=1)

By connecting this function as a console script, our Python package makes this command available to us when we install it.

When we exectute the script at the command line, we will be running this function.

But before we try it out, let’s update the name we use so we don’t have to type that whole big mess.

In change initialize_learning_journal_db to setup_db:

main = learning_journal:main
setup_db = learning_journal.scripts.initializedb:main

Then, as you have changed, re-install your package:

(ljenv)$ python develop

Now that the script has been renamed, let’s try it out.

We’ll need to provide a configuration file name, let’s use development.ini:

(ljenv)$ setup_db development.ini
2015-01-05 18:59:55,426 INFO  [sqlalchemy.engine.base.Engine][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2015-01-05 18:59:55,434 INFO  [sqlalchemy.engine.base.Engine][MainThread] COMMIT

The [loggers] configuration in our .ini file sends a stream of INFO-level logging to sys.stdout as the console script runs.

So what was the outcome of running that script?

(ljenv)$ ls

We’ve now created an sqlite database.

You’ll need to add *.sqlite to .gitignore so you don’t inadvertently add that file to your repository.

Once you’ve done so, commit the change to your repository

Interacting with SQLA Models

It’s pretty easy to play with your models from in an interpreter.

But before we do so, let’s make a nicer interpreter available for our project

You’ve been using iPython in class, we can use it here too.

Just install it with pip:

(ljenv)$ pip install ipython pyramid_ipython

Once that finishes, you’ll be able to use iPython as your interpreter for this project.

And Pyramid provides a way to connect your interpreter to the application code you are writing:

The pshell command

Let’s fire up pshell and explore for a moment to see what we have at our disposal:

(ljenv)$ pshell development.ini
Python 3.5.0 (default, Sep 16 2015, 10:42:55)
Type "copyright", "credits" or "license" for more information.

IPython 4.0.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

  app          The WSGI application.
  registry     Active Pyramid registry.
  request      Active request object.
  root         Root of the default resource tree.
  root_factory Default root factory used to create `root`.

The environment created by pshell provides us with a few useful tools.

app          The WSGI application.
registry     Active Pyramid registry.
request      Active request object.
root         Root of the default resource tree.
root_factory Default root factory used to create `root`.
  • The app is our new learning journal application
  • The registry provides us with access to settings and other useful information
  • The request is an artificial HTTP request we can use if we need to pretend we are listening to clients
  • ...

Let’s use this environment to build a database session and interact with our data:

In [1]: from sqlalchemy import engine_from_config
In [2]: engine = engine_from_config(registry.settings, 'sqlalchemy.')
In [3]: from sqlalchemy.orm import sessionmaker
In [4]: Session = sessionmaker(bind=engine)
In [5]: session = Session()
In [6]: from learning_journal.models import MyModel
In [7]: session.query(MyModel).all()
2015-12-21 18:06:05,179 INFO  [sqlalchemy.engine.base.Engine][MainThread] SELECT AS models_id, AS models_name, models.value AS models_value
FROM models
2015-12-21 18:06:05,179 INFO  [sqlalchemy.engine.base.Engine][MainThread] ()
Out[7]: [<learning_journal.models.MyModel at 0x105f30208>]

We’ve stolen a lot of this from the script

Any interaction with the database requires a session.

This object represents the connection to the database.

All database queries are phrased as methods of the session.

In [8]: query = session.query(MyModel)
In [9]: type(query)
Out[9]: sqlalchemy.orm.query.Query

The query method of the session object returns a Query object

Arguments to the query method can be a model class or columns from a model class.

You can iterate over a query object. The result depends on the args you passed.

In [10]: q1 = session.query(MyModel)
In [11]: for row in q1:
   ....:     print(row)
   ....:     print(type(row))
<learning_journal.models.MyModel object at 0x105f30208>
<class 'learning_journal.models.MyModel'>

You can iterate over a query object. The result depends on the args you passed.

In [12]: q2 = session.query(,, MyModel.value)
In [13]: for name, id, val in q2:
   ....:     print(name)
   ....:     print(type(name))
   ....:     print(id)
   ....:     print(type(id))
   ....:     print(val)
   ....:     print(type(val))
<class 'str'>
<class 'int'>
<class 'int'>

You can view the SQL that your query will use:

In [14]: str(q1)
Out[14]: 'SELECT AS models_id, AS models_name, models.value AS models_value \nFROM models'

In [15]: str(q2)
Out[15]: 'SELECT AS models_name, AS models_id, models.value AS models_value \nFROM models'

You can use this to check that the query the ORM is constructing looks like you expect.

It can be helpful in debugging.

The methods of the Query object fall into two rough categories

  1. Methods that return a new Query object
  2. Methods that return scalar values or model instances

Let’s start by looking quickly at a few methods from the second category

A good example of this category of methods is get, which returns one instance only.

It takes a primary key as an argument:

In [16]: session.query(MyModel).get(1)
Out[16]: <learning_journal.models.MyModel at 0x105f30208>
In [17]: session.query(MyModel).get(10)
In [18]:

If no item with that primary key is present, then the method returns None

Another example is one we’ve already seen.

query.all() returns a list of all rows returned by the database:

In [18]: q1.all()
Out[18]: [<learning_journal.models.MyModel at 0x105f30208>]

In [19]: type(q1.all())
Out[19]: list

query.count() returns the number of rows that would have been returned by the query:

In [20]: q1.count()
Out[20]: 1

Before getting into the other category, let’s learn how to create new objects.

We can create new instances of our model just like normal Python objects:

In [21]: new_model = MyModel(name='fred', value=3)
In [22]: new_model
Out[22]: <learning_journal.models.MyModel at 0x105f4af28>

In this state, the instance is ephemeral, our session knows nothing about it:

In [23]:
Out[23]: IdentitySet([])

For the database to know about our new object, we must add it to the session:

In [24]: session.add(new_model)
In [25]:
Out[25]: IdentitySet([<learning_journal.models.MyModel object at 0x105f4af28>])

We can even bulk-add new objects:

In [26]: new = []
In [27]: for name, val in [('bob', 34), ('tom', 13)]:
   ....:     new.append(MyModel(name=name, value=val))
In [28]: session.add_all(new)
In [29]:
Out[29]: IdentitySet([<learning_journal.models.MyModel object at 0x105f4af28>,
                      <learning_journal.models.MyModel object at 0x105f4a4a8>,
                      <learning_journal.models.MyModel object at 0x105f30550>])

Up until now, the changes you’ve made are not permanent.

In order for these new objects to be saved to the database, the session must be committed:

In [30]: other_session = Session()
In [31]: other_session.query(MyModel).count()
Out[31]: 1
In [32]: session.commit()
In [33]: other_session.query(MyModel).count()
Out[33]: 4

When you are using a scoped_session in Pyramid, this action is automatically handled for you.

The session that is bound to a particular HTTP request is committed when a response is sent back.

(don’t worry if this seems confusing, more to come next week)

You can edit objects that are already part of a session, or that are fetched by a query.

Simply change the values of a persisted attribute, the session will know it’s been updated:

In [34]: new_model
Out[34]: <learning_journal.models.MyModel at 0x105f4af28>
In [35]:
Out[35]: 'fred'
In [36]: = 'larry'
In [37]: session.dirty
Out[37]: IdentitySet([<learning_journal.models.MyModel object at 0x105f4af28>])

Commit the session to persist the changes:

In [38]: session.commit()
In [39]: [ for model in other_session.query(MyModel)]
Out[39]: ['one', 'larry', 'bob', 'tom']

Returning to query methods, a good example of the second type is the filter method.

This method allows you to reduce the number of results, based on criteria:

In [40]: [(, o.value) for o in session.query(MyModel).filter(MyModel.value < 20)]
Out[40]: [('one', 1), ('larry', 3), ('tom', 13)]

Another typical method in this category is order_by:

In [41]: [o.value for o in session.query(MyModel).order_by(MyModel.value)]
Out[41]: [1, 3, 13, 34]

In [42]: [ for o in session.query(MyModel).order_by(]
Out[42]: ['bob', 'larry', 'one', 'tom']

Since methods in this category return Query objects, they can be safely chained to build more complex queries:

In [43]: q1 = session.query(MyModel).filter(MyModel.value < 20)
In [44]: q1 = q1.order_by(
In [45]: [(, o.value) for o in q1]
Out[45]: [('larry', 3), ('one', 1), ('tom', 13)]

Note that you can do this inline as well (s.query(Model).filter().order_by())

Also note that when using chained queries like this, no query is actually sent to the database until you require a result.

Cleaning Up After Ourselves

When you are experimenting with a new system, you often create data that is messy or incomplete.

It’s good to remember that none of the information we’ve persisted to our database is vital to us.

For homework this week we’ll be making new models, and the data we have in our current database will only get in the way.

Until you have real production data it is always safe simply to delete the database and start over:

$ rm learning_journal.sqlite

You can always re-create it by executing setup_db


Okay, that’s enough for the moment.

You’ve learned quite a bit about how models work in SQLAlchemy

It’s time to put that knowledge to good use.

For the first part of your assignment this week you will begin to define the data model for our learning journal application.

I’ll provide a specification, you define the model required to do the job.

I’ll also ask you to define a few methods to complete the first part of our API.

The Model

Our model will be called an Entry. Here’s what you need to know:

  • It should be stored in a database table called entries
  • It should have a primary key field called id
  • It should have a title field which accepts unicode text up to 255 characters in length
  • The title should be unique and it should be impossible to save an entry without a title.
  • It should have a body field which accepts unicode text of any length (including none)
  • It should have a created field which stores the date and time the object was created.
  • It should have an edited field which stores the date and time the object was last edited.
  • Both the created and edited field should default to now if not provided when a new instance is constructed.
  • The entry class should support a classmethod all that returns all the entries in the database, ordered so that the most recent entry is first.
  • The entry class should support a classmethod by_id that returns a single entry, given an id.

Remember that in order to have your new model table created, you will have to re-run the initialize_learning_journal_db script after creating your model.

Use the documentation linked in this presentation to assist you. SQLAlchemy has fantastic documentation, but it can be a bit overwhelming. Everything you require for this assignment is on one or more of the pages linked above.

As you define this new model for our application, make frequent commits to your github repository. Remember to write meaningful commit messages.

Don’t be afraid to start up a Python interpreter and play with your model. Try things out. Learn how this all works by making mistakes. Remember the pshell command and how we set up a session once the shell is running.

Errors at the SQL level can sometimes leave your session unusable. To restore it, use the session.rollback() method. You’ll lose uncommitted changes, but you’ll gain a session that can be used again.

I want to be able to review your code (and you want to be able to share it).

To submit this assignment, you’ll need to add this learning_journal repository to GitHub.

On the GitHub website you can create a new repository. Set it up to be completely empty. Name it learning_journal and give it any description you like.

When you’ve created an empty repository in GitHub, you should see a set of directions for connecting it to a repository that you’ve already built. Follow those instructions to connect your emtpy GitHub repository as the origin remote to your learning_journal repository on your machine.

Finally, push your master branch to your new origin remote on GitHub.

When you are done, send me an email with the URL for your new repository.

Our work next week will assume that you have completed this assignment

Do not delay working on this until the last moment.

Do not skip this assignment.

Do ask questions frequently via email (use the class google group).

See you next week!