July 30, 2010

web2py and metaclasses

Filed under: Uncategorized — mdipierro @ 2:26 pm


In the web2py world we always think about how to we make things easier for users. Here we want to show how you can use metaclasses to make your code very readable.

Out goal is to define some new syntax that has immediate meaning in English and can be reused in any web2py application.


Create an empty application (called for example thing_manager) and define

# black box
is_a = type('_',(),{'__getattr__':(lambda s,n: (lambda *a, **b: db.define_table(n,*a,**b)))})()
with_a = type('_',(),{'__getattr__':(lambda s,n: (lambda *a, **b: Field(n,*a,**b)))})()
this = type('_',(),{'__getattr__': (lambda s,n: db[n][request.args(0)])})()
please = crud
get_me = lambda a,*b,**c: db(a).select(*b,**c)
delete = lambda a,*b,**c: db(a).delete(*b,**c)
update = lambda a,*b,**c: db(a).update(*b,**c)
to_visitor = lambda *a: dict(page=DIV(*[DIV(i) for i in a]))
# end black box

The model

Now in your model you can define tables using the new syntax:

the_thing = is_a.thing(with_a.name(), with_a.category())
my_things = the_thing.id>0

where ‘thing’ is the name of the table, ‘name’ and ‘category’ are fields. You can specify named field attributes of name and category using normal Field(…) attribute syntax such as in


The Controller

Now in the default controller you can use the new syntax to define functions. Here is one for example:

def index():
    if this.thing:
        form = please.update(the_thing, this.thing)
        form = please.create(the_thing)
    things = get_me(my_things)
    return to_visitor(form,things)

This function can be called by
(will make a create interface for thing) or
(will make an update for for thing with id==1)

This action creates a complete thing_management interface. get_me(my_things) returns all current thing records. please.create and please.update simply map into the corresponding crud functions.


Here is how to looks like:

Other class

Of course there is nothing special about thing. Let’s use the same trick for a table “product” and different actions:

the_product = is_a.product(with_a.name(), with_a.price('double'))
my_products = the_product.price>0.0

# http://...new_product                                                                                                        
def new_product():
     return to_visitor(please.create(the_product))

def update_product():
     return to_visitor(please.update(the_product,this.product))

def list_products():
     products = get_me(my_products)
     return to_visitor(products)

Some explanation

The magic happens in is_a, with_a and this. There are instances of three different temp classes defined by the type function. They take attributes of arbitrary names and return lambda function that return the desired function:

  • is_a.thing(…) maps into db.define_table(‘thing’,….)
  • with_a.name(…) maps into Field(‘name’,….)
  • this.thing() maps into db[‘thing’][request.args(0)] this standard web2py syntax for fetching the record with id==request.args(0) (the [id] in http://…/index/%5Bid%5D) or None if the record does not exist.

March 22, 2010

Skinning web2py

Filed under: Uncategorized — mdipierro @ 6:13 pm

Web2py comes with a default web2py/applications/views/layout.html.

You can change web2py skin by editing this file and adding requires static files in web2py/applications/static/

We have created a number of skins for you to make your life easier. We have been experimenting with three different approaches:

1) A single web2py plugin that comes with hundreds of skins downloaded from he web. You can download and try it here (it is a large plugins because it comes with lots of skins):


2) Multiple plugins where each plugin provides a different template. We converted almost all free drupal themes into web2py plugins. You can get them here:


3) A single plugin that provides one HTML layout but multiple “pure CSS” skins. You can find a here a proof of concept that links skins from zengarden


Eventually we will unify all these options in a single resource and add a simple admin interface to apply skins.

March 7, 2010

web2py ajax and forms

Filed under: Uncategorized — mdipierro @ 10:06 pm

The web2py scaffolding application “welcome” has a “layout.html” that {{include “web2py_ajax.html”}} within the <head>…</head> tags. “web2py_ajax.html” defines a few useful JavaScript functions and includes all required CSS and JS files (jquery.js, calendar.js, calendar.css).

The functions defined in “web2py_ajax.html” are designed so that you do not need to write any JS in order to use them.

Web2py comes with everything you need for the examples below to work, including jQuery.

Here are some examples:

Calendar and pre-validation

If you have:

<input class = "date" name="fieldname">

when you click on the input field it will automatically show a popup calendar. Similarly class “datetime” shows a calendar with time-picker and “time” shows a time-picker.

If you have:

<input class = "double" name="fieldname">

web2py will prevent you from typing anything that is not a floating point number. Similarly class “integer” will prevent you from typing anything that is not a valid “integer”.

We call this “pre-validation” to avoid confusion with the proper server-side validation.

All forms generated by web2py (SQLFORM) are CSS friendly all <input /> tags have classes that correspond to the type of the corresponding field, therefore they automatically provide the functionality described here.


If you have

<div class="flash">content</div>

and if content is not empty, it will slide down the content of the flash and will fade it out when you click on it.

web2py defines a function called “ajax” built on top of jQuery’s own ajax. Here is a example of usage:

<input name="key" onKeyUp="ajax('callback',['key'],'target');"/>
<div id="target"></div>

OnKeyUp the ajax function


calls the action at the url specified as first argument (for example ‘callback’) and passes as post variables those specified in the second argument (for example the value of <input name=’key’>), then stores the AJAX return value in the html tag with ID specified by the third argument (for example ‘target’). If ‘target’ has a value attribute the AJAX return value goes into value instead of the inner HTML. If ‘target’ is replaced by ‘:eval’, the AJAX return value is interpreted as JS code and executed.

Components and Forms

When applications get more and more complex you may want to build more modular ones. web2py provides a mechanism to build very modular applications by embedding an action into the template of another action and capture form submissions from the inner one.  Here is an example. Consider the following two actions in controller default.py

def index():
   return dict()

def myform():
   form = SQLFORM.factory(Field('name',requires=IS_NOT_EMPTY()))
   if form.accepts(request.vars): return 'Hello %s' % form.vars.name
   return form

and the following template for the index() action in default/index.html:

{{extend 'layout.html'}}
<h1>Page Title</h1>

When calling the “index” action ( the output of the index() function is rendered by the template which, in turn, embeds the output of “myform” action ( via ajax. The LOAD function also generated all required JS code to handle forms inside the loaded content (is the example there is a single form). When the form is submitted, only the “myform” action is called and only the LOADed component is re-loaded, not the entire page. If the form does not pass validation, the form is displayed again including validation errors (this is all handled by the accept function). If the form does pass validation, it returns “Hello <your name>”. The text replaces the actual form, again without disturbing the outer HTML. You can LOAD as many components as you like in one page. The LAOD function also takes additional parameters including args=[] and vars={} to be passed to the called action.

The callback function can be any regular web2py action and can have its own template (for the LOADed component only). Just make sure the template of a component does NOT {{extend ‘layout.html’}} since the layout is provided by the LOADing template.

The LOAD function can also load a URL hosted on a different server (and it does not need to be running web2py):


Sometimes it happens that your LOADed action needs to return more than HTML/TEXT but also instructions to the calling page, in the form of JS code embedded in the browser. This can be done by storing JS code in an HTTP header returned by “myform”. The following code for example makes the form “fadein” when it appears

def myform():
   form = SQLFORM.factory(Field('name',requires=IS_NOT_EMPTY()),_class='myformclass')
   if form.accepts(request.vars): return 'Hello %s' % form.vars.name
   return form


Sometimes you need JSON. Any web2py action can return json. Just call the action with .json.

For example:

def myvalues():
    return dict(values=['a','b','c'])

And call it with

(works also for .html and .xml out of the box. For any other user-defined extension, you just need to create a views/generic.ext file to handle it).

Special care must be used when retuning DB Rows objects, since they are not serializable in JSON. They must be serialized in Python first. as_dict() accomplished that.

For example:

def myusers():
    return dict(users=db(db.auth_user.id>0).select().as_dict())

And call it with

If you want to program lower level web2py includes simplejson in gluon.contrib.simplejson. It has been modified to handle date and datetime.

March 4, 2010

web2py Bayesian classifier and databases

Filed under: Uncategorized — mdipierro @ 3:05 pm

We have all run into the problem of pre-populating a database for the purpose of debugging or demoing a program. The problem is complicated by references in database tables and different field types.

web2py provides a solution for this problem and includes a minimalist Bayesian classifier trained and adapted to the scope.

Here is an example of usage:


Start the web2py interactive shell:

$ python web2py.py -S welcome -M

Define some tables, for example two related to each other (people and their comments):

>>> db.define_table('person',Field('name'))

>>> db.define_table('comment',Field('author',db.person),Field('body'))

Import the “populate” function

>>> from gluon.contrib.populate import populate

Ask it to populate the database tables with 100 records each

>>> populate(db.person,100)
>>> populate(db.comment,100)

See what you got

>>> for comment in db(db.comment.id>0).select(limitby=(0,3)):
        print comment.author.name, comment.body

Cocomoto SAYS Rischgitz collection. j. In connection with large that they become.
Saducece SAYS Circumvent the latest measurements at least 600 a dull white except.
Popotadu SAYS Frequenting forests of specialized member. But a list of view.

If you like it, commit your changes.

>>> db.commit()

The generate function is powerful enough to understand different field types (‘string’, ‘text’, ‘integer’, ‘date’, ‘reference’, etc.) their validation constraints and populate them accordingly. It is also very fast.


Sometimes you may want to train the Bayesian classifier with your own text and generate new text based on that. Here is how:

Import the Learner

>>> from contrib.populate import Learner

Get some text, for example Alice in Wornderland

>>> import urllib
>>> text = urllib.urlopen('http://www.gutenberg.org/files/11/11.txt').read()

Have the learner learn the text:

>>> learner = Learner()
>>> learner.learn(text)

Ask the learner to generate new text (1000 words) that “sounds similar” to the learned text.

>>> print learner.generate(1000)

Be. it further. so very angrily. it something; and took me there goes like a few minutes and she went to anyone providing access to repeat tis so bill thought alice and the footman in addition to drop the mouse come here. and she could hear you. and peeped into the use in the blame on slates and be of the paper has agreed to the hookah into the question is a table all access to do practically anything you see: the centre of the accident of a little glass table half the rest of receiving it gave her arm for you shouldn t remember half shut. this and was not be two.


He came between them at the door began talking such things to give your tongue said the hatter with seaography: that said was close to watch.

Of course the text does not make any sense. That is a feature.

March 2, 2010

shell only web2py

Filed under: Python, Web Framework — mdipierro @ 5:22 pm

So you are old fashion? So am I. The web2py admin is nice but sometimes it gets in the way. Here is how you install and develop web2py apps without using admin. (Mind that in web2py “admin” refers to the web based IDE, while “appadmin” referes to the web based database administrative interface).

Download, Install and Start

$ wget http://web2py.com/examples/static/web2py_src.zip
$ unzip web2py_src.zip
$ cd web2py
$ python web2py.py -i -p 8000 -a 'chooseapassword' &

web2py is now installed and running. You can try it:

‘chooseapassword’ will be used in case you later decide to use admin. If you set an empty password the admin interface is completely disabled.

My First Application

An app in web2py is a folder. At a minimum it needs a controller action else it does nothing. We are going to create a simple action that says “Hello World”

$ cd applications
$ mkdir myapp
$ mkdir myapp/controllers
$ echo 'def index(): return "Hello World"' > myapp/controllers/test.py

Done! Now you can try it.

(Notice that there is nothing else to type to deploy the app, nor there is any metadata anywhere).

Exploring “myapp”

By calling the app, web2py has created the proper folder structure, which we can explore:

$ cd myapp
$ ls
models       (the data representation)
controllers  (logic and business flow)
views        (presentation)
static       (css, js, images, etc)
modules      (anything to be reused)
languages    (translation files)
cache        (shelves for cache.disk)
databases    (.table, sql.log and storage.sqlite)
sessions     (unless session disabled or in DB)
errors       (all tracebacks logged here)
cron         (contains web2py crontab)
private      (any other files needed by the app)
uploads      (files uploaded by the app)

Currently all the folders are empty except for controllers/ which contains test.py and sessions (since each user is associated to a session file unless disabled).

Deploying an existing app

Deploying and app is as easy as zipping an application folder and unzipping it under a different web2py installation. For example if you want to install the CRM shown here all you need to do is

$ cd ../
$ mkdir mycrm
$ cd mycrm
$ wget http://web2py.com/appliances/default/download/app.source.9ee5fc505dd76ea2.7765623270792e6170702e63726d2e773270.w2p
$ tar zxvf app.source.9ee5fc505dd76ea2.7765623270792e6170702e63726d2e773270.w2p

And now the CRM up and running at

You can manage it form

web2py apps are normally distributed as .w2p files. They are just tar gzipped folders.

Request Variables

Request variables are in request.vars. Here is an example

$ cd ../myapp
$ echo '
def index():
    return "Hello %s" % request.vars.name
' > controllers/test.py

Now if you call

You get

Hello John

Session Variables

Each visitor is uniquely associated to a session. If for example you want to count how many times the visitors loads the index URL:

$ echo '
def index():
    session.my_counter = (session.my_counter or 0)+1
    return "You have been here %s times" % session.my_counter
' > controllers/test.py

Now if you now visit

You get

You have been here 3 times

(notice that session.anything returns None if anything is not stored in the session)


If an action (i.e. a function in a controller) returns a dictionary. the output is rendered by a view. A view can be .html, .json, .xml or any other extension. You specify the extension in the URL (defaults to html). If a view is not present web2py uses a generic one to try render the dict. For example:

$ echo '
def index():
    return dict(message="Hello World")
' > controllers/test.py
$ mkdir views/test
$ echo '
' > views/test/index.html

Now if you now visit

You get

<html><body><h1>Hello World</h1></body></html>

Notice that {{=message}} automatically escapes the variable message before printing it to avoid XSS injections.


It is common to define a layout so that all pages look similar:

$ echo '
<html><body>{{include}}</body></html>' > views/mylayout.html
$ echo '
{{extend "mylayout.html"}}
' > views/test/index.html

The content of the index.html view replaces the {{include}} of the extended view.

Database Models

Let’s create a simple app that allows users to post comments and read each other comments:

$ echo '
db = DAL("sqlite://storage.sqlite")
' > models/db.py
$ echo '
def index():
    if form.accepts(request.vars):
    messages = db().select(db.message.ALL)
    return dict(form=form, messages=messages)
' > controllers/test.py
$ echo '
{{extend "mylayout.html"}}
<h1>Post a Message</h1>
<h1>Previous Messages</h1>
  {{for message in messages:}}<p>{{=message.body}}</p>{{pass}}
' > views/test/index.html

Now if you visit the usual url you get a form with built-in validation that displays errors (if submitted values for no pass validation) and displays a list of all posted messages.

Notice that You DO NOT NEED to create the table ‘message’ or access the DB using third party tools. web2py takes care of it for you automatically.

If you want the database administrative interface

copy from the welcome folder into your app the following files


Cloning an app

Normally you do not start from scratch. For example you may want to start from a clone of the scaffolding app “welcome” (this is what “admin” does when you create a new app).

$ cd ../myapp
$ cp -r ../welcome/* ./

The scaffolding app includes a lot of useful stuff such as “appadmin”, “auth”, “crud”, “service”, secure upload/download, generic views, default layout and some Javascript/Ajax tools.

Deploying the app on Google App Engine

To deploy an app on GAE edit web2py/app.yaml (already provided) and specify your GAE application id.

db = DAL('sqlite://....')


db = DAL('gae')

and deploy it

appcfg.py update web2py/

The last line must be executed from outside the folder that contains web2py.

Interactive mode

You can enter into interactive mode for any specific app

$ python web2py.py -S myapp -M
>>> print db.tables
>>> print db.message.fields
>>> db.message.insert(body="this is a test")
>>> db.commit()

Your code would be executed as if it were in a controller. You have to db.commit() or db.rollback() explicitly.

Use web2py.py -h for more command line options.


Web2py controllers and models are executed, not imported (this is different from any other Python framework and similar to Ruby on Rails). This means they DO NOT “import web2py” but they CAN “import any_other_module” you may need.

There is no meta-data stored anywhere.

There is no process monitoring files for changes.

In web2py there is a single event (the http request). When an http request arrives, web2py locates the requested app/controller/action, re-creates any folder that may be missing, creates any database table that is required (or alters the db appropriately), executes the models, the requested actions and its associated view. If the app uses the db, everything is done in a transaction. Any uncaught error exception triggers a rollback and a ticket is issued to the user. All errors are logged. Errors are in the myapp/errors folders and they are just Python pickles which contain the traceback.

If you clone “welcome” everything has a default: each model has an “appadmin” interface and each action has default generic views in html, json, and xml.


About the Author

Massimo was born in 1971. He has a Ph.D. in High Energy Physics. He is an Associate Professor of Computer Science and the Director of the Master in Computational Finance at the School of Computing of DePaul University in Chicago. He is the lead developer of web2py.

Blog at WordPress.com.