dbpool module

Synopsis

The dbpool module is a wrapper for Python DB-API 2.0-compliant database modules to (a) keep a pool of physical connections available and (b) upgrade the modules to threadsafety level 2, which means that threads can share logical database connections.

Usage

The module provides only one function not described by the DB-API 2.0 spec. The module is initialized by calling set_database with a reference to the underlying database module to be used and the number of physical database connections to keep in the pool. The dbpool module is then configured to act precisely like the underlying database module, except that physical database connections are not always closed when you have finished with them, and if you try to create more than one cursor on a single logical connection, a new physical connection is created for the new cursor.

When a logical database connection is no longer being used (i.e. you call the close function or all references to the connection are deleted), the physical connection is returned to the pool to be used the next time a connection is requested. If the pool is full (i.e. the limit set in the call to set_database is reached) then the physical database connection is closed.

Note that your code can either create one logical database connection, and then create many cursors from that, or create many logical database connections, and create one or more cursors from each of these. Either way the behaviour of the dbpool module will be the same - it will pool physical database connections, and only create one cursor per physical database connection.

Example:

import jon.dbpool as dbpool
import MySQLdb, MySQLdb.cursors
dbpool.set_database(MySQLdb, 5)
dbh = dbpool.connect(user="example", passwd="s3cr3t", db="example",
  cursorclass=MySQLdb.cursors.DictCursor)
dbc1 = dbh.cursor()
dbc2 = dbh.cursor()

Note that unfortunately, due to the way the DB-API works, once you have configured dbpool to act as a wrapper for a database module, you cannot re-configure with a different module.

Globals

Variables

apilevel

The string 2.0, indicating the DB-API level.

threadsafety

The integer 2, meaning that threads may share the module, and database connections.

paramstyle

This variable is only available after set_database has been called. It will contain the value from the underlying database module.

Warning

Error

InterfaceError

DatabaseError

DataError

OperationalError

IntegrityError

InternalError

ProgrammingError

NotSupportedError

These exception classes are only available after set_database has been called. They are copied from the underlying database module.

Functions

set_database(dbmod, minconns, timeout=0, postconnect=None)

dbmod: module
minconns: integer
timeout: integer
postconnect: callable

Configures the dbpool module to act as a wrapper around the specified DB-API module. dbmod is a reference to the database module to use. minconns is an integer, which must be 1 or greater, which indicates the number of physical database connections of each 'type' to keep in the pool. Physical database connections are of the same 'type' if all the parameters to the DB-API connect function are the same (i.e. they are connecting to the same database, on the same host, with the same username, with the same options).

timeout is an integer, which must be 0 or greater, which indicates the maximum time in seconds that a connection is allowed to have sat idle before it will not be re-used. A value of 0 means that no timeout will be applied. This timeout is intended to prevent dbpool from re-using connections that the database server will have closed due a timeout on the server side, so a value that is less than the server-side timeout should be chosen.

postconnect is a callable. If it is provided then it will be called immediately after any connection to the database is created. The first argument will be the database connection, and the remaining positional and keyword arguments will be identical to those passed to the database's connect function. This can be used to perform any configuration required on the database connections that cannot be done via arguments to connect - for example, enable to auto-commit on MySQL connections.

The database module to be used must have a threadsafety level of at least 1, i.e. if the database cannot cope at all with multi-threading then there is nothing dbpool can do to make it work.

Note that you can only call this function once.

Example:

def setautocommit(dbh, *args, **kwargs):
  dbh.autocommit(True)
set_database(MySQLdb, 5, postconnect=setautocommit)

connect(...)

Returns: database connection instance

This function may only be used after set_database has been called. The parameters are dependent on the underlying database module being used. A logical database connection from the pool corresponding to database connections with the parameters given is returned. If the pool is empty then a new physical connection is created.

Example:

dbh = dbpool.connect(user="example", passwd="s3cr3t", db="example")
dbc = dbh.cursor()

$Id: dbpool.html,v 3e2fedadb6ed 2015/11/10 12:52:17 jon $