JPype DBAPI2 Guide

Introduction

One common use of JPype is to provide access to databases used JDBC. The JDBC API is well established, very capable, and supports most databases. JPype can be used to access JDBC both directly or through the use of the Python DBAPI2 as layed (see PEP-0249). Unfortunately, the Python API leaves a lot of behaviors undefined.

The JPype dbapi2 module provides our implementation of this Python API. Normally the Python API has to deal with two different type systems, Python and SQL. When using JDBC, we have the added complexity that Java types are used to communicate with the driver. We have introduced concepts appropriate to handle this addition complexity.

Module Interface

Constructors

Access to the database is made available through connection objects. The module provides the following constructor for connections:

jpype.dbapi2.connect(dsn, *, driver=None, driver_args=None, adapters=<object object>, converters=<object object>, getters=<function GETTERS_BY_TYPE>, setters=<function SETTERS_BY_TYPE>, **kwargs)

Create a connection to a database.

Arguments to the driver depend on the database type.

Parameters:
  • dsn (str) – The database connection string for JDBC.
  • driver (str, optional) – A JDBC driver to load.
  • driver_args – Arguments to the driver. This may either be a dict, java.util.Properties. If not supplied, kwargs are used as as the parameters for the JDBC connection.
  • *kwargs – Arguments to the driver if not supplied as driver_args.
Raises:

Error if the connection cannot be established.

Returns:

A new connection if successful.

Globals

JPype dbapi2 defines several globals that define the module behavior. These values are constants.

apilevel
The apilevel for the module is “2.0”.
threadsafety

The threadsafety level is 2 meaning “Threads may share the module and connections”. But the actual threading level depends on the driver implementation that JDBC is connected to. Connections for many databases are synchronized so they can be shared, but threads must execute statement in series. Connections in the module are implemented in Python and have per object resources that cannot be shared. Attempting to use a connection with a thread other than the thread that created it will raise an Error.

Sharing in the above context means that two threads may use a resource without wrapping it using a mutex semaphore to implement resource locking. Note that you cannot always make external resources thread safe by managing access using a mutex: the resource may rely on global variables or other external sources that are beyond your control.

paramstyle

The parameter style for JPype dbapi2 module is qmark

paramstyle Meaning
qmark Question mark style, e.g. ...WHERE name=?

Exceptions

The dbapi2 module exposes error information using the following exceptions:

class jpype.dbapi2.Warning

Exception raised for important warnings like data truncations while inserting, etc.

class jpype.dbapi2.Error

Exception that is the base class of all other error exceptions. You can use this to catch all errors with one single except statement. Warnings are not considered errors and thus should not use this class as base.

class jpype.dbapi2.InterfaceError

Exception raised for errors that are related to the database interface rather than the database itself.

class jpype.dbapi2.DatabaseError

Exception raised for errors that are related to the database.

class jpype.dbapi2.DataError

Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range, etc.

class jpype.dbapi2.OperationalError

Exception raised for errors that are related to the database’s operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc.

class jpype.dbapi2.IntegrityError

Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails.

class jpype.dbapi2.InternalError

Exception raised when the database encounters an internal error, e.g. the cursor is not valid anymore, the transaction is out of sync, etc.

class jpype.dbapi2.ProgrammingError

Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc.

class jpype.dbapi2.NotSupportedError

Exception raised in case a method or database API was used which is not supported by the database, e.g. requesting a .rollback() on a connection that does not support transaction or has transactions turned off.

Python exceptions are more fine grain than JDBC exceptions. Whereever possible we have redirected the Java exception to the nearest Python exception. However, there are cases in which the Java exception may appear. Those exceptions inherit from :py:class:jpype.dbapi2.Error. This is the exception inheritance layout:

Exception
|__Warning
|__Error
   |__InterfaceError
   |__java.sql.SQLError
   |  |__java.sql.BatchUpdateException
   |  |__java.sql.RowSetWarning
   |  |__java.sql.SerialException
   |  |__java.sql.SQLClientInfoException
   |  |__java.sql.SQLNonTransientException
   |  |__java.sql.SQLRecoverableException
   |  |__java.sql.SQLTransientException
   |  |__java.sql.SQLWarning
   |  |__java.sql.SyncFactoryException
   |  |__java.sql.SyncProviderException
   |
   |__DatabaseError
      |__DataError
      |__OperationalError
      |__IntegrityError
      |__InternalError
      |__ProgrammingError
      |__NotSupportedError

Type Access

JPype dbapi2 provides two different maps which serve to convert data between Python and SQL types. When setting parameters and fetching results, Java types are used. The connection provides to maps for converting the types of parameters. An adapter is used to translate from a Python type into a Java type when setting a parameter. Once a result is produced, a converter can be used to translate the Java type back into a Python type.

There are two lookup functions that select the behavior to decide how a column or parameter should be treated. These are getters and setters.

adapters

Whenever a Python type is passed to a statement, it must first be converted to the appropriate Java type. This can be accomplished in a few ways. The user can manually convert to the correct type by constructing a Java object or applying the JPype casting operator. Some Java types have built in implicit conversions from the corresponding type. For all other conversions, an adapter. An adapter is defined as a type to convert from and a conversion function which takes a single argument that returns a Java object.

The adapter maps are stored in the connection. The adapter map can be supplied when calling connect , or added to the map later through the adapters property.

setters

A setter transfers the Java type into a SQL parameter. There are multiple types can an individual parameter may accept. The type of setter is determined by the JDBC type. Each individual JDBC type can have its own setter. Not every database supports the same setter. There is a default setter may that would work for most purposes. Setters can also be set individually using the types argument to the .execute*() methods. The setter is a function which processes the database metadata into a type.

Setters can supplied as a map to connect or by accessing the setter property on a Connection.

jpype.dbapi2.SETTERS_BY_META(cx, meta, col, ptype)

Option for setters to use the metadata of the parameters.

On some databases this option is useless as they do not track parameter types. This method can be cached for faster performance when lots of parameters. Usually types can only be determined accurately on inserts into defined columns.

jpype.dbapi2.SETTERS_BY_TYPE(cx, meta, col, ptype)

Option for setters to use the type of the object passed.

This option looks at the type of the parameter being passed from Python after adapters have been applied to determine the best setter.

converters

When a result is fetched the database, it is returned as Jave type. This Java type then has a converter applied. Converters are stored in a map holding the type as key and a converter function that takes one argument and returns the desired type. The default converter map will convert all types to Python. This can be disabled by setting the converters to None.

The converter map can be passed in to the connect function, or set on the Connection using the converters property. It can be supplied as a list or a map to the .fetch*() methods.

getters

JDBA provides more than one way to access data returned from a result. In the native JDBC, each executed statement returns a result set which acts as a cursor for the statement. It is possible to access each column using a different get method. The default map will attempt to fetch according to the most general type. The getter is a configurable function that uses the metadata to find the most appropriate type.

jpype.dbapi2.GETTERS_BY_TYPE(cx, meta, idx)

Option for getters to determine column type by the JDBC type.

This option is the default option that uses the type code supplied in the meta data. On some databases it is better to use the name. If the type code is OTHER, it will attempt to find a type by name. New types can be created with JDBCType for database specific types.

jpype.dbapi2.GETTERS_BY_NAME(cx, meta, idx)

Option to getters to determine column type by the column name.

This option uses the column name to select the type. It looks up the column type name, converts it to uppercase, and then searches for a matchine type. It falls back to the type code meta information if the typename can not be found in the registery. New types can be created using JDBCType for database specific types such as JSON.

Connection Objects

A Connection object can be created using the using connect function. Once a connection is established the resulting Connection contains the following.

class jpype.dbapi2.Connection(jconnection, adapters, converters, setters, getters)

Connection provides access to a JDBC database.

Connections are managed and can be as part of a Python with statement. Connections close automatically when they are garbage collected, at the end of a with statement scope, or when manually closed. Once a connection is closed all operations using the database will raise an Error.

exception DataError

Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range, etc.

exception DatabaseError

Exception raised for errors that are related to the database.

exception Error

Exception that is the base class of all other error exceptions. You can use this to catch all errors with one single except statement. Warnings are not considered errors and thus should not use this class as base.

exception IntegrityError

Exception raised when the relational integrity of the database is affected, e.g. a foreign key check fails.

exception InterfaceError

Exception raised for errors that are related to the database interface rather than the database itself.

exception InternalError

Exception raised when the database encounters an internal error, e.g. the cursor is not valid anymore, the transaction is out of sync, etc.

exception NotSupportedError

Exception raised in case a method or database API was used which is not supported by the database, e.g. requesting a .rollback() on a connection that does not support transaction or has transactions turned off.

exception OperationalError

Exception raised for errors that are related to the database’s operation and not necessarily under the control of the programmer, e.g. an unexpected disconnect occurs, the data source name is not found, a transaction could not be processed, a memory allocation error occurred during processing, etc.

exception ProgrammingError

Exception raised for programming errors, e.g. table not found or already exists, syntax error in the SQL statement, wrong number of parameters specified, etc.

exception Warning

Exception raised for important warnings like data truncations while inserting, etc.

adapters

Adapters are used to convert Python types into JDBC types.

Adaptors are stored in a mapping from incoming type to an adapter function. Adapters set on a connection apply only to that connection. Adapters can be overriden when calling the .execute*() method.

Adapters can also be set on the JDBC types directly.

autocommit

Property controlling autocommit behavior.

By default connects are not autocommit. Setting autocommit will result in commit and rollback producing a ProgrammingError.

Type:bool
close()

Close the connection immediately (rather than whenever .__del__() is called).

The connection will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the connection. The same applies to all cursor objects trying to use the connection. Note that closing a connection without committing the changes first will cause an implicit rollback to be performed.

commit()

Commit any pending transaction to the database.

Calling commit on a cooonection that does not support the operation will raise NotSupportedError.

connection

Get the JDBC connection that is backing this Python Connection object.

This can be used to retrieve additional metadata and other features that are outside of the scope of the DBAPI driver.

converters

Converters are applied when retrieving a JDBC type from the database.

cursor()

Return a new Cursor Object using the connection.

getters

Getters are used to retrieve JDBC types from the database following a .fetch*().

Getters should be a function taking (connection, meta, col) -> JDBCTYPE

rollback()

Rollback the transaction.

This method is optional since not all databases provide transaction support. Calling rollback on a cooonection that does not support will raise NotSupportedError.

In case a database does provide transactions this method causes the database to roll back to the start of any pending transaction. Closing a connection without committing the changes first will cause an implicit rollback to be performed.

setters

Setter are used to set parameters to .execute*() methods.

Setters should be a function taking (connection, meta, col, type) -> JDBCTYPE

typeinfo

The list of types that are supported by this driver.

This is useful to find out the capabilities of the driver.

Type:list

Cursor Objects

These objects represent a database cursor, which is used to manage the context of a fetch operation. Cursors created from the same connection are not isolated, i.e., any changes done to the database by a cursor are immediately visible by the other cursors. Cursors created from different connections may or may not be isolated, depending on how the transaction support is implemented (see also the connection’s rollback and commit methods).

class jpype.dbapi2.Cursor(connection)

Cursors are used to execute queries and retrieve results.

Part PreparedStatement, part ResultSet, Cursors are a mixture of both. The native resultSet can be accessed with resultSet.

Cursors are managed and can be as part of a Python with statement. Cursors close automatically when they are garbage collected, at the end of a with statement scope, or when manually closed. Once a cursor is closed all operations using the database will raise an Error.

arraysize

Specify the number of rows to fetch with .fetchmany().

This read/write attribute specifies the number of rows to fetch at a time with .fetchmany(). It defaults to 1 meaning to fetch a single row at a time.

callproc(procname, parameters=(), *, types=None)

Call a stored procedure.

(Not all JDBC drivers support this method)

Call a stored database procedure with the given name. The sequence of parameters must contain one entry for each argument that the procedure expects. The result of the call is returned as modified copy of the input sequence. Input parameters are left untouched, output and input/output parameters replaced with possibly new values.

For type output and input/output arguments, it is best to use types keyword argument to select the appropriate getters for the returned arguments. Converters are applied to output parameters.

The procedure may also provide a result set as output. This must then be made available through the standard .fetch*() methods.

close()

Close the cursor now (rather than whenever __del__ is called).

The cursor will be unusable from this point forward; an Error (or subclass) exception will be raised if any operation is attempted with the cursor.

description

Description is read-only attribute is a sequence of 7-item sequences.

Each of these sequences contains information describing one result column:

  • name
  • type_code
  • display_size
  • internal_size
  • precision
  • scale
  • null_ok

This can only be used if the last query produced a result set.

execute(operation, parameters=None, *, types=None, keys=False)

Prepare and execute a database operation (query or command).

Parameters may be provided as sequence and will be bound to variables in the operation. Variables are specified in a qmark notation. JDBC does not support mapping style parameters.

After executing a statement, the rowcount will be updated. If the statement has no result set then the rowcount will be -1. A statement can produce multiple result sets. Use .nextset() to traverse the sets.

Parameters:
  • operation (str) – A statement to be executed.
  • parameters (list, optional) – A list of parameters for the statement. The number of parameters much match the number required by the statement or an Error will be raised.
  • keys (bool, optional) – Specify if the keys should be available to retrieve. (Default False)
Returns:

This cursor.

executemany(operation, seq_of_parameters, *, types=None, keys=False)

Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters.

Modules are free to implement this method using multiple calls to the .execute() method or by using array operations to have the database process the sequence as a whole in one call.

Use of this method for an operation which produces one or more result sets constitutes undefined behavior, and the implementation is permitted (but not required) to raise an exception when it detects that a result set has been created by an invocation of the operation.

The same comments as for .execute() also apply accordingly to this method.

Parameters:
  • operation (str) – A statement to be executed.
  • seq_of_parameters (list, optional) – A list of lists of parameters for the statement. The number of parameters much match the number required by the statement or an Error will be raised.
  • keys (bool, optional) – Specify if the keys should be available to retrieve. (Default False) For drivers that do not support batch updates only that last key will be returned.
Returns:

This cursor.

fetchall(*, types=None, converters=<object object>)

Fetch all (remaining) rows of a query result, returning them as a sequence of sequences (e.g. a list of tuples). Note that the cursor’s arraysize attribute can affect the performance of this operation.

An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.

fetchmany(size=None, *, types=None, converters=<object object>)

Fetch multiple results.

Fetch the next set of rows of a query result, returning a sequence of sequences (e.g. a list of tuples). An empty sequence is returned when no more rows are available.

The number of rows to fetch per call is specified by the parameter. If it is not given, the cursor’s arraysize determines the number of rows to be fetched. The method should try to fetch as many rows as indicated by the size parameter. If this is not possible due to the specified number of rows not being available, fewer rows may be returned.

An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.

Note there are performance considerations involved with the size parameter. For optimal performance, it is usually best to use the .arraysize attribute. If the size parameter is used, then it is best for it to retain the same value from one .fetchmany() call to the next.

fetchone(*, types=None, converters=<object object>)

Fetch the next row of a query result set, returning a single sequence, or None when no more data is available.

An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.

lastrowid

Get the id of the last row inserted.

This is not supported on all JDBC drivers. The .execute*() must have been executed with keys set to True.

Returns:None if there is no rowid, the rowid if only one row was inserted, or a list of row ids if multiple rows were inserted.
nextset()

Get the next result set in this cursor.

Not all databases support multiple result sets.

This method will make the cursor skip to the next available set, discarding any remaining rows from the current set.

If there are no more sets, the method returns None. Otherwise, it returns a true value and subsequent calls to the .fetch*() methods will return rows from the next result set.

An Error (or subclass) exception is raised if the previous call to .execute*() did not produce any result set or no call was issued yet.

parameters

(extension) Parameters is read-only attribute is a sequence of 6-item sequences.

Each of these sequences contains information describing one result column:

  • type_name
  • jdbc_type
  • parameter_mode (1=in, 2=in/out, 4=out)
  • precision
  • scale
  • null_ok

This can only be used after execute or callproc.

resultSet

Get the Java result set if available.

The object will be closed on the next call to .execute*().

rowcount

This read-only attribute specifies the number of rows that the last .execute*() affected (for DML statements like UPDATE or INSERT).

The attribute is -1 in case no .execute*() has been performed on the cursor or the rowcount of the last operation is cannot be determined by the interface. JDBC does not support getting the number of rows returned from SELECT, so for most drivers rowcount will be -1 after a SELECT statement.

setinputsizes(sizes)

This can be used before a call to .execute*() to predefine memory areas for the operation’s parameters.

sizes is specified as a sequence — one item for each input parameter. The item should be a Type Object that corresponds to the input that will be used, or it should be an integer specifying the maximum length of a string parameter. If the item is None, then no predefined memory area will be reserved for that column (this is useful to avoid predefined areas for large inputs).

This method would be used before the .execute*() method is invoked.

(not implemented)

setoutputsize(size, column=None)

Set a column buffer size for fetches of large columns (e.g. LONGs, BLOBs, etc.).

The column is specified as an index into the result sequence. Not specifying the column will set the default size for all large columns in the cursor.

(not implemented)

Cursors can act as an iterator. So to get the contents of table one could use code like:

with connection.cursor() as cur:
    cur.execute("select * from table")
    for row in cur:
       print(row)

SQL Type Constructors

Many databases need to have the input in a particular format for binding to an operation’s input parameters. For example, if an input is destined for a DATE column, then it must be bound to the database in a particular string format. Similar problems exist for “Row ID” columns or large binary items (e.g. blobs or RAW columns). This presents problems for Python since the parameters to the .execute*() method are untyped. When the database module sees a Python string object, it doesn’t know if it should be bound as a simple CHAR column, as a raw BINARY item, or as a DATE.

This is less of a problem in JPype dbapi2 than in a typically dbapi driver as we have strong typing backing the connection, but we are still required to supply methods to construct individual SQL types. These functions are:

jpype.dbapi2.Date(year, month, day)

This function constructs an object holding a date value.

jpype.dbapi2.Time(hour, minute, second)

This function constructs an object holding a time value.

jpype.dbapi2.Timestamp(year, month, day, hour, minute, second, nano=0)

This function constructs an object holding a time stamp value.

jpype.dbapi2.DateFromTicks(ticks)

This function constructs an object holding a date value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

jpype.dbapi2.TimeFromTicks(ticks)

This function constructs an object holding a time value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

jpype.dbapi2.TimestampFromTicks(ticks)

This function constructs an object holding a time stamp value from the given ticks value (number of seconds since the epoch; see the documentation of the standard Python time module for details).

jpype.dbapi2.Binary(data)

This function constructs an object capable of holding a binary (long) string value.

For the most part these constructors are largely redundant as adapters can provide the same functionality and Java types can directly use to communicate type information.

JDBC Types

In the Python DBAPI2, the SQL type system is normally reduced to a subset of the SQL types by mapping multiple types together for example STRING covers the types STRING, CHAR, NCHAR , NVARCHAR , VARCHAR, and OTHER. JPype dbapi2 supports both the recommended Python types and the fine grain JDBC types. Each type is represented by an object of type JBDCType.

class jpype.dbapi2.JDBCType(name, code=None, getter=None, setter=None)
get(rs, column, st)

A method to retrieve a specific JDBC type.

To use a getter add the fetch method to the JDBC type matching the column type to be pulled. For example, to set the getter for FLOAT to use the OBJECT getter, use cx.getter[FLOAT] = OBJECT.get.

Not all getters are available on all database drivers. Consult the database driver documentation for details.

set(ps, column, value)

A method used to set a parameter to a query.

To use a setter place the set method in the setter dict corresponding. For example, if the database supports Blob types, the default handler for BLOB can be changed from OBJECT to BLOB with cx.setter[BLOB] = BLOB.set.

Not all setters are available on all database drivers. Consult the database driver documentation for details.

The following types are defined with the correspond Python grouping, the default setter, getter, and Python type. For types that support more than one type of getter, the special getter can be applied as the converter for the type. For example, the defaulf configuration has getter[BLOB] = BINARY.get, to get the Blob type use getter[BLOB] = BLOB.get or specify it when calling use.

Group JDBC Type Default Getter Default Setter PyTypes Special Getter
DATE DATE getDate setDate datetime.datetime  
DATETIME TIMESTAMP getTimestamp setTimestamp datetime.datetime  
TIME TIME getTime setTime datetime.datetime  
DECIMAL DECIMAL getBigDecimal setBigDecimal decimal.Decimal  
DECIMAL NUMERIC getBigDecimal setBigDecimal decimal.Decimal  
FLOAT FLOAT getDouble setDouble float  
FLOAT DOUBLE getDouble getDouble float  
FLOAT REAL getFloat setFloat float  
NUMBER BOOLEAN getBoolean setBoolean bool  
NUMBER BIT getBoolean setBoolean bool  
NUMBER TINYINT (0..255) getShort setShort int  
NUMBER SMALLINT (-2^15..2^15) getShort getShort int  
NUMBER INTEGER (-2^31..2^31) getInt getInt int  
NUMBER BIGINT (-2^63..2^63) getLong getLong int  
BINARY BINARY getBytes setBytes bytes  
BINARY BLOB getBytes setBytes bytes getBlob
BINARY LONGVARBINARY getBytes setBytes bytes  
BINARY VARBINARY getBytes setBytes bytes  
TEXT CLOB getString setString str getClob
TEXT LONGNVARCHAR getString setString str  
TEXT LONGVARCHAR getString setString str  
TEXT NCLOB getString setString str getNClob
TEXT SQLXML getString setString str getSQLXML
STRING NVARCHAR getString setString str  
STRING CHAR getString setString str  
STRING NCHAR getString setString str  
STRING VARCHAR getString setString str  
           
ASCII_STREAM getAsciiStream      
BINARY_STREAM getBinaryStream      
CHARACTER_STREAM getCharacterStream      
ASCII_STREAM getAsciiStream      
BINARY_STREAM getBinaryStream      
CHARACTER_STREAM getCharacterStream      
NCHARACTER_STREAM getNCharacterStream      
URL getURL      

Some of these types never correspond to a SQL type but are used only to specify getters and setters for a particular parameter or column.

Other

The default getter will attempt to look for the column type by name if the type is OTHER. This allows for user defined types to be added if supported by the database.

User defined types

A user can declare a new type using JDBCType. The arguments are the name of new type which must match a SQL typename. Use typeinfo on the connection to get the list of available types.

It may necessary to define a custom getter function which defining a new type so that the custom return type accurately reflects the column type.

class JSONType(dbapi2.JDBCType):
   def get(self, *args):
       rc = JDBCType.get(self, *args)
       # Custom return converter here
       return rc
JSON = JSONType("JSON")

Conclusion

This wraps up the JPype dbapi2 module. Because JDBC supports many different dataase drivers, not every behavior is defined on every driver. Consult the driver specific information to determine what is available.

The dbapi2 does not fully cover all of the capabilities of the JDBC driver. To access functions that are not defined in DBAPI2, the JDBC native objects can be accessed on both the connection and the cursor objects.