Journaled Files and Transactions

Note: JKEYED files are deprecated. SQL Transactions are not specific to BBj, and more information on using them correctly is available online; see for example Java Tutorials: Using Transactions. Use the SQLCOMMIT and SQLROLLBACK verbs to use SQL transactions in BBj. In general, you should avoid using transactions as much as possible and keep them as brief as possible when you do use them. Use them when a set of operations needs to be treated as atomic, e.g. either all three inserts should happen or none of them.

Overview

BBj 2.0 and higher has two new features that safeguard data and preserve its integrity: journaled files and transactions.

Journaled files and transactions let mission-critical file operations be done atomically. In other words, applications can process a set of file operations without the risk that in a system crash only a portion of the set will be saved to disk. A journaled system with transactions controls the process of grouping file operations so that either all operations are on disk or no operations are on disk.

The principal benefit of journaled files is their resistance to corruption. Because all operations on journaled files are atomic, the operations either completely succeed or completely fail. Files cannot be left in a corrupted state by a partially completed update.

Transactions build on this characteristic of journaled files, enabling the application programmer to define sets of operations to be treated as a single operation against one or more files. This set of operations will either completely succeed or completely fail.

BBj offers both read-write transactions and read-only transactions. Read-only transactions are somewhat faster than read-write transactions because the READs are not logged. The BBj program uses transaction handles to access the transaction functionality.

To set up a Journaled file system, refer to Enterprise Manager - Server Information.

Important Difference With Journaled Files

It is very important to understand a difference between the Journaled file system and the traditional BBx file system. The traditional BBx file system allows for creating directories and subdirectories while the Journaled file system does not. During the configuration process (Configuring BBjServices via the Enterprise Manager ) the administrator sets the location of the base directory for the Journaled file system. When the Journaled file system creates files, it stores those files in this base directory. The only accessible files are those created by the Journaled file system and located in this base directory.

Using Journaled Files

To allocate a handle for a read/write transaction, use the syntax:

id = TOPEN(int{,ERR=stmt})

For a read-only transaction add a MODE as follows:

id = TOPEN(int,MODE="READONLY"{,ERR=stmt})

The value returned by TOPEN(), the transaction id, should be saved by the program. It will be used when opening a journaled file as well as when doing a commit or rollback.

The integer argument to the TOPEN() function, the isolation level, determines how multiple concurrent transactions interact. There are three isolation levels – 1, 2 and 3. When using isolation level 3, the filesystem guarantees that data read by the application cannot be changed by any other application until after executing either a commit or a rollback. When using isolation level 2, the filesystem guarantees that if another program is in the process of changing the data that is being read then the read will not return until the other application's transaction is either committed or rolled back. When using isolation level 1, the application is able to read data that has been written by another application even if the transaction of the other application has not yet been committed or rolled back.

At isolation level 1, the application obtains results more quickly. However, the results that are obtained may have been written by another application that could subsequently rollback the write.

Depending on the isolation level of the transaction with which the JKEYED file is associated, input and output to the file throws an !ERROR=0 when the action violates the rules of that isolation level.

Some IO operations reposition the file pointer. In some situations, an !ERROR=0 is generated when the file pointer is repositioned (because it violates the rules of that isolation level). Note that the READ verb has an implicit DIR=1 if no DIR= option is specified.

After opening one or more journaled files using the JOPEN verb (see example below), the program can read from and write to those channels using the usual READ and WRITE verbs. At some point the program will need to either commit or rollback. The syntax for commit and rollback is:

TCOMMIT(id{,ERR=stmt})
TROLLBACK(id{,ERR=stmt})

TCOMMIT causes all changes made during the transaction to be written to disk. TROLLBACK causes all changes made in the transaction since the last TCOMMIT to be undone.

A transaction may also be closed and all changes rolled back that were made since the last commit by using the TCLOSE verb. Use the following syntax:

TCLOSE(id{,ERR=stmt})

Journaled files are a new file type in BBj. Only journaled files are able to interact with transactions. All WRITEs to a journaled file must involve a transaction. A journaled file can either be opened inside of a transaction or in auto-commit mode. In a transaction, all file operations take place in the transaction and are controlled by the commit and rollback of that transaction. A file opened in auto-commit mode involves an implicit transaction that will commit each operation as it occurs.

To associate a journaled file with an open transaction:

JOPEN(channelno,TID=id{,ERR=stmt})fileid

To open a journaled file in auto-commit mode:

JOPEN(channelno{,ERR=stmt})fileid

The auto-commit mode is inherently slower because an implicit TCOMMIT is executed after every operation. However, it allows existing code to use journaled files without adding explicit TCOMMIT and/or TROLLBACK statements. A JKEYED file opened in auto-commit mode has an isolation level of 3.

Journaled files are created with the JKEYED verb, which uses syntax identical to the MKEYED and XKEYED verbs, with one important difference: JKEYED files work with true variable-length records. The defined record size is actually a maximum record size. Unlike all other record-oriented BBx and BBj file types, JKEYED records are not padded to the maximum defined length with $00$ characters. A READ RECORD on a JKEYED channel will get back exactly the number of characters that were originally written to the file.

To erase a journaled file, use the JERASE verb:

JERASE filename{,ERR=stmt}

The names used to refer to journaled files are not complete path names. All journaled files reside under a specific data directory, defined by the administrator using the BBj Enterprise Manager. The purpose of this data directory is to facilitate backups of the journaled system.

Transactions build on this ability of journaled files to enable sets of operations to be treated as a single operation on one or more files. This means that, in addition to the set of operations being treated as atomic, that the operations can be rolled back as a unit. The operations can be discarded as if none of them has ever happened. In the case of a system crash, the operations will all be rolled back and none of them will have taken place. Conversely, all of the operations in any transaction that has been committed are guaranteed to have been stored.

All open transactions are closed by the BEGIN, END, START, and RELEASE verbs. This has the effect of doing a rollback on any uncommitted transactions.

Restoring From Backup

Use the utility class called "com.basis.server.JournalUtil" to restore a journaled filesystem from a backup. In order to use this utility class, the environment variable CLASSPATH will have to be set to all .jar files in the <bbj install dir>/lib directory. If this class is run without any arguments, there will be a brief usage summary as shown below:

Usage: java com.basis.server.JournalUtil <action> <action arguments> ...

create <location> ...

restore <backup directory> <data directory> <location> ...

recreate <backup directory> <log directory> <data directory> <location> ...

The three possible actions are as follows:

create	Creates a new journal, which is equivalent to creating a new journal from the Enterprise Manager.
restore	Restore an entire journaled system from a backup. This should restore the journaled system to the current state at the time in which the backup was made.
recreate	Recreate a journaled system from a backup and a current journal. This will take a backup and recreate a journaled system up to the time that the current journal ends.

The arguments are defined as follows:

<location> ...	A list of one or more directories where the new journal files will be located.
<backup directory>	Specifies the directory where the backup files can be found, which corresponds to the directory specified while doing a backup.
<data directory>	Specifies the directory in which the new data files will be placed.
<log directory>	Specifies the directory of the log file to restore.

The "create" action will not normally be used. In general, journaled files should be created from the BBj Enterprise Manager. The "restore" action can be used either to restore a journaled system from a backup, as in rebuilding a machine, or to copy a journaled system from one machine to another. The "recreate" action is used very rarely. It should only be used when all of the data files in a running journaled system are lost. For example, the "recreate" action could be used if the data drive has a fatal hard drive crash. It can use a backup and existing copies of the current journal to restore the journaled system to its original state at the time of the crash. It is recommended that the backup be located on a different drive.