Record Set

An in-memory representation of tabular data

In the last twenty years, the dominant way to represent data in a database is the tabular form of relational database. Backed by database companies big and small, and a fairly standard query language, almost every new development I see uses relational data.

On top of this came a wealth of tools to help build UI's quickly. These data aware UI frameworks rely on the fact that the underlying data is relational, and provide UI widgets of various kinds that make it easy to view and manipulate this data with hardly any programming.

The dark side of these environments, was the fact that while they made display and simple updates ridiculously easy, they had no real facilities to place any business logic. Any validations beyond the "is this a valid date", and any business rules or computations have no good place to go. Either they get jammed into the database as stored procedures or they get mingled with UI code.

The idea of the Record Set is to have your cake and eat it, by providing an in memory structure that looks exactly like the result of a SQL query, but can be generated and manipulated by other parts of the system.

How it Works

A Record Set is usually something that you won't build yourself. Usually it's provided by a vendor of the software platform you are working with. Examples include the Data Set of ADO.NET and the row set of JDBC 2.0.

The first essential element of a Record Set is that it looks exactly the same as the result of a database query. This way you can use the classical two-tier approach of issuing a query and throwing the data directly into a UI with all the ease that these two-tier tools give you. The second essential element is that you can easily build a Record Set yourself, or take a Record Set that is the result of a database query and easily manipulate it with domain logic code.

Although platforms often give you Record Set you can create them yourself. The problem is that there isn't that much point without the data-aware UI tools, which you would also need to create yourself. However it's fair to say that the approach of building a Record Set structure as a list of maps, which is common in dynamically typed scripting languages, is a fair example of Record Set.

The ability to disconnect the Record Set is very valuable. A disconnected Record Set is one that is separated from it's link to the data source. This allows you to pass the Record Setaround a network without having to worry about database connections. Furthermore if you can then easily serialize the Record Set it can also act as a Data Transfer Object for an application.

Disconnection raises the question of what happens when if you update the Record Set. Increasingly platforms are allowing the Record Set to be a form of Unit of Work. In this way you can take the Record Set, modify it, and then return it to the data source to be committed. A data source can typically use Optimistic Offline Lock to see if there's any conflicts, and if not write the changes out to the database.

When to Use it

To my mind the value of Record Set comes from having an environment that relies on it as a common way of manipulating data. If you have a lot of UI tools that use Record Set, then that's a compelling reason to use them yourself. If you have such an environment, then that's a big reason to use Table Module to organize your domain logic. In this style you get a Record Set from the database, pass it to a Table Module to calculate derived information, pass it to a UI for display and editing, pass it back to a Table Module for validation, and that commit updates to the database.

In many ways the appearance of the tools that make Record Set so valuable occurred because of the ever-presence of relational databases and SQL. There hasn't really been an alternative common structure and query language. But now, of course, there is XML which has a widely standardized structure and a query language in XPath. Thus I think it's likely that we'll tools appear that use a hierarchic structure in the same way that tools now use Record Set. In this case this is perhaps really a particular case of a more generic pattern: something like Generic Data Structure. But I'll leave thinking about that pattern until then.