The first XML file we are going to import is the UCF_Issuers_List, as this is one of the simplest of all of the UCF lists to both import and work with.

The XSD for this list is here:

http://unifiedcompliance.com/NFI/XSDs/UCF_Issuers_v2_1.xsd

We are going to be using this XSD to create an XSL (an Extensible Style Sheet) which will be used to transform the data in our matching XML sample file into something that the database can understand and import.

Your base transform

The method we espouse when creating XSL files for importing is to begin with a database-centric XSL that will act as your base. This base transform will have four jobs; define tables, define rows define columns and define fields in a language that your database can understand. Because this is specific to each and every database type out there, we aren't going to cover what goes into each of the four template definitions as it just becomes confusing. Take it from us, a good Google search or good book on your database of choice and XML will yield your base transform table.

If you want to see what our Filemaker version looks like, you can grab it from our website here:

http://unifiedcompliance.com/it_compliance/xml_standard/xslt/FileMaker.xslt

Your specific instance transforms must equal the XSD

The first thing you'll want to do (so that you can replicate the process multiple times) is set up the specific instance XSL for import. The foundation information in that file that you'll need follows:

Notice that there are two areas where we said you'd be naming individual fields and later their corresponding XML simple elements. Those two areas are important as that's what we'll be covering next.

Building out your import XSL specifics from the XSD structure

The Issuers List XSD has a single list element:

UCF_Issuers_List

The list contains a single Item complex element:

UCF_Issuer_Item

And that item is broken down into two complex elements of Meta Data and Basic Info:

UCF_Meta_Data and UCF_Basic_Info

Therefore, our XSL template is going to have to account for gathering data from both the Meta Data and the Basic Info complex elements.

The Meta Data complex element for this file is comprised of eight simple elements as shown in the diagram that follows.

The Meta Data complex element

The Basic Info complex element for this file is comprised of five simple elements as shown in the diagram that follows.

The Basic Info complex element

There are two places in your XSL file that you are going to need to leverage, and the two should correspond. The first goes into the top portion of your XSL template where we said you'd be naming your fields. The XSL for that portion looks like this for the first element (UCF_Issuer_Release_Version):

The above names the field within the database. The code below extracts the corresponding data from the XML file. This code is placed in the bottom portion of your XSL template where we said you'd be naming the simple elements of the XML file. Notice that the UCF_Meta_Data complex element is called in the string that calls the UCF_Issuer_Release_Version element. Each of the simple elements must be preceeded by the complex element to which it belongs.

You will repeat this dual process of naming the fields and To put this into perspective, if you were only pulling the first two elements (Release Version and ID), the entire XSL template would look like this:

Using the XSL to create a table in a database

The database that we are going to use, as mentioned earlier, is the FileMaker database. Our blank example is below which currently only has a blank layout and absolutely no tables in it:

Blank Filemaker database

We are going to import our sample Issuers List XML file by selecting the XML Data Source option in the Import Records menu, and then specifying both the XML source and our XSL import we just created.

Importing the XML data into the blank database

This will bring up a Field Mapping dialog that allows FileMaker to create a new table matching the XML elements we just defined.

Mapping the XML elements to a new table and new fields in that table

Clicking the Import button creates the new table, assigns a new layout, and imports the records. We have the entire process recorded online (viewable here).

If you have a copy of the forms, templates, and samples for the UCF, you'll have a directory called Import XSLTs. In that directory you'll find not only this XSL example, but examples to fit every one of the UCF's export tables.

You will also have a directory called UCF_Sample_Data that has all of the XML samples for all of the tables in the UCF.

In addition, there is a sample XML Teaching Tool Filemaker database that goes along with the XMLs and XSLs.

Most folks don't think long term when they start to put together compliance programs. They don't think that the stack of compliance regulations on their desk as a list of regulations. They don't think of the stack of audit questions as a list of audit questions.

We do. Which is probably why we built the Unified Compliance Framework™. So lets start with the ID elements.

IDs must be consistently formatted numbers

Think of your Social Security Number or Passport number. The format of NNN-NN-NNNN for US Social Security Numbers is a consistent format for every person who was ever issued a number. Some number will start with a zero, such as the last number set in 312-98-0422. And that's okay, as there is as much information in the pattern as there is in the number itself.

To a computer system, 00009 is much different than 9. If your compliance program is going to be designed to hold a million records, then your ID number system should begin with 0000000, not 0 and definitely not 1. Why begin with all zeros? You have to have a root record in any well formed system. That root record should always be your equivalent of a null value.

IDs must be validated

When entering numbers, we humans have a tendency to screw up the entry or copying of those numbers. A Dutch mathematician named Jacobus Verhoeff conducted a study of 12,000 numerical errors and from that, proposed a check digit calculation scheme that catches all single errors as well as all adjacent transpositions and most other errors.

To ensure that the IDs assigned by the system have integrity during input as well as distribution while being transferred into various formats (such as Excel, Word, Text, XML), each ID should have its own checksum value stored in a checksum field. Currently, the best methodology we can find for creating and verifying the checksum follows the Verhoeff calculation format.

Here's the Verhoeff calculation as a formula you can use in your database:

Format
Verhoeff ( numericString ; index ; checkSum )

Parameters
numericString - a string of numeric characters (digits) or field containing numeric characters
index - indicates the digit position of the current iteration - needs to be initialized to zero (0) when calling the function
checkSum - indicates the check digit of the current iteration - needs to be initialized to zero (0) when calling the function
Data type returned = number

Description
Returns the Verhoeff dihedral check digit of numericString. Use this function to verify a numeric string protected by Verhoeff check digit, or to generate the correct Verhoeff check digit for a given numeric string.

Calculation
Let ( [
n = Right ( numericString ; 1 ) ;
p = Let ( [
array = "01234567891576283094580379614289160435279453126870428657390127938064157046913258" ;
start = 10 * Mod ( index ; 8 ) + n + 1
] ;
Middle ( array ; start ; 1 )
) ;
d = Let ( [
array = "0123456789123406789523401789563401289567401239567859876043216598710432765982104387659321049876543210" ;
start = 10 * checkSum + p + 1
] ;
Middle ( array ; start ; 1 )
) ;
len = Length ( numericString ) ;
nextString = Left ( numericString ; len - 1 )
] ;
Case ( len > 1 ; Verhoeff ( nextString ; index + 1 ; d ) ; d )
)

This allows us to enter a calculation whereby we can either create a check digit for our ID field, or after editing or copying the ID to another location, we can call the function and check for the validity of an ID using its Verhoeff check digit:

Not Verhoeff(ID & CheckDigit; 0 ; 0 ) = Valid

IDs must be unique and persistent

Some might think this goes without saying, but we've seen instances in systems where this was not the case. Yes, once an ID number has been assigned to any record in any list, it should never be reused.

On a similar note, once you've assigned an ID to any record in your system, you should never delete that ID. Does this mean you can't get rid of the records? Yes and no. You will need to deprecate records, meaning that you keep the ID in the system and then notify those who land in the record with that ID that the record is no longer available. The reason you need to use this deprecation method instead of just deleting the ID is that databases don't have an easy way of looking for items that aren't there any longer. So keeping the ID there, but marking the record as deprecated, solves this problem.

In the old days, when our founder's hair was dark brown and his belly was flat (or at least flatter than today), databases were described using Document Type Definition (DTD) files that outlined the document or database structure. DTDs have gone the way of the Sony Walkman. XSDs, or XML Schema Definitions, have taken their place as the language and grammar for the markup allowed in XML files and structured data environments.

What's an XML Schema?

An XML Schema:

• defines elements that can appear in a document

• defines attributes that can appear in a document

• defines which elements are child elements

• defines the order of child elements

• defines the number of child elements

• defines whether an element is empty or can include text

• defines data types for elements and attributes

• defines default and fixed values for elements and attributes

One of the greatest strengths of XML Schemas is the support for data types. With support for data types:

• It is easier to describe allowable document content

• It is easier to validate the correctness of data

• It is easier to work with data from a database

• It is easier to define data facets (restrictions on data)

• It is easier to define data patterns (data formats)

• It is easier to convert data between different data types

Setting up a simple list of websites and URLs in XML

Let's go through a quick scenario of creating a simple list of websites in XML format. For our scenario we want to track the following fields:

• ID

• Site Name

• Site URL

• Date Modified

The XML List

The XML representation of our list would look like this:

The XML Schema for the list

The following example is the XML Schema for the two record list shown above:

Notice that it says the elements named Website_List and Website are a "complex type". This is because they contain other elements. In other words, Website_List contains Website, and Website contains ID, Site_Name, etc. Those sub-elements (ID, Site_Name, etc.) are considered simple types because they do not contain other elements.

Defining Complex Elements

Complex elements are simply elements that have other elements inside of them. In our example, WebSite is a complex element that contains several simple elements:

Sequence tells us that the elements that follow must appear in order. This is a good thing, as it ensures that the XML output attached to the XSD is consistent.

Defining Simple Elements

Anyone managing a compliance list in XML is going to run across these types of XML elements that are built-in data types:

• xs:string

• xs:decimal

• xs:integer

• xs:boolean

• xs:date

• xs:time

• xs:anyURI

If you looked close at our example above, it used the xs:string, xs:integer, xs:date, and xs:anyURI data types. When an XML element or attribute has a data type defined, it puts restrictions on the element's or attribute's content. If an XML element is of type "xs:date" and contains a string like "Hello World", the element will not validate. Nor will it validate with the date format "1/1/2010" because the XML format for both date and time are very specific in order to avoid confusion between the US date formats (month/day) and the rest of the world's date formats (day/month). With XML Schemas, you can also add your own restrictions to your XML elements and attributes.

Adding Occurrence Indicators

Let's take the same XML list and add a new element called the "cool factor". But we are only going to add it to one of the two records below:

When representing this new simple element in the XSD, we have to ensure that we can tell the XML structure that the element isn't always required. Therefore, instead of representing the element this way:

It has to be represented this way:

Adding Restricted Values

Restrictions on XML elements are called facets. One of the things we've talked about in reference to IDs is that they should be uniform in length (001 versus 1). A great part of XML is that restrictions, or facets, can be set for each and every element in an XML Schema. Let's take our ID integer for example, and say that we wanted to restrict it to being a three digit number. The schema representation changes this:

To this:

Notice that there are three sets of "[0-9]" in the pattern. This means that it is looking for three characters from 0 through 9. No more, no less.

Another type of facet is much like assigning a pop-up list to an element. Let's say that we want to restrict what the user can enter for the "Cool Factor" for our websites, and we want this Cool Factor to either be Okay, Cool, or Very Cool. We would change the element type from:

To:

The URL for the Common Elements XML structure is as follows:

http://unifiedcompliance.com/NFI/XSDs/UCF_Common.xsd

The primary goal of the Common Elements XML schema is to provide a single source for re-used XML elements and complex elements. As such, when we find that we are re-using elements between XML schemas, we will migrate the most-used elements to this file.

The elements

Here is the current listing of the XML common elements:

UCF_Metric_Chart_Type

This is a listing of all of the various kinds of metric chart types, such as

• • Bar

  • Pie

  • Stacked bar

  • Waterfall

UCF_ID_Type

This defines the string restriction for the original ID type, only used in the Control ID entries. This ID type is limited to a five character string.

UCF_ID2_Type

This defines the string restriction for the rest of the ID types used by the UCF. This ID type is limited to a seven character string.

UCF_Text_Type

This text type is used to limit certain fields to 255 characters in length.

UCF_AD_Parent_Type

This is an enumerated list of all of the parent categories found within the UCF. The current list is as follows:

• • Sarbanes Oxley Guidance

  • Banking and Finance Guidance

  • NASD NYSE Guidance

  • Healthcare and Life Science Guidance

  • Energy Guidance

  • Payment Card Guidance

  • US Federal Security Guidance

  • US Internal Revenue Guidance

  • Records Management Guidance

  • NIST Guidance

  • General Guidance

  • US Federal Privacy Guidance

  • System Configuration Guidance

  • ISO Guidance

  • ITIL Guidance

  • EU Guidance

  • UK and Canadian Guidance

  • Latin American Guidance

  • Other European and African Guidance

  • Asia and Pacific Rim Guidance

  • Organizational Guidance

  • Other Configuration Guidance

  • Vendors

  • Deprecated

  • US State Laws and Protectorates Guidance

Information technology assets are the combination of logical and physical components and resources and are grouped into the specific classes (Operating System [includes drivers], Application [includes drivers], Storage, Hardware, Network, Power or Air, and Facility [including containers]). Assets that need to be individually managed are also configuration items. For example, the door lock on a computer room or a consumable item might not be a configuration item, but the software firewall on a notebook computer would be a configuration item. In the context of financial management, items below a specific value are not considered to be assets as it would not be cost effective to track and manage them. However, in the context of auditing information systems, even logs (of little financial worth in and of themselves) are considered assets because they are artifacts that need to be examined or tested.

Asset classes

This is the grouping of the various organizational assets into distinct assortments based upon a process of arranging them according to type. Currently, the UCF tracks the following asset classes:

1. Operating System [includes drivers]

2. Application [includes drivers]

3. Storage

4. Hardware

5. Network

6. Power or Air

7. Facility

Remember that all data, records, and databases are already being tracked through the information classification tables (databases should be listed as supporting records), and compliance documents, such as policies, standards, and procedures are also tracked separately. Let us go through each of the above to further define them.

Operating System

An Operating system provides the software platform which directs the overall activity of a computer, network or system, and on which all other software programs and applications can run. In many ways, choice of an operating system will effect which applications can be run. Operating systems perform basic tasks, such as recognizing input from the keyboard, sending output to the display screen, keeping track of files and directories on the disk and controlling peripheral devices such as disk drives and printers. For large systems, the operating system has even greater responsibilities and powers - becoming a traffic cop to makes sure different programs and users running at the same time do not interfere with each other. The operating system is also responsible for security, ensuring that unauthorized users do not access the system. Examples of operating systems are UNIX, DOS, Windows, LINUX, Macintosh, and IBM's VM. Operating systems can be classified in a number of ways, including: multi-user (allows two or more users to run programs at the same time - some operating systems permit hundreds or even thousands of concurrent users); multiprocessing (supports running a program on more than one CPU); multitasking (allows more than one program to run concurrently); multithreading (allows different parts of a single program to run concurrently); and real time (instantly responds to input - general-purpose operating systems, such as DOS and UNIX, are not real-time).

Application

In the broadest sense, the use of information resources (information and information technology) to satisfy a specific set of user requirements. When speaking about software, any program designed to perform a specific function directly for the user or, in some cases, for another application. A computer program designed to help people perform a certain type of work, including specific functions, such as payroll, inventory control, accounting, and mission support. Depending on the work for which it was designed, an application can manipulate text, numbers, graphics, or a combination of these elements. An application contrasts with systems program, such as an operating system or network control program, and with utility programs, such as copy or sort.

Storage

Storage is the catch-all term for any type of media and medium that holds information or information assets, such as a direct access storage device, an electronic storage system, a storage area network, offline storage, or even open storage.

Direct Access Storage Device

Any storage device, such as a hard disk, that provides the capability to access and/or manipulate data as required without having to access all preceding records to reach it. In contrast to direct or random access, sequential access devices, such as tape drives, require all preceding records to be read to reach the required data.

Electronic storage system

An electronic storage system is a system to prepare, record, transfer, index, store, preserve, retrieve, and reproduce files, organizational books and records.

Offline storage

Electronic records stored or archived on removable disk (optical, compact, etc.) or magnetic tape used for making disaster-recovery copies of records for which retrieval is unlikely. Accessibility to off-line media usually requires manual intervention and is much slower than on-line or near-line storage depending on the storage facility. The major difference between near-line data and offline data is that offline data lacks an intelligent disk subsystem, and is not connected to a computer, network, or any other readily-accessible system.

Open storage

Storage of classified information within an accredited facility, but not in General Services Administration approved secure containers, while the facility is unoccupied by authorized personnel.

Storage Area Network

A high-speed subnetwork of shared storage devices. A storage device is a machine that contains nothing but a disk or discs for storing data. A SAN's architecture works in a way that makes all storage devices available to all servers on a LAN or WAN. As more storage devices are added to a SAN, they too will be accessible from any server in the larger network. In this case, the server merely acts as a pathway between the end user and the stored data. Because stored data does not reside directly on any of a network's servers, server power is utilized for business applications, and network capacity is released to the end user. See also Network.

Hardware

The physical components of information technology including the computers, peripheral devices such as printers, discs, and scanners, and cables, switches, and other elements of the telecommunications infrastructure.

Network

A group of computers and associated devices that are connected by communications facilities in order to share resources. A network can involve permanent connections, such as cables, or temporary connections made through telephone or other communications links. A network can be as small as a local area network consisting of a few computers, printers, and other devices, or it can consist of many small and large computers distributed over a vast geographic area. A telecommunications medium and associated components responsible for the transmission of information. A local-area network (LAN) refers to connected computers and devices geographically close together (i.e. in the same building). A wide-area network (WAN) refers generally to a network of PC's or other devices, remote to each other, connected by telecommunications lines. Typically, a WAN may connect two or more LAN's together.

Power or Air

Power is an organizational asset that describes both full time power from an electrical grid and emergency power (whether from a generator or an uninterruptible power supply). Air is an organizational asset that ensures heating, cooling, and humidity are kept in the right proportions according to organizational asset needs.

Facility (includes containers)

A facility in terms of organizational assets can be anything from a lockable and transportable container, through stationary containers, through individual buildings, to campuses.

Containers

A container is any object that can be used to hold things, and in our context, information, information assets, or other organizational assets. Containers can either be strong plastic or metal box-like objects of standardized dimensions that can be loaded from one form of transport to another.

Assets as configurable items

A configurable item is a part of an asset specifically called out during the audit process. They are used as objects that help determine that status of the asset (the log as a configurable item to determine who accessed the asset), the configuration of the asset (a software configurable item being checked for its configuration status), or the activities surrounding an asset (the transit bill showing where a container has been). A configurable item may also help determine a specific process (such as a procedure document as audit configurable item), or define behavioral norms (a policy as an audit configurable item). It should be noted that configurable items are not evidence. A configurable item is the object itself submitted for testing or examination. The evidence is found within the configurable item -- it is the means by which an alleged matter of fact is established or disproved. If the configurable item is a log, then the evidence for whom accessed the system is the logged system access records within with configurable item.

All too often, we see IT security staff categorizing information with "public, internal, restricted, and secret" labels which provide very little in the way of describing to end users what they are supposed to do with information labeled as such. Worse yet, most of these classification practices only deal with the confidentiality of information and either partially or completely ignore guidelines surrounding the integrity of the information or the availability of the information. And yet, laws such as Sarbanes Oxley and international privacy laws ask us to provide guidance for the integrity of the information that falls under their jurisdiction and categorization. US laws provide governmental guidance through documents like NISPOM, ITAR, and EAR which are very strict about not only the confidentiality, but the availability (or non-availability) of the information in their classifications. The Payment Card Industry mandates protective measures for confidentiality, integrity, and availability for information that would be classified under their jurisdiction.

The reason for information categorization is to provide guidance to those who will first identify and then protect the confidentiality, integrity, and availability of that information.

In order to properly roll out an information categorization program, you will need to accomplish three goals.

• • Develop a general understanding of how the various authority documents define the various types of information and describe information categorization.

  • Understand, and internalize, who should use information classification, how they should use it, and when they should use it.

  • Roll out a basic information categorization program that links to your compliance control framework.

As always, the material in the UCF is harmonized across all authority documents we are currently working with.

The reason that the UCF team has taken this approach to crafting Compliance Documents is because we have had a very hard time (as well as everyone else) connecting the dots between Authority Documents stating your organization has to have certain policies, standards, and procedures, versus what gets put into the policies, standards, and procedures your organization uses. Until now, there hasn't been a direct correlation, or "norm" for what should go into these documents and how these documents should be linked back to the sources that call for them.

As a case in point, when we examined many of the online "policy builder" or "policy template" websites, we couldn't even find a policy for managing cryptographic controls in their "must have" lists. And of those websites that we did find advertising such a policy, we couldn't find any direct reference to the Authority Documents' controls specifically called for.

So our first aim is to create a structure to directly relate the content of Compliance Documents to the Authority Documents that call for the controls contained within them.

Our second aim is to dispel the myth that everything is a Policy. Many organizations are creating Policies that are really standards or checklists. And many folks who are purportedly "in the know" are calling procedures standards and visa versa. So after a lot of hoopla, angst, and research, we've broken things down as follows:

• • Policies deal with high-level behaviors. Therefore, a policy is put into play when the organization needs to coordinate and execute various activities or compliance controls.

  • Procedures are the operational how-to's or "do this then do that's" and therefore document processes or steps that people take in their functions, roles, and jobs. Groups of procedures form organizational Plans.

  • Standards and their informal counterparts, checklists, are normally used as a list of items to be appraised ("examine this, that, and the other") or acted on ("configure this, that, and the other").

Therefore, the UCF employs all of the Compliance Document types mentioned above in our Compliance Documents.

The structure of Compliance Documents

After reviewing, oh, about 500 variations of Compliance Documents, we found that there is a generic structure that can be derived and applied across all of the document types. That structure is defined here.

Title and Description

For now, we've chosen to "anchor" all of our Compliance Documents on a chosen control from within the UCF. For example, our first Compliance Document (which is a policy) is entitled Create and maintain a policy for encryption management and cryptographic controls, and that title is drawn directly from a harmonized control title within the UCF's Controls List (UCF CE ID 04546).

The description for the policy is a brief and to the point version of the Control Title that the Compliance Document is based upon.

Content of the Compliance Documents

The content of each of these Compliance Documents, whether they are policies, standards, procedures, or whatnot, can all be taken directly from the list of the UCF's controls. Beginning with version 2.1 of the UCF, all controls are written as active verbs. Therefore, "Digitally Sign Emergency and Critical e-mail notifications" [UCF CE ID 04841], can be used as a policy statement.

The great part about using the UCF as a basis for the content of Compliance Documents is that

1. the content can be linked directly back to each and every Authority Document that has called for it (by using the UCF CE IDs as references),

2. the UCF's inherent taxonomic genealogy can be brought directly into the document structure, and

3. as additions, changes, and deprecations to the UCF are made, they can easily be incorporated into the Compliance Documents.

The format of each content element

Our current methodology is to directly restate the UCF control as the main content element for each item in the list, followed by the Roles associated with that control and then the Assets associated with that control.

Generate strong keys [UCF CE ID 01299] Assigned to Role IDs: (0000101); Assigned to Asset IDs: (0001478)

This allows the end user to "de-scope" the individual elements within each policy if they don't have any of the assets (or assets like those) currently assigned to the control. It also serves as an indicator for which roles need to be assigned to the individual items.

The taxonomy of the content element

The UCF is a hierarchical taxonomy based upon control dependencies. In plain English, that means if the UCF says "do this" at level one, and then "do that" at level two, the "do that" is dependent upon the "do this" getting done. And if you think that was plain English, you need to be a lawyer. Even more plainly, we figured out what to do when you want to add section 2, and 2.1 to a document, followed immediately by section 5.8.7.2.2 and 6.3.9.2. Without a "taxonomic slider", you'd get an outline like this:

1 do this

1.1 do that

5.8.7.2.2 do something else

6.3.9.2 get some coffee

Not good. So we've built a "taxonomic slider" into our system to figure out that 2.1 must show up as dependent upon 2. And that in this particular schema, since neither 5.8.7.2.2 nor 6.3.9.2 have any dependent controls, they should both be at level 1. In the UCF's Compliance Documents taxonomy, the outline looks like this (much better):

1.0 do this

1.1 do that

2.0 do something else

3.0 get some coffee

Tracking content changes

Again, we come back to the database problem faced in the original Jurassic Park movie: they weren't tracking their dinosaurs the right way. We won't make that mistake. Once any particular UCF control has been added to a Compliance Document, it is in there and can't be removed. In order to not show the UCF control in an exported document, we mark the UCF controls as being deprecated like we do in each of our other lists. That simply means that the control has been deprecated from this particular Compliance Document, not the main UCF Controls list.

And speaking of that list, a control is automatically deprecated from a Compliance Document if the control is deprecated in the main UCF Controls List.

Scope and Assignment

Because each of the content items are individually mapped to both Roles and Assets, we can easily create a consolidated Scoping (Assets) and Assignment (Roles) section of the Compliance Document. The contents in each of these sections are automatically rolled up from the complete list of controls found within the document plus the addition of the mapped Assets and assigned Roles found in the base control used for the Compliance Document title.

This allows the organization to more easily go through the Compliance Document and pick out the controls that don't apply to them because they don't have those types of assets in their organization. It also helps them determine which Roles need bodies assigned to them if the Roles can't be traced back to actual humans who have to perform the tasks mentioned.

What we mean by that is within one organization the CIO might be in charge of security, while, in another organization, it is the CISO, and in yet another organization it is the Director of IT.

Therefore, we've pretty much thrown titles out of what we write about and instead cover roles. In so doing, we are closer to Microsoft's Operational Management Framework than any other system we've seen so far.

The NIST authority documents that focus on metrics, among the many documents that contain roles and responsibilities guidelines, assigns their responsibilities to titles instead of functional roles. For instance, they state that the Senior Agency Information Security Officer should be assigned the responsibility for

integrating information security measurement into the process for planning, implementing; evaluating, and documenting remedial actions to address any deficiencies in the information security policies; procedures, and practices of the organization.

Because each organization might not have someone of that title, and we at the UCF have to map to as broad an audience as possible, we've chosen to map such responsibilities to roles instead of titles. We've mapped the same type of responsibility to the role called Define Security Requirements and Implement Security Solutions. We've found that it is much easier for people involved in compliance projects to refer to common roles than titles. By focusing on roles, rather than titles, multi-disciplinary teams within medium to large sized organizations can:

• • clearly understand their functional roles and responsibilities within the project or operational environment, and

  • bring clarity to operational projects that involve partners and customers who might also have differing titles, but can clearly understand functional roles.

Notice that we said roles above instead of job descriptions. The UCF team has adopted and further developed one of the Microsoft Frameworks for dealing with the assignment of IT controls, namely, its IT Occupation Taxonomy version 3.0. Within the document the masterminds have identified critical roles as those baseline functions performed by the various members of the organization or its extended third parties. To a degree we share their views, but have chosen a different level of mapping activities to each role.

The basis of roles

We agree with Microsoft that, regardless of a person's title that labels a job role, it is the critical tasks performed that determine which job roles are the right match for anyone to participate as a team member on any given compliance effort. Therefore, instead of assigning an individual control to a job title, the UCF assigns the control to any number of roles that apply. As an example, for the control of "Report on the percentage of policy compliance reviews where there were no violations of compliance noted," we have assigned the responsibility for carrying out that control to the role of "perform monitoring and management."

Each role's title represents what that function is supposed to achieve. Remember that this is much different than a job title, such as CIO, Database Administrator or Security Manager. This doesn't describe a person. It describes the collective set of responsibilities that are assigned to the role. Hence, Role Titles are represented as "Define and Manage Business Value," "Conduct Security Administration," or "Manage IT and Compliance Policies and Standards." As such, any number of roles can be assigned to any different IT title to which the organization sees fit to assign the role.

Ensuring that roles are assigned

One of our key tenets also follows directly in line with Microsoft's views. Anyone can perform multiple roles. In addition, multiple people may perform the same roles. For example, most CIOs will be assigned multiple roles such as "Provide Strategic Business Direction for Technology", "Identify and Select Strategic Partners" and "Manage IT and Compliance Policies and Standards (by either editing or approving them)." At the same time, multiple people might also be assigned the same role of "Manage IT and Compliance Policies and Standards" through writing them, providing technical edits to them, etc.

One thing that is for certain: every organization should be able to identify at least one person to perform each of the roles. If an organization does not have at least someone on staff whom is assigned and can carry out the role, then that role would be a candidate for either recruiting or outsourcing.

An example list of roles

Below is an example list of roles (if you are reading this electronically, clicking an item will bring you to the description on our website) that the UCF is working with. Notice down in the list that yes, we even have a role for the End User.

• Define and Manage Business Value [UCF_Role_ID 0000002]

• Project Management [UCF_Role_ID 0000005]

• Product/Solution Architecture Development [UCF_Role_ID 0000006]

• IT Infrastructure Management [UCF_Role_ID 0000022]

• Provide IT General Support [UCF_Role_ID 0000023]

• Provide IT Operational Metrics and Reporting Support [UCF_Role_ID 0000024]

• Analyze and Determine Systems Categories [UCF_Role_ID 0000028]

• Define Security Requirements and Implement Security Solutions [UCF_Role_ID 0000029]

• Analyze and Design Databases [UCF_Role_ID 0000030]

• Provide Strategic Business Direction for Technology [UCF_Role_ID 0000032]

• Provide IT Infrastructure Planning [UCF_Role_ID 0000033]

• Manage ongoing Configuration and Implementation [UCF_Role_ID 0000035]

• Provide Strategic Direction for Systems Configuration and Interoperability [UCF_Role_ID 0000036]

• Manage Solution Definition Process and User Expectations [UCF_Role_ID 0000037]

• Provide High-Level Technology Logistics [UCF_Role_ID 0000038]

• Manage the Overall IT Compliance Framework [UCF_Role_ID 0000039]

• Administer and Maintain Databases [UCF_Role_ID 0000043]

• Perform Testing [UCF_Role_ID 0000045]

• Perform Monitoring and Management [UCF_Role_ID 0000046]

• Develop and Manage Business Partner Relationships [UCF_Role_ID 0000048]

• Provide Strategic Direction for Partner Relationship Program [UCF_Role_ID 0000050]

• Develop, Manage, and Negotiate Contracts [UCF_Role_ID 0000051]

• Analyze and Evaluate Performance of Partners and Effectiveness of Relationships [UCF_Role_ID 0000052]

• Perform Development-related Security Administration [UCF_Role_ID 0000057]

• Define and Manage Project Scope [UCF_Role_ID 0000059]

• Define and Manage Project Plan and Timeline [UCF_Role_ID 0000060]

• Manage Project Quality Process [UCF_Role_ID 0000062]

• Manage Project Human Resources [UCF_Role_ID 0000063]

• Analyze and Manage Risks [UCF_Role_ID 0000065]

• Manage Procurement Project Processes [UCF_Role_ID 0000066]

• Manage Customer Expectations and Customer Interaction Processes [UCF_Role_ID 0000067]

• Perform Hardware and Software Installation, Configuration, Upgrades, And Updates [UCF_Role_ID 0000075]

• Perform System Operations, Monitoring, and Maintenance [UCF_Role_ID 0000076]

• Perform Security Administration [UCF_Role_ID 0000077]

• Create and Manage User Accounts [UCF_Role_ID 0000079]

• Develop Product and Service Solution with Customers and Users [UCF_Role_ID 0000080]

• Develop Test Strategy and Plan [UCF_Role_ID 0000085]

• Implement Test Plans [UCF_Role_ID 0000087]

• Perform Usability Testing [UCF_Role_ID 0000090]

• Analyze Effectiveness of Test Plan, Strategy, and Processes [UCF_Role_ID 0000091]

• Analyze Training Needs Through Skills Evaluation [UCF_Role_ID 0000092]

• Develop IT Training Solutions [UCF_Role_ID 0000093]

• Deliver IT Training [UCF_Role_ID 0000094]

• Analyze Training Effectiveness [UCF_Role_ID 0000095]

• Systems Continuity, Backup, and Recovery [UCF_Role_ID 0000097]

• Manage Human Resources [UCF_Role_ID 0000098]

• Perform Audit Log Management [UCF_Role_ID 0000099]

• Perform and Manage the Audit Process [UCF_Role_ID 0000100]

• Encryption Management [UCF_Role_ID 0000101]

• Compliance Documentation Creation Editing and Management [UCF_Role_ID 0000102]

• Data Custodian [UCF_Role_ID 0000103]

• Data Trustee [UCF_Role_ID 0000104]

• Incident Response [UCF_Role_ID 0000105]

• Perform Inventory Management [UCF_Role_ID 0000106]

• Create and Maintain Network and Infrastructure Documentation [UCF_Role_ID 0000107]

• End User [UCF_Role_ID 0000108]

• Information and Technology Governance [UCF_Role_ID 0000109]

• Analyze and Design Business Applications [UCF_Role_ID 0000110]

• Define Compliance Requirements and Implement Compliance Solutions [UCF_Role_ID 0000111]

• Perform System Management [UCF_Role_ID 0000112]

• Compliance Oversight [UCF_Role_ID 0000113]

• Records Management [UCF_Role_ID 0000114]

• Storage Management [UCF_Role_ID 0000115]

• Records Management Oversight [UCF_Role_ID 0000116]

To control is an activity conducted to bring into check (to manage or to verify), or to constrain (to restrict or confine) something, the results of which bring forth a demonstrable outcome.

In this light, when we are employing the controls within our organization, we are creating a demonstrable outcome. Let's say that the organization must follow this control:

Establish assurance levels for information types The organization will ensure that its assurance classification guidelines take account of business needs for sharing of restricted information and the business impacts associated with such needs (e.g., unauthorized access or damage to the information). [CCI 00602]

This calls for the organization to apply its assurance strategy for information classification (and therefore information sharing) on all key IT assets. How would you go about reporting that you've accomplished this? One way would be to divide the number of key IT assets for which an assurance strategy has been implemented (you can determine this by checking their configuration policies) by the total number of IT assets as found within your configuration management database.

In formal terms, metrics present a system of measurement and analysis. What this system must define is:

1. what should be measured,

2. the unit of measurement,

3. the target for success, and

4. which sources must be drawn from for the measurement to be accurate.

Formally stated, the metric for the control in our description would read like this:

• • Title Key assets for which an assurance strategy has been implemented

  • Formula # of key assets for which an assurance strategy has been implemented / # of IT assets in total

  • Target 100%

  • Data sources The number of assets that have assigned configuration standards and other controlling policies and procedures / the total number of assets in the organization's configuration management database

In order to be useful, these metrics must indicate the degree to which goals are being met. In other words, they need to drive actions that will improve the organization's compliance processes. As one of our field editors stated, "metrics are only meaningful if they answer a question that someone wants answered." The same field editor pointed out that, "as with most if not all management decisions there are trade offs that need to be made. What these metrics need to speak to is risk -- again where are we relative to where we need to be? For example, if a given investment will reduce our exposure then what are the metrics that will improve as a result? Are we seeing such results?" Indeed, that is the type of input that metrics should be providing.

So now you're probably thinking, okay, I get it. Should I be making up a bunch of these metrics for each of my controls? The answer is, only if you are really, really bored and have nothing else to do. Because there are over 125 metrics that have already been defined from which you can choose.

As a matter of record, there are four well documented sources (and one revision) dealing specifically with information assurance measurement and metrics. The first was developed as early as 2003 and the latest in 2008.

• • July 2003 - Security Metrics Guide for Information Technology Systems, NIST SP 800-55

  • November 2004 - Corporate Information Security Working Group: Report of the best practices and metrics teams; subcommittee on technology, information policy, intergovernmental relations and the census; Government Reform Committee, United States House of Representatives (aka CISWG I)

  • March 2005 - IIA Global Technology Audit Guide (GTAG): Information Technology Controls

  • May 2006 - Guide for Developing Performance Metrics for Information Security, NIST SP 800-80

  • July 2008 - Performance Measurement Guide for Information Security, NIST SP 800-55 Revision 1

In addition to these resources, the Center for Internet Security, SecurityMetrics.org, and other such groups are continuously developing and documenting new metrics to meet the information assurance reporting challenges we all face.

The three audiences for metrics

More specifically, the four cornerstone metrics documents call out three groupings of metrics and their audiences:

• • Governance implementation metrics measure the implementation of assurance policies & procedures in connection with its information assurance responsibilities. These metrics should be reported at the Board and CXO level for use in connection with its information assurance responsibilities.

  • Effectiveness & efficiency metrics measure the results of individual policies and procedures. These metrics should be reported to the CIO, CISO, and IT Director level to determine the effectiveness of their processes and procedures.

  • Technical measurement metrics are reports such as those created by the backup and recovery tools, various logs, incident management tickets, etc. These IT operational metrics should be reviewed by IT managers, incident managers, HR managers, and others to ensure that their various control systems are in place and working effectively.

How to judge a metric

Each of the four documents above seems to greatly overlap the other in terms of both scope and content, with four points being reiterated in each of the documents:

1. Is the metric useful to (and desired by) the stakeholders of the organization?

Metrics should identify gaps in performance and expectations. The size and nature of the gap, as well as the importance of the activity being measured, will define the need to close the gap.

2. Is the metric based on defined and repeatable policies and procedures?

For the metric to be real, it has to be based upon repeatable measurements. A solitary report shows no trend. Also, in establishing a metrics program, the organization must establish the program around its current policies and procedures. Why? Because policies and procedures should be establishing what the organization does, and the metrics should be used to document how well those policies and procedures are working.

3. Can the data supporting the metric be readily and cost-effectively obtainable?

This is the big one - obtaining readily available and cost effective measurements. What you can gather, and the cost effectiveness of gathering those measurements, will be determined by your level of compliance maturity. Think for a moment in the terms of a Capability Maturity Model (CMM), that measures awareness, policies and procedures, skills and expertise, responsibility and accountability, and tools and automation, with level 1 being no formal compliance processes and level 5 being your organization is completely buttoned up and continuously improving. At level 1 and 2 of those areas, your metrics won't be trusted because it would take a Herculean effort to even collect the data, which is probably non-existent. It won't be until the organization is into level 3 and 4 that you are getting anything more substantial than simple percentages or binary metrics.

4. Does the metric yield quantifiable (percentage, average, etc.) information?

This is the million dollar question. Most "how-to" metrics guides that we have read come up with all sorts of wild numbers, costs, and items to measure. How accurate are they? Mostly, not. The good news, though, is that the Unified Compliance Framework has mapped, and will continue to map, all of the metrics guidelines published by the various authority documents. There are over 150 measurements that should be taken, and each of those measurements has been pre-defined along with calculations for how to systematize and report those measurements.

Presenting metrics

With the exception of Andrew Jacquith's book on security metrics (which is outstanding), if you look online at the various offerings of metrics books, metrics chart packs, and metrics how-tos, you'll find that each of them promises well over 500 different metrics from which to choose, and almost as many chart types to present the information. One book we found on Amazon promised over 900 different metrics. A chart series we found had around 60 different metrics reports, and over 50 different ways of presenting those metrics. Is more really better? In a word, no.

After examining all of the metrics called for within the hundreds and hundreds of IT regulatory guidelines being mapped by the UCF, we found that there are over 125 different metrics described by the various authority documents. That's it. Not 500, and especially not 900. And of those different metrics described by the various authority documents, we found that all of them can be presented in one of three different chart types:

1. A simple pie chart

2. A waterfall chart

3. A stacked column chart

So why complicate things if you don't have to? Each of the three chart types above fits the three basic types of comparisons found within the world of IT metrics: component comparison, component build, and time series categorized comparison. Each of the three comparisons presents a different message to the audience and each should be presented in a different graphic manner to give the audience clues for recognizing that message.

Component comparisons: a simple percentage of the total

Component comparisons show the size of each part as a percentage of the whole. Any time the word percentage, portion, or share is used, you are more than likely dealing with a component comparison. A component comparison should be presented using a simple pie chart.

Component comparison

Almost 99% of the predefined compliance metrics are component comparisons as they read something like this sample "Report on the current security plans that have been reviewed and adjusted to reflect the current conditions and risks."

Component build: an itemized breakdown of the total

When dissecting the parts of a whole in order to show the itemized breakdowns of the parts to the whole, a pie chart isn't as clear as one would hope. That's when you should turn to a waterfall chart. A waterfall is a simple plus and minus system. It adds and/or subtracts to either build or take away from a whole. Waterfalls take the place of columns of numbers that are summed. They visually display the values of the numbers being summed and, therefore, communicate a stronger message.

Component build

The base metric "Report on the estimated damage or loss in dollars resulting from all security incidents" is best displayed as a waterfall instead of a pie because of the number of elements being dissected.

Time series categorized comparison: changes over time

When comparing how categorized data has changed over time (whether the changes show a trend or not), the pie chart is not sufficient. However, a stacked column (or bar) can show the segments of a whole like a pie would. Use stacked bar/column charts that have the segments represented as percentages to take the place of multiple pie charts on a page. A single stacked bar/column has the same information as a single pie chart; however, many bars/columns can reside on a baseline and, therefore, visually display comparisons between the segments much better than multiple pies on a page.

Time series

While the base metric for "Report on the number of security incidents that took place that did not cause a loss of confidentiality, integrity, or availability beyond SLAs for thresholds" is a component comparison, comparing the last several quarters is a perfect example for the use of a time series comparison.

Always remember that you are presenting the message found in the data

Remember. you are presenting the message found within the data; that the servers are protected, or incidents are down this quarter, or unexpected changes have been brought down to nothing. You aren't presenting metrics to show off your PowerPoint prowess. You are there for a business reason, one that should be communicated clearly and precisely. You can do that with just three chart types. If you keep it simple and keep it clear, the message will carry the day.

When management wants glitz

So, what do you do when management wants glitz and glamour in your metrics presentations? What we in the Unified Compliance Framework do is follow Nancy Reagan's advice and "just say no." But we also know that we aren't the final word. So we are running an online poll to find out what you would do. As of this writing, here are the stats (you can take the poll yourself by clicking HERE):

• • 17% of you allow the glitz and glamour to override the message.

  • 38% of you tell management that you are not going for glamour and that you are focusing on content instead.

  • 13% of you sidestep the issue and ask management to redesign the presentations to fit (knowing that will never happen).

  • 4% of you simply file any and all requests for glitz and glamour in the trash.

  • 29% of you follow another route.

Here's what some of our field editors have had to say about this:

I have spent quite a bit of time in this area (4 years) and have studied the importance of presentation and the significance of no glitz because it is simply noise and prohibits the user from quickly understanding the data so that decisions can be made. How I've worked around this issue in the past is to work with the user to identify the real requirements, make suggestions for a better display of the data, all the while explaining the science and goal of displaying data. I've been 100% successful with this method. -- Signe Jackson

We would simply explain why management shouldn't want glitz and glamour in metrics presentations. But anyway, they do not want that and never will. -- Markku Povari

On the other hand, another distinguished field editor, Hugh Burley, had this comment on why every so often glitz does do the job:

"When management wants glitz, it is usually because they need to present content in a way that is compelling, in order to maximize the time they have in front of the executive or some other group. The challenge is to add the glitz while including the core message(s).

As an example, at a meeting of the executive level Information Security Committee, we needed to get action on the highest priority items in an orderly manner. This committee consists of Legal Council, the AVP Finance, the CIO and AVP of ITS, the Director of Facilities, a faculty member, and a student. They meet once a month for one hour. Several meetings had taken place with little progress, although large amounts of detailed information had been presented and discussed. The committee was in danger of losing momentum. The amount of information seemed overwhelming and the group appeared to be stuck in an endless cycle of discussion and reiteration of previous detail.

In order to get this process moving, a single, somewhat glitzy slide that crammed the information into a visual image capturing all the pertinent data was presented. There was some joking about the use of color and translucence to convey data, but the decision to move ahead with the highest priority item was taken, the committee was re-energized, and moved on to address the most critical issue. Score one for glitz."

And there you have it.