by Jane O. Smith (March 2004)
(Reprinted by permission of BFMA International, Inc.)
In early 2001, I joined a well-respected firm based in Germantown, Maryland to help promote its software research and development efforts in complex data management. For more than ten years, the majority of Fenestra Technologies' efforts were dedicated to the United States Census Bureau, and preparing to launch the Generalized Instrument Design System, an innovative system designed and built for the U.S. 2002 Economic Census.
Since then, much has happened throughout the federal government in the area of metadata-driven forms and surveys. Just last year, Fenestra's CEO, Rick Rogers, was asked to lead the E-Forms for E-Gov pilot project on behalf of the Federal CIO Council. The value of this multi-agency, multi-vendor pilot is far-reaching, as the solutions architecture for the Small Business Administration's new Business Gateway relies on the pilot's list of components.
It continues to be an exciting time in this industry! The following features excerpts of papers and web sites that offer a glimpse of the evolution of the forms industry in the government-to-business sector.
Every five years, the Bureau of Census conducts a wide-ranging census of economic activity involving 650+ individual survey forms (each is approximately 10 pages long) that are sent to millions of United States businesses. Questions change every cycle, to reflect the country's dynamic economy.
The creation and implementation of the Generalized Instrument Design System (GIDS) for the 2002 Economic Census, enables new IT activities at the U.S. Census Bureau, such as:
The Bureau's performance metrics improve through:
The GIDS framework is sufficiently general that making modifications for new applications is a relatively painless task, such as electronically formatting the publication of 2000 Decennial Census data.
Improving the flexibility and extensibility of GIDS is a long-term goal. It is hoped that the majority of formatting problems may be solved with a single, general-purpose descendant, using only schemas and scripts to drive the process.
Making metadata available for cross-platform dissemination, analysis and publishing in paper and electronic formats, were key drivers behind customizing software for the 2002 Economic Census.
The infrastructure of the Economic Census consists of a central repository, containing both metadata, which define the data model and the characteristics of the forms distributed to respondents, and response data. Information is centrally located for ease of access, validation and consistency checking.
In addition to conventional paper forms, many respondents are given the option to complete surveys electronically. Responses from electronic surveys are automatically validated and entered directly into the repository.
Survey metadata, which define how a form is to appear to the user, are segmented into content, layout and behavioral components. Content metadata control the actual wording of questions, instructions, etc. Layout metadata control the placement and typographical characteristics of text and images. Behavioral metadata (for electronic forms only) control the behavior of the form when buttons are clicked, inconsistent responses are entered, etc.
Four major software applications work with the metadata in the central repository - a Forms Designer, Autoformatter, Previewer and Surveyor. These applications are integrated with the repository through eXtensible Markup Language (XML; see http://www.w3.org/XML/) as the basis for information interchange.
The "E-Forms for E-Gov" pilot team was chartered by the XML Web Services Working Group of the Federal CIO Council to provide quality information on E-Forms issues in support of the E-Gov (electronic government) initiatives in 2002. The primary goal of the pilot was to determine best practices and identify business metrics for changing to an e-forms process to support the Government Paperwork Elimination Act (GPEA) - the President's E-Gov mandate, and also the Performance Reference Model (PRM) as required by OMB's Federal Enterprise Architecture (FEA). This was a multi-agency and multi-vendor pilot, which culminated in a series of recommendations that were published late last year.
The following are some of the technical issues presented in the final paper of this pilot project. (See http://www.fenestra.com/eforms for the complete report.)
An electronic form (e-form) is a self-contained collection of declarative meta-data that defines how a form is to be "realized" via generalized computer software - in other words, an "e-form" defines how a form should look, behave, and operate on a computer. An e-form allows the capture of a response dataset, which represents the data provided by a respondent via keyboard entry, importing, or other means.
Many software applications capture responses - for example, any application that includes a user-interface that allows input of data values from the keyboard supports response dataset capture. E-Forms are distinct from standard custom-coded data-entry applications in the respect that e-forms are self-contained declarative meta-data, and applications that process e-forms are generic and meta-data driven.
This generalized meta-data driven aspect of e-forms offers powesrful technical advantages:
Reduced costs. Generic e-forms software applications can be used to realize and process any number of e-forms, thus reducing the number of applications to be procured/built, maintained, deployed, and supported. Also, collecting data electronically is demonstrably cheaper than collecting it via paper.
Component-based architecture. Self-contained declarative meta-data facilitates a component-based architecture, because different components can be plugged into parts of the system to operate on the standardized and generalized e-forms meta-data.
Reuse and Harmonization. Constructing electronic forms from meta-data promotes the reuse of meta-data across forms, which in turn reduces forms authoring costs, and facilitates harmonization across forms.
Scalability. Adding a new e-form to an existing e-forms system simply entails defining the e-form meta-data - it does not entail creating any new software. With appropriately user-friendly e-forms authoring tools, non-programmers can easily add new e-forms to the system.
Consistency. Because e-forms capture the way a form should look, behave, and operate in declarative meta-data, there is a large degree of inherent consistency across forms. This consistency benefits the respondents, and also the back-end processing systems that use the response datasets.
Timeliness and Accuracy. Capturing data from e-forms is nearly always demonstrably faster than capturing data from paper, and the built-in validations in e-forms ensure a higher degree of accuracy.
E-Forms incur a few disadvantages, which must be recognized:
Restrictiveness. This is the flip side of consistency. A custom-coded application can do literally anything that the host programming language allows - this provides maximum flexibility. The declarative meta-data driven approach of e-forms means that the capabilities and features of the form are constrained by the meta-data standard used to define the form, and the extent to which generic software components correctly and completely implement that standard.
Potential for Vendor Lock-in and Dependency. There are many e-forms meta-data definitions, a few of which are fully based on Voluntary Community Standards (as defined by OMB Circular A-119), and many of which are not. In addition, the standards in this domain are currently in flux. Systems that are not standards-based are frequently only supported by one specific vendor, and even some standards-based systems have support from only one vendor.
This means that in the current landscape, using e-forms involves some degree of vendor lock-in, because the vendor must provide and maintain the generic components to operate on their "flavor" of e-form meta-data. Hopefully this will change over time as the many e-forms standardization efforts (such as XForms) become more widely adopted and supported.
The current e-forms landscape faces some challenges that are worth noting:
Incomplete Integration with Web Browsers. The most ubiquitous, generic, and popular user-interface is the generic HTML web browser. However, web browsers do not natively support many of the advanced features and capabilities present in e-forms meta-data standards. For example, the presentation language of the web, HTML, is quite limited in its typographical capabilities, particularly for domains where precise typography is required (such as legally binding documents).
This has led to a situation where nearly all of the major e-forms vendors implement their e-forms user-interfaces in applications that are separate from the web browsers, and which are deployed as stand-alone client applications, or browser plug-ins, or both. These applications are known as form-fill applications, or as e-forms user agents.
This situation forces a difficult choice between targeting the ubiquitous but limited web, versus adopting a richer e-forms meta-data system that requires a separate application to be deployed. Because of the high cost of deploying custom applications, the current situation has slowed adoption of e-form systems.
Lack of Standard Component Stack. There are many different standards that cover different segments of the e-forms problem domain (e.g., XML, XML Schema, XML Web Services, WebDav, ISO 11179, UBL, SVG, HTML, etc.).
There are also many types of generic e-forms services that could leverage these standards and be exposed as standardized components (e.g., forms catalog, schema registry, forms authoring component, forms user-interface agent, response dataset submission server, response dataset repository, etc.).
However, there is no standards-based component stack that pulls together all the individual standards to define how the components should work together. While many e-forms product vendors support selected standards, the lack of a standards-based component stack means that different e-forms solutions are only marginally interoperable with each other. This situation further increases vendor lock-in and dependency.
Security and Privacy. The most secure/private e-forms solutions (for example, Public Key Infrastructures) also involve significant expense and logistical tradeoffs. The lack of an inexpensive and easy-to-manage security infrastructure hampers adoption of e-forms that capture sensitive information.
U.S. businesses, particularly small businesses, must comply with numerous federal, state, and local laws and regulations. According to the Small Business Administration (SBA), the average "highly regulated" business (e.g., restaurants, gas stations, dry cleaners) needs to apply for, and receive, an average of 10 to 15 licenses and permits. It can be difficult, time-consuming, and ultimately very costly for businesses to determine which laws and regulations apply and learn how to comply.
Each year, complying with laws and regulations cost American businesses nearly $500 billion and consumes over eight billion hours annually of otherwise productive time. The volume of existing laws (i.e., over 140,000 pages in the Code of Federal Regulations alone), the expense of finding nearly inaccessible information, multi-jurisdictional systems, and the lack of smart on-line tools contribute to this overwhelming cost.
The Business Gateway will save American businesses time and money by making it easier to find, understand and comply with applicable laws and regulations. It will also enable Federal regulatory agencies to comply with the Government Paperwork Elimination Act (GPEA) and improve their administrative efficiencies.
The Business Gateway will be the cross agency portal for businesses that integrates the content and functionality of number of federal business web sites (e.g., business.gov, businesslaw.gov, sba.gov). Working with businesses, the initiative will design an "XML solution" that significantly reduces the regulatory paperwork burden and changes the way businesses interact with government. (See http://www.whitehouse.gov/omb/egov/gtob/compliance.htm.)
The Gateway will offer:
By providing quick access to forms, legal information, regulatory compliance assistance tools, and on-line transactions in one well-organized, user-friendly portal, the Business Gateway can reduce the burden hours by 10 percent saving American businesses an estimated $2.4 billion annually and help federal agencies achieve a 75 percent GPEA compliance rate.
The solutions architecture for the Small Business Administration's Business Gateway relies on the E-Forms for E-Gov Pilot' list of components which include:
E-Forms Authoring Component: Allows a non-programmer to define the meta-data that comprises an e-form, including presentation, content, structure, and behavior. Includes a user-interface that allows people to design e-forms.
E-Forms Generator Component: Builds e-forms automatically from existing meta-data. In some scenarios e-forms do not need to be manually designed with the authoring component - instead, they can be generated automatically from existing meta-data.
E-Forms User Agent Component: Provides a user-interface that allows a respondent to create the response dataset for the e-form. Also known as a "form fill" component. It supports filling forms, signing forms, and submitting forms.
E-Forms Web Broker Component: Translates the meta-data contained in an e-forms document into standard HTML and serves it up via a standard web server to the standard web browsers. This serves as an alternative to an E-Forms User Agent, because it allows the standard web browser to serve as the user-interface.
E-Forms Catalog Component: Serves as a registry for published e-forms, and includes respondent profile information so that respondents can find forms that apply to them.
E-Forms Schema Registry Component: Serves as a registry for the response dataset schemas used by e-forms. Facilitates schema reuse and harmonization, and facilitates direct information exchange (since exchange uses the same schemas).
E-Forms Submission Server Component: Serves as a collection point for submitted e-forms, and routes (and possibly reformats) the submissions to appropriate back-end processing systems. Maintains submission status and history.
E-Form Authentication Component: Authenticates e-forms respondents and their signatures.
E-Forms Workflow Component: Performs standard workflow management for e-forms documents, including routing, approval, etc.
Next steps for the Business Gateway include: