What to Read Next
Many managers believe that quality information is key to their success. Do they act on that belief? Not convincingly. Most senior executives have experienced the costs of decisions based on poor information. Most general managers have dealt with the frustration of knowing that they have data within the firm but they cannot access it in the integrated form needed.1 Most chief information officers have faced the discomfort of explaining why, in light of the company’s costly investments in IT, the data are of poor quality or inaccessible. Firms recognize the need for quality information and many strive to satisfy it. All too often, however, the results are disappointing.
During the past decade, we have investigated the information quality problems that organizations encountered. What clearly stands out from our research is the need for companies to treat information as a product. Often, however, companies treat information as a by-product; they focus on the systems or the events that produce the information instead of the information content. To treat information as a product, a company must follow four principles:
- Understand consumers’ information needs.
- Manage information as the product of a well-defined production process.
- Manage information as a product with a life cycle.
- Appoint an information product manager (IPM) to manage the information processes and resulting product.
We call the application of these principles the information product (IP) approach. It is the keystone on which the delivery of high-quality information depends. In this article, we argue for adoption of the IP approach and provide a framework for its implementation. We use four cases, drawn from our field research experiences, to illustrate the IP approach’s principles and the negative consequences that result without them.
Introducing the Cases
Financial Company is a leading investment bank with extensive domestic and international operations. Its customers needed to trade immediately after opening a new account. The new account had to be linked to other accounts that the customer may have opened, and the information in all accounts had to be accurate, up to date, and consistent. The bank required real-time account balance information to enforce minimum account balance rules across a customer’s multiple accounts. Failure to obtain that information exposed the bank to potentially large monetary losses. By statute, the bank had to close all of a customer’s accounts when informed by federal authorities of criminal activities by that customer. Adhering to the statute required timely, integrated information.
Eyewear Company sells eyewear products at its nationwide retail outlets. The retail outlets generated specifications for the eyeglass products and forwarded them to lens grinders at one of four laboratories. The laboratories received 25,000 eyeglass orders each week. The ability to produce eyeglasses that met consumer needs depended on the quality of information provided to the grinders at the laboratories. If the grinder’s information needs were not met, the production of lenses was negatively affected.
Chemical Company, a Fortune 500 company, is a major presence in the petrochemical industry. For each chemical product, Chemical Company was legally required to produce a material safety data sheet (MSDS) that identified the product’s potential hazards, the symptoms of the hazard, and the actions to take should a symptom be observed. Because of the high product liability costs of not reporting potential hazards, the company had every incentive to provide accurate, complete, timely, and understandable MSDSs. Chemical Company had a well-defined process for creating MSDSs. When a new chemical was developed, the MSDS group contacted experts who contributed to the specification.
Data Company captures data from hundreds of millions of transactions a week generated by tens of thousands of retail stores. Clients purchased information that was refined from the raw data collected from these retail stores. Data Company had built sufficient intelligence into its systems to provide high-quality information. It had been using neural networks for several years and had built an imputation process that estimated any missing data. For example, the system enabled the company to correct data that were corrupted by failed or bad transmissions.
Overlooking the Four Principles
The experiences of these companies suggest the negative consequences that companies face if they overlook the four principles of the IP approach.
Failure to Understand Consumers’ Needs
Financial Company and Eyewear Company had to determine the needs of two types of consumer: the external customer and the internal information consumer. Each company provides an example of what can happen when a business fails to understand those needs.
For its external customers, Financial Company must open new accounts quickly, maintain up-to-date customer risk profiles, and know all the accounts related to a particular customer. Inappropriately risky investments would cause high customer dissatisfaction and potential indemnification to customers for losses. For its internal consumers, Financial Company must provide real-time information about any changes in customers’ account balances. Not doing so causes different internal customers to use locally maintained databases that often are inconsistent.
Satisfying the needs of Eyewear’s external customers translated into providing the proper lenses, which depended on the retail outlet sending the correct lens specifications to the laboratory. When the opticians, the writers of the specifications, misunderstood the information needs of the lens grinders (the internal customer), the result was that many lenses needed to be reworked. Regrinding lenses led to additional costs and to delays that lowered external customer satisfaction.
Poorly-Defined Information Production Process
Financial Company maintained a centralized customer account database. Every night it posted transactions that occurred throughout the day to the database and updated customer account balances. Other customer information, such as the customer’s risk profile, however, was updated on an ad hoc basis when convenient. This ad hoc approach resulted from treating information solely as the by-product of a physical event rather than as the product of a well-defined production process. A change would be triggered by, say, a customer request. A well-defined production process, in this context, would require systematic examination of customer behavior and the attendant update of the risk profile.
Financial Company’s internal customers viewed customer account information as unreliable. As one vice president joked, “Everyone but the consultant has update privilege to customer account information.” Hoping to solve these problems, individual departments developed home-grown customer account databases. The result was a proliferation of local customer databases with inconsistencies among them. These local databases were tailored to the needs of each department and contained more current information on the department’s customers than did the central database. Each department collected and processed information as a by-product of its local operations, independent of the bank’s need for integrated customer account information.
No Product Life Cycle
In adapting the classical marketing notion, we define the information product life cycle as the stages through which information passes from introduction to obsolescence.2 The life cycle can be divided into four stages: introduction (creation), growth, maturity, and decline. Chemical Company is an example of an organization that did not follow this principle. Its well-defined process for creating MSDSs did not extend to maintaining the quality of the information over the life cycle of the product. As a result, information on new hazards, based on accumulated experiences with the product and emerging scientific evidence, was incorporated into a product’s MSDS erratically. Over time, information quality deteriorated.
At Financial Company, changes in the operating environment called for updated production processes to improve the bank’s information products. Financial Company, however, did not manage the life cycle of its customer account information to accommodate the new global, legal, and competitive environment, resulting in potentially huge problems for the bank. It was not poised to leverage customer account information in its global operations. For example, customers with sufficient credit across accounts could not trade or borrow on their full balance. Tracking customer balances for individual and multiple accounts, closing all the accounts of a customer because of criminal activities, and ensuring an accurate customer risk profile could not be accomplished without significant, error-prone human intervention.
Lack of an IPM
At Financial Company, an IPM would have been responsible for monitoring and capturing consumer information needs continually, reconciling those varied needs, and transforming the knowledge gained into a process of continuous improvement. Without an IPM, Financial Company established few information process measures or controls. For example, the bank had no controls to ensure that customer risk profiles were regularly updated.3 It neither standardized nor inspected the account creation process. As a result, it established no metrics to determine how many accounts were created on time and whether customer information in those accounts was updated. Because management gave its attention to revenue-generating operations such as trading, the IT department found itself responding reactively to ad hoc requests from the trading department for updated customer account information. Had Financial Company appointed an IPM, it would have enjoyed better risk management and customer service — two factors critical for success in the financial industry.
Product or By-Product: Five Factors
Managers can always view and manage information as the by-product of a system or an event. From the perspective of the people using the information, however, it is a product, not a by-product. In contrasting the by-product approach to the information-as-product approach, there are five factors to analyze: what is managed, how it is managed, why it is managed, what is success, and who manages it (see Table 1).
What Is Managed?
Organizations often focus inappropriately on managing the life cycle of the hardware and software systems that produce the information instead of on the information itself. As a result, they fail to capture the additional knowledge necessary to achieve meaningful information quality. For example, at Eyewear Company, the grinding instructions were distinct from, and just as important as, the actual lenses. If the instructions were incorrect, the lenses would be incorrect. In the course of selling the eyeglasses to the customer, the focus was on the system that captures and transmits the information and not on the form in which the information was delivered to the grinder.
How Is Information Managed?
In the by-product approach, because the organization’s focus is on the life cycle of the hardware and software systems, the means of producing information become ends in themselves. Managers focus on individual components of the systems and attempt to establish cost controls on those components. They view these components in isolation from one another instead of treating them as an integrated whole.
The situation at Eyewear Company demonstrates what can happen when an organization focuses purely on components. When asked for lens rework figures, the IT director stated, “We know we have 15 percent errors.” Left unstated was the assumption that everyone attributed the errors to the grinding machines. All the attention was on the hardware and software components of lens production, not on the lens specifications. It took outside observers to recognize that communication problems between opticians and grinders led to the high error rate. Many problems resulted from mismatches between how opticians wrote orders and how grinders read those orders. For example, opticians used special instruction lines on the form to add information. The grinders ignored that information because they did not expect to find grinding specifications there. IT also contributed to the problem. The MIS director gave his attention to upgrades of hardware and software and did not pay sufficient attention to how each function interpreted the information being transmitted. No one held a cross-functional perspective.
Failing to treat lens specifications as an information product with the grinder as consumer resulted in the regrinding of approximately 40,000 lenses per year. The problem was costing the company more than $1 million annually in reworking expenses, not to mention other costs such as those associated with customer dissatisfaction. In focusing on the components of lens production, Eyewear Company was not treating the opticians and grinders, in their roles as information suppliers and consumers, as part of an integrated system.
Financial Company provides another example of a misplaced emphasis on individual components. As noted earlier, databases were optimized at the local level and were not truly integrated. The account managers, who were the internal information consumers, had local information for local needs. But these managers did not have access to the integrated, global information required to exploit emerging opportunities. The partial information that account managers received hindered their ability to manage risk, improve service, and increase revenue.
Why Manage Information?
Too often, IT departments emphasize improving the quality of the delivery system and its components rather than optimizing the quality of the information product delivered to the consumer.4 The latter approach requires a thorough knowledge of the consumer’s information needs and quality criteria.
Eyewear Company illustrates the misplaced emphasis on component improvement. Opticians putting information in the special instruction section of the order forms was an example of “improving” the order-writing component as opposed to improving the quality of information supplied to the grinder. It made the opticians’ task easier at the consumers’ (grinders’) expense.
In contrast, Data Company was rapidly evolving toward managing information as product. It was beginning to manage the entire information delivery process as an integrated system. It adopted a companywide total quality management program and invested in modernizing its technology infrastructure.5 The company worked with its information suppliers to produce better information. It instituted procedures that enabled information consumers to report information quality problems directly. Because the company tailored its information products to individual clients, it recognized the need to be a proactive partner with its clients.
Even with all this attention to an integrated production process, communication problems remained. For example, Data Company had problems pricing its product because fixed costs were not communicated effectively within the company. The cost data were accurate but were not represented in a way that the marketing department could use. For example, data such as management overhead were not appropriately communicated to the sales force. As a result, Data Company’s sales force did not trust the IT department’s cost data and continued to ignore the management overhead in its contract bidding pricing. The situation prompted the marketing executive vice president to state, “I may not price strictly based on cost, but I need to know the cost.” The company’s profit margins remained low despite its predominant position in a competitive market. One reason that a company manages information as product is to avoid the above problems by ensuring that everyone understands the purpose behind the need for information.
What Is Success?
Employing an information-as-product approach instead of a by-product approach changes the measures of success. Rather than judge the success of the computer system based on having no bugs, companies that take an IP approach measure the information’s fitness for consumer use. Focusing on the computer system reflects a short-term perspective. The initial information produced is without error, but little attention is given to changes occurring later in the product life cycle.
Chemical Company’s experience with the quality of its MSDSs over time, mentioned earlier, provides an example of failing to recognize the need to deliver a quality information product over the product’s life cycle. Although the information quality of MSDSs was initially high, it deteriorated over time. The company was expected to update its MSDSs as new evidence of hazards arose through product use and new scientific discoveries. It often failed to do so. Chemical Company measured its success by the quality of its initial MSDS. The life cycle of the MSDS product, however, is not necessarily the same as the life cycle of the chemical product. A truer measure of success would account for the difference and assess the quality of the MSDS over its life cycle.
Who Manages Information?
If companies take an IP approach and manage across functions, they must adopt a management structure to this environment by appointing an information product manager. The positions of IPM and CIO are not identical. The CIO oversees the management of the firm’s data repositories, among other responsibilities. The database administrator (DBA), a subordinate function reporting to the CIO, directly manages the repositories. In the four companies we studied, solutions to information quality problems required management intervention. Our research suggests that the person doing the intervening should not be the DBA, whose traditional focus has been to control what enters the databases. The DBA does not typically focus on the production and delivery of information as a cross-functional, integrated system involving suppliers, producers, and consumers.
What was striking about the companies we studied was their intuitive sense that they needed an IPM. Financial Company, Chemical Company, Data Company, and Eyewear Company each had an individual performing duties that we would associate with those of an IPM, although they did not use that designation. They have taken an information-as-product view and have begun to manage information with that in mind.
Acting on the IP Approach Principles
In examining the problems encountered by the four companies, the need for an integrated approach to managing information becomes evident. By adopting the IP approach, companies add value in diverse ways (see Table 2). There are better internal communications. Internal operations become more efficient. Company and customer objectives are aligned. Costs are reduced, revenues are increased, and new opportunities emerge. Each improvement, in turn, strengthens the firm’s competitive position in the market. Adopting an information-as-product approach forces the firm to focus on the information consumer, supplier, and manufacturer collectively. This focus leads to the discovery of new opportunities, the exploitation of those opportunities, and the deployment of resulting applications for enhanced profitability, competitive advantage, and market dominance.
We have argued for the need and benefits for managing information as a product. Understanding these needs and benefits, however, is insufficient for delivering quality information to consumers. Organizations must not only adopt the information product view but also promote that view throughout. They must use an integrated approach to managing information, treating the supplier, the manufacturer, and the consumer of information collectively as a system. This integrated approach demands that firms adhere to the four IP management principles.
1. Understand Needs
The information-as-product approach has one clear goal: to deliver quality information to the consumer. The information delivered is a total product; it exhibits all the attributes that, in combination, meet or exceed the consumer’s expectations.6 Consumers’ needs must be clearly established and understood during every phase of the information product’s development and manufacture.7 The goal is to ensure that the product is fit for consumer use because, ultimately, consumers judge a product’s quality on its fitness.8 For example, stock quotes delayed by fifteen minutes are accurate but of little value to real-time stock traders. Companies should apply the house-of-quality method to the task of delivering quality information products.9
Our research has shown that the context in which the consumer operates is an important determinant of information needs.10 We have identified sixteen information quality dimensions, grouped into four categories (see Table 3) that reflect the information consumer’s perceptions of what constitutes quality information.11 Managers should embed the evaluations of these dimensions in all the processes that produce information for consumers, including systems design and development and the daily production and delivery of information.12 They should use the quality attributes as vehicles for describing, measuring, and discussing problems.
At Financial Company, for example, the IP approach would require the bank to understand the needs of its information consumers in the different functional departments. With knowledge of its information consumers’ needs, the company can design and develop a new companywide customer account database. A new database alone, however, will solve neither their short-term nor long-term problems. Financial Company must institute standardized and controlled procedures for collecting and updating data, opening new accounts, and maintaining investor risk profiles. It also needs policies that state when and how to perform updates and who should authorize them.
2. Manage the Process
Treating information as a product requires treating the information production process as though it were producing a physical product for a customer. The process must be well defined and must contain adequate controls, including quality assurance, inspection, and production and delivery time management.13 In contrast, when information is treated as a byproduct, the production process exists but often lacks controls beyond the integrity constraints applied to information entering the database.
The information production process that management articulates must be validated against the actual operating process. Eyewear Company’s failure to understand how its process operated cost the company $1 million per year. The recognition that the opticians’ measurements served as information products for the grinders would have led the company to impose quality controls on the optician.
Chemical Company had a process to create each material safety data sheet. Although the company developed the process to create an information product, it did not manage it with the same controls that the company used for its chemicals. At Financial Company, the process of creating customer accounts was neither well defined nor controlled; the account managers who created customer accounts knew the goal — a new customer account should exist in the computer system — but they used whatever techniques were convenient to create an account. If a customer account existed, managers used that information from the account to open a new account without verifying whether the information was still valid. A well-defined and managed process would have included quality controls to ensure that new accounts had verifiable, up-to-date information.
3. Manage the Life Cycle
Information should be managed over its product life cycle in the same way as a successful physical product.14 With a physical product, modification and improvement at different stages of a product’s life cycle ensures that the product remains competitive in the market. The extent of the changes and the time interval between changes depend on the nature of the product, the need it satisfies, and the competitive environment in which it is sold. So it is with information. The degree and frequency of changes to information products depend on the type and nature of the information, the tasks the information supports, and the changing context in which the information is used.
Chemical Company provides an example of why an information product needs to be managed over its life cycle. As new hazards are discovered, the new scientific evidence must be incorporated into the MSDS. This iterative process of obtaining new evidence of hazards and updating the MSDS to reflect the new evidence is a classic example of the life cycle of an information product.
At Financial Company, customer account information was an information product. Over time, what constituted high-quality customer account information changed. Failure to continuously improve the production process and the product resulted in information that was no longer fit for use. Consumers either stopped using the product and developed their own information products or continued to use the inferior product, suffering the consequences of deterioration in the quality of work. Had Financial Company applied the ISO 9000 standards for quality management, these problems could have been mitigated.15
The World Wide Web sites of many companies provide a further example. A Web site is an information product, and site visitors are information consumers. Presumably, the Web site contains high-quality information that the company wishes to convey to its customers. Unfortunately, many companies do not treat the Web site as an information product. As the Wall Street Journal reported, businesses leave “cyber-space increasingly littered with digital debris — Web sites neglected or altogether abandoned by their creators.”16
4. Appoint an IPM
The designation of an information product manager is necessary to ensure that relevant, high-quality information products are delivered to consumers. The IPM’s key responsibility is to coordinate and manage the three major stakeholder groups: the supplier of raw information, the producer or manufacturer of the deliverable information, and the consumer of the information. To carry out this responsibility, the IPM must apply an integrated, cross-functional management approach. The IPM orchestrates and directs the information production process during the product’s life cycle to deliver quality information to the consumer.
De Facto IPMs.
As indicated previously, the four companies had an intuitive, if not clearly articulated, sense of how the information product should be managed.
Financial Company hired a new IT director, who was knowledgeable about information quality concepts, process engineering, and business applications, thus inaugurating the bank’s transition toward an IP perspective. Acting as a de facto IPM, the director began to institute a cross-functional approach. With support from the CEO, he constructed a work-flow model of the customer account IP process that integrated client services, business operations, and the IT department.
At Data Company, the IT director and a senior vice president who reported to the CEO shared the functions of the IPM. Together, they oversaw and managed the company’s information suppliers, the in-house information production, and the clients who were consumers of their information products. Chemical Company also had an IPM in the making. This manager was assigned to create the necessary cross-functional approach to ensure that the MSDS met high-quality standards throughout the life cycle of the corresponding chemical product, and, as such, throughout the life cycle of the MSDS.
The IPM must incorporate the requirements of the three stakeholders at each stage of the information product life cycle (see Table 4). Each stakeholder group, however, may have more than one constituent. Financial Company had many sources of raw information that were used in developing its final information products, hence, multiple constituents in the information supplier stakeholder group. Financial Company also serviced many information consumers. The same information, however, might be a different product to each consumer.
In eliciting the stakeholders’ requirements and integrating these into the information product, the IPM is responsible for different tasks over the course of the life cycle. During the introductory stage, the primary task is design and development, an engineering effort. During the growth stage, the emphasis is on improving and monitoring the information product. As the information product matures, the main task is maintenance, with an emphasis on monitoring. Attention to these major tasks over the life cycle is critical. For example, Chemical Company ensured proper development and production of its information product, the MSDS, during the introductory stage. Its legal liability problems arose because the company paid insufficient attention to the maintenance of the MSDSs after their initial production.
During design and development, the IPM must concentrate on accurately specifying the consumers’ needs and reconciling those needs with the capabilities of the information suppliers and information manufacturers. To assess the current quality of the information products delivered to consumers, the IPM must develop appropriate metrics.17 Further, there are information quality assessment tools that measure the consumers’ subjective assessment of information quality.18 Such assessments establish the baseline from which to make and measure future improvements.
As part of the monitoring process, the IPM must continually measure the product’s production and use to determine when to modify it. Indeed, even “old” physical products require a product manager to keep them “new and improved.” The emerging information quality assessment tools can provide the mechanisms to measure and monitor the process. Periodic reassessments can determine whether the quality of information is improving — both objectively and as perceived by information consumers.
Having a set of standards is critically important to the monitoring process. There must be standards that define information along all the important quality dimensions of the information consumer. Furthermore, under the leadership of the IPM, these standards should be continuously reviewed and improved to ensure that the quality of the information itself is continuously improved. Analysis of variances between the quality of the information products and consumers’ quality expectations provides the basis for improvement.
The IPM uses knowledge obtained by monitoring to initiate changes to the product — the continuous improvement process. Continuous improvement should start as soon as the information product is developed. As analyses during monitoring reveal gaps, the IPM should undertake improvement. Because the standards are continuously improved, gaps between the standard and the actual will continue to exist. These gaps drive the information quality improvement effort. The IPM’s task is to set new quality goals based on evolving information consumer needs and to translate these goals into improvements in the raw information provided and the information production process.
The IPM’s duties are, in essence, process management and coordination. Traditionally, the IT function performed information process management, if it was performed at all. The IT function, however, has seldom viewed information consumers as part of the process it managed, even in IT organizations with a strong consumer focus. The poor quality of organizational information is commonly caused by the lack of coordination and of shared knowledge among information consumers, producers, and suppliers. Thus the role and responsibilities of the IPM are critical for delivering information that is fit for use by information consumers.
The IPM does not replace the CIO. The leadership of the IT function and its agenda rest with the CIO. The IPM reports to the CIO and is charged with implementing and managing the integrated IP approach —the keystone of providing quality information throughout the organization.
The twenty-first century will witness only two kinds of companies: those that exploit IT and those that are out of business. But for what purpose will they exploit IT? To deliver quality information. Until businesses manage information as a product, they will not deliver high-quality information to consumers consistently and reliably. The results will be lower margins, missed opportunities, and tarnished images.
The path to an IP approach begins with a strong customer orientation. Meeting customers’ information quality needs is foremost. Top management must make a commitment to the IP approach and support the IPM function. With the proper focus and commitment in place, the IPM can diagnose the information quality problem and develop innovative solutions. In so doing, the IPM can adapt proven best practices and deploy established total quality management methodologies.
In an increasingly networked world, information of varying quality is being aggregated for business use. Senior management’s course of action is clear: view, manage, and deliver information as a product.
1. S. E. Madnick, “Database in the Internet Age,” Database Programming & Design, January 1997, pp. 28–33.
2. T.C. Kinnear and K. Bernhardt, Principles of Marketing, third edition (Glenview, Illinois: Scott Foresman/Little, Brown, 1990).
3. R.Y. Wang, Y.W. Lee, and D. Strong, “Can You Defend Your Information in Court?,” Proceedings of the 1996 Conference on Information Quality (Cambridge, Massachusetts: MIT Total Data Quality Management Program, 1996), pp. 53–64.
4. B.K. Kahn, D.M. Strong, and R.Y. Wang, “Information Quality Benchmarks: Product and Service Performance,” Communications of the Association for Computing Machinery, forthcoming.
5. P.B. Crosby, Quality Without Tears (New York: McGraw-Hill, 1984);
E.W. Deming, Out of the Crisis (Cambridge, Massachusetts: MIT Center for Advanced Engineering Study, 1986); and
A.V. Feigenbaum, Total Quality Control, third edition (New York: McGraw-Hill, 1991).
6. M.H. Meyer and M.H. Zack, “The Design and Development of Information Products,” Sloan Management Review, volume 37, Spring 1996, pp. 43–59.
7. D.A. Garvin, “Quality on the Line,” Harvard Business Review, volume 61, September–October 1983, pp. 65–75.
8. J.M. Juran, Juran on Leadership for Quality: An Executive Handbook (New York: Free Press, 1989).
9. J.R. Hauser and D. Clausing, “The House of Quality,” Harvard Business Review, volume 66, May–June 1988, pp. 63–73.
10. D.M. Strong, Y.W. Lee, and R.Y. Wang, “Data Quality in Context,” Communications of the ACM, volume 40, May 1997, pp. 103–110.
11. R.Y. Wang and D.M. Strong, “Beyond Accuracy: What Data Quality Means to Data Consumers,” Journal of Management Information Systems, volume 12, Spring 1996, pp. 5–34.
12. R.Y. Wang, “A Product Perspective on Total Data Quality Management,” Communications of Association for Computing Machinery, volume 41, February 1998, pp. 58–65.
13. D.P. Ballou, R.Y. Wang, H. Pazer, and G.K.Tayi, “Modeling Information Manufacturing Systems to Determine Information Product Quality,” Management Science, volume 44, April 1998, pp. 462–484; and
D.M. Strong and S.M. Miller, “Exceptions and Exception Handling in Computerized Information Processes,” ACM Transactions on Information Systems, volume 13, issue 2, 1995, pp. 206–233.
14. P. Cykana, A. Paul, and M. Stern, “DoD Guidelines on Data Quality Management,” Proceedings of the 1996 Conference on Information Quality (Cambridge, Massachusetts: MIT Total Data Quality Management Program, 1996), pp. 154–171.
15. ISO 9000 International Standards for Quality Management (Geneva, Switzerland: International Standard Organization, 1992).
16. J. Sandberg, “At Thousands of Web Sites, Time Stands Still: Many Web Sites Need Updating,” The Wall Street Journal, 11 March 1997, p. B1.
17. Y.W. Lee, D.M. Strong, L. Pipino, and R.Y. Wang, A Methodology-Based Software Tool for Total Data Quality Management (Cambridge, Massachusetts: MIT Total Data Quality Management Program, TDQM-97-02, 1997).
18. Information Quality Assessment Survey: Administrator’s Guide (Cambridge, Massachusetts: Cambridge Research Group, 1997).