Kaplan and Lamotte (2001) argue that quality program performance measurement need not be exclusive of balanced scorecard measurement systems. They point out differences and synergies between the frameworks:
- The BSC emphasizes explicit causal links through strategy maps ad cascaded objectives more than the quality programs do.
- The BSC targets breakthrough performance whereas the quality programs rely on benchmarking approaches
- The BSC sets strategic priorities for process enhancements.
- The BSC integrates budgeting, resource allocation, target-setting, reporting and feedback on performance into ongoing management processes.
Quality programs, while grounded in product quality improvement and applied to many other business processes (Wruck & Jensen, 1998), are continuous improvement frameworks that might not be best suited to help manage discontinuities in business strategies (Kaplan and Lamotte, 2001). Despite the differences between quality programs and the BSC, Kaplan and Lamotte (2001) see a symbiosis between the two frameworks. They point out similarities in the four perspectives of the BSC and the four sub-categories (customer-focused results, financial and market results, human resource results and organizational effectiveness results) in the Baldrige Business Results category. The EFQM is also converging on more detail in assessing the organization’s results. Proponents of the Baldrige frameworks also note the shift over the years to integrated measurement. In 1997, the Baldrige criteria moved “further away from a perceived narrow focus on ‘managing quality’ to a comprehensive framework for improving overall organizational performance excellence” (Evans, 1997).
Customer Value Analysis
With the recent strong focus on customers, businesses have begun to deploy technologies, and measurement systems, to manage business activities that directly or indirectly interact with the firm’s customers. These customer relationship management (CRM) technologies are providing firms with better data integration and hence better measurement regarding customers. With the obvious strategic importance of customers, it is natural for businesses to begin exploring more robust ways of measuring customers and the related business activities. Gale (1994) explains the role of managing customer value in the context of the Baldrige National Quality Award’s customer focus and satisfaction criteria, which comprises about 30% of the overall score for the award. Customer value analysis is sufficiently rich and complex to require more robust analytical tools and frameworks for measurement and Gale (1994) offers seven tools:
1. The market-perceived quality profile which breaks down the firm’s offering(s) into a set of attributes that are scored and weighted from the customers’ perspective.
- The market perceived price profile which breaks down the firm’s offering price (in many businesses and markets price is composed of multiple aspects) into a set of attributes that are scored and weighted from a customers’ perspective.
- The customer value map depicts the firm’s relative perceived offering price and relative perceived offering performance along and 2-dimensional grid against competitors.
- Won/lost analysis researches further reasons and facts about why a customer defected or decided to buy the firm’s offering.
- Head-to-head area chart of customer value is a graphic display of how the firm’s offering is performing against a single competitor.
- Key events timeline depicts how the firm’s and competitors’ actions change the markets perception of performance of each offering attribute.
- A what/who matrix is a tool for tracking who is responsible for what actions that will change the firm’s ability to improve its ability to manage customer value.
Rust, et al (2000) decompose the customer problem down to three top-level areas (with further decomposition beneath each of the three):
- Value equity refers to the customers’ perceptions of value
- Brand equity refers to the customers’ subjective appraisal of the brand
- Retention equity refers to the firm building relationships with customers and encouraging repeat-purchasing
These three areas correspond to three distinct disciplines in the CRM and marketing literature (brand management, customer value analysis and customer loyalty analysis), each with its own detailed measurement approaches. Numerous other CRM measurement frameworks exist (Kellen, 2002). The implications for BPM systems are clear: measuring business activities and outcomes regarding customers is becoming increasingly complex and increasingly important to the firm’s strategy.
Action-Profit Linkage
Epstein and Westbrook (2001) developed the Action-Profit Linkage (APL) model to help firms identify, measure and understand the causal links between company actions and profits. This framework is a multiple-stakeholder behavior model that lays out the chain of effects as a result of changes in stakeholder behavior. The APL model starts with the corporate strategy and moves to the four main components: company actions, delivered product/service, customer actions and economic impact. Behaviors (and perceptions and attitudes) are measured in each of the components.
For example, management and employee behavior can be measured as activities in the company action component by measuring learning, workload, reward and recognition and culture. Product/service characteristics such as price and quality can be measured along with employee or customer perceptions of the product/service. Customer behaviors such as purchase rates, share of requirements, repeat purchases, cross-sell rates and new referrals can be measured as well as attitudes such as customer satisfaction and intent to purchase. Finally, economic impact such as customer revenues and profitability, number of new customers and market share can be measured.
By presenting managers with an explanation of behavioral linkages between strategy, company actions, product/service improvements, customer behavior and economic impact, Epstein & Westbrook (2001) claim that firms can show some dramatic improvements. Their model shares features in common with BSCs and ABC. Like ABC, APL looks at a firm as a collection of activities. Unlike ABC, APL describes these activities in a strict behavioral linkage. Like the BSC, the APL model accommodates measures from different aspects of a firm and combines internal and external measures and lagging and leading indicators. While the BSC does have this notion of causal linkages between elements within the BSC, the APL model ascribes behavioral causal linkages and because of this focuses more on the company’s actions (Epstein & Westbrook, 2001). Despite the differences between APL and the BSC, Epstein & Westbrook stress the compatibility between the APL and the BSC frameworks.
Reference models
Well-established standard measures for financial performance already exist. Not so for non-financial measures. Different standards groups are developing standard approaches to non-financial measures. TQM, Six Sigma and the EFQM Excellence model are methodologies and management systems that can provide leading indicators of financial performance. The International Organization for Standardization (ISO), along with the large public accounting firms, is working on the processes that generate these non-financial numbers. The Supply Chain Council and the Product Development and Management Association are developing reference models for business processes, which also include specific performance measurements (Smith, 2001).
Just a few BPM reference models exist. Most likely this is due to popularity of the balanced scorecard as a strategic measurement tool and the various quality programs that have become integrated business performance measurement systems more closely supporting the business strategy. Since strategies are often highly differentiated, BPM systems that support the strategy become highly differentiated and propriety as well, including specific sequences of metrics. Hence, reuse on specific metrics and arrangement of metrics is less likely to occur in these strategically aligned frameworks. A reference model is more likely to cover non-core processes or processes that are highly consistent and diffuse across many companies, such as new product development and supply chain management. While many practitioners may desire reference models to ease the initial design and implementation of a BPM, they do not address how to construct a BPM system that is tightly integrated with a differentiated strategy.
Issues with BPM
Commentary on performance measurement can be placed along a continuum ranging from views supporting conventional approaches based on a rational view of decision-making at one end to unconventional approaches that derive from a view based on ambiguity, irrationality or bounded-rationality. In addition, commentaries cover a wide range of topics from data integration and quality, best practices to cognitive psychology that address multiple levels of analysis – including the individual, the workgroup and team, the firm and cultures.
Diversity
Perhaps the first and most important issue with BPM is its diversity. Neely (2002) cites 12 million web sites dedicated to performance measurement, up from 200,000 in 1997, a significant rise in the number of conferences world-wide on the subject and wide-spread adoption of the BSC in large organizations. He also cites tremendous diversity in the academic field as well, with experts in accounting, economics, human resource management, marketing, operations management, psychology and sociology all exploring the subject independent of each other. In 1998 at a multi-disciplinary conference on performance measurement in the U.K., the 94 papers presented cited 1,245 books and articles, of which less than 10% were cited more than once and only 0.3% were cited more than five times. More importantly, Neely argues that there is little agreement on what are the most important themes and theories in performance measurement.
Adoption rates
Citing studies by technology research firm Gartner, Frigo and Krumwiede (2000) report that the balanced scorecard approach is in use at about 40% of Fortune 1000 companies. In the public sector, only 33% of U.S. counties with populations of more than 50,000 were using performance measurement in any form with a similar adoption rate among cities (Berman, 2002). Performance measurement may be following similar diffusion patters as other productivity improvements, which can take a generation to achieve widespread acceptance. Wide variation in the use of information technology may be part of the problem as well. Performance measurement efforts may have more success in measuring activities and outputs, versus outcomes. Outcomes require stakeholder or customer perceptions of timeliness, quality and usefulness of services, which involve data not widely gathered (Berman, 2002).
Data quality
BPM systems typically draw their data from data warehouses that in turn draw their data from source enterprise systems and numerous ancillary software and data sources throughout an enterprise. Bad data quality is affecting the usefulness for data warehouses in general. The Data Warehousing Institute (TDWI) reports in its study of 647 companies on data quality reports that 40% of the companies surveyed have suffered losses, problems or costs due to poor data quality (Eckerson, 2002). Sources of data quality are: lack of validation routines in data entry systems or in system loading; mismatched syntax (first name, last name versus last name, first name), data formats (6-byte versus 4-byte data fields) and code structures (male/female versus m/f); unexpected changes in source systems; the number and complexity of system integration interfaces; poor system design; data conversion errors (Eckerson, 2002).
Strategy and measurement
BPM systems come in two distinct flavors: strategic and operational. The balanced scorecard is an example of a strategic BPM system. Operational BPM systems help managers with specific operational process control issues that may or may not be directly related to the strategy. Numerous researchers have discussed the link between strategy, measurement and success.
Measurement plays a crucial role in translating business strategy into results (Lingle & Schiemann, 1996). In the area of executive management, only 6 in 10 executives place confidence in the data presented to them. Factors that prevent successful measurement include fuzzy objectives (more precise objectives needed), unjustified trust in informal feedback systems, and existing entrenched measurement systems. Those that measure gain agreement on the strategy, clarity of communication, focus and alignment and organizational culture advantages (Lingle & Schiemann, 1996). The survey included 203 executives, 72 percent top executives. 50% of the respondents were from companies with more than 500 employees.
Not all measures are good ones to include in a strategic measurement system. Strategy and performance measurements need to be intertwined, and as such are likely to be unique for each company. Companies should measure how parts of their value chain actually fit together for an overarching advantage instead of relying on process-by-process metrics (Porter, 2002).
Three forces are at work in shaping BPM: increased demands from capital markets for forecasting accuracy; shorter product cycles, quicker market shifts and expanding partnerships; and the growing sophistication and availability of information technology, including ERP software, and improved database and analytic capabilities (Krell, 2002). Despite the widespread understanding of the link between strategy, measurement and success and the need for some balance between internal/external, leading/lagging indicators, internal, lagging metrics dominate performance measurement, rather than external and leading indicators (Krell, 2002).
BPM system satisfaction
The Institute of Management Accountants (IMA) conducted an annual survey of its 1,300 members on performance measurement systems, practices and trends and performed follow-up interviews with respondents. The 2001 IMA survey reported that 80% of its respondents reported making changes in their performance measurement systems during the last three years. Changes ranged from radical (dumping an existing system) to incremental (adding or deleting performance measures, refining the mix of measures). 33% reported the change as a major overhaul. Overall, only 31% of the respondents reported their existing performance measurement system as less than adequate to poor in supporting management’s business objectives and initiatives. Only 15% considered their PM systems as very good to excellent in communicating strategy. Balanced Scorecard (BSC) users fared much better in perception.
|
Percent rating PM system Either “very good” or “excellent” |
||
|
|
BSC users |
Non-users |
|
Supporting management’s objectives and initiatives |
52% |
5% |
|
Communicating strategy to employees |
48% |
3% |
|
Supporting innovation |
22% |
2% |
The key challenges for performance measurement is in intangible assets (human and information capital) and innovation. (Frigo, 2001). In the IMA survey, 60% of the respondents said that innovation was a key part of the firm’s mission statement yet more than 50% rated the BPM system as poor or less than adequate in this area. Overall, less than 10% of the respondents rated performance measures for intangible assets as very good or excellent (Frigo, 2002).
The American Institute of Certified Public Accountants and Lawrence Maisel, from the Balance Scorecard Collaborative, conducted a study to determine current perceptions and practice regarding performance measurement systems. The study included 2000 respondents to a survey and on-site interviews with a smaller number of companies. Only 35% rate their BPM systems as effective and 80% consider the information from their BPM systems as merely “adequate” if not “poor.” The study cited many other points. Many respondents indicated using BSCs even though their BSC system fails to meet the criteria set by the BSC creators (Kaplan and Norton). Performance measurement involves change management, and therefore staff buy-in, education and leadership are all required. Benefits include improved organizational development and leadership, financial performance, operating performance, decision-making, and strategy and alignment. Common barriers cited include issues related to buy-in, leadership, education and the measurement process itself. Better information quality and technology was cited as one of the areas of needed improvement. Financial professionals see PM systems as more effective than their accounting counterparts (Maisel, 2001)
A recent KPMG study of U.S and European business and government executives revealed that one of the most common disappointments reported was the lack of data integrity and the inability of their system to product meaningful information to support decision-making. The study also discovered that BPM systems are not aligned with strategic business measures, dependant on lagging, not leading indicators, are poorly integrated with internal and external information and rely too heavily on financial measures. Some factors for failed BPM systems included: measuring things that are easily measured versus what should be measured, data inaccuracy; measures that were too complicated, and users didn’t understand the system and its measures (KPMG, 2001).
Building measurement systems
BSCs work best if employees have input into the formation of its parameters (Ho & McKay, 2002). The number and type of parameters must be well thought out in advance. Firms should avoid an unmanageable number of parameters so that the using the BSC does not become too cumbersome and time-consuming.
Van Aken & Coleman (2002) identify a process for building BPM systems. After defining the need for measurement and improvement, the process proceeds through the following steps:
· Creating a common understanding of what the organization does (its mission, key processes, and key outputs).
- Defining key performance areas and understanding the metrics (so everyone knows if the process was successful) are next.
- After a balanced and focused set of metrics has been defined, the measurement system must be implemented, taking into account required resources, technology, training, and communication.
- In the remaining steps, the management team must systematically use the measurement system to assess performance, determine improvement actions, and review the impact of these actions.
Van Aken & Coleman (2002) also report that the time it takes to design and implement a measurement system varies and can take as little as one to two months if no major technologies or tools are needed. Kaplan and Norton (2001) also report that the time to create a BSC can be, on average, eight weeks, which is down from the 16 weeks setting up a BSC took in 1996. The use of industry templates for strategy maps and more experience with the BSC contribute to the improvement in time. However, Kaplan and Norton (2001) report that many companies find the BSC harder than it initially looked. Many companies do not implement a complete BSC. Many measures are not collected and over time measures are added and refined. Kaplan and Norton (2001) encourage firms to think of the BSC as a living document that evolves over time.
Jensen & Sage (2000) describe a process for selecting and refining metrics (depicted in Figure 1). Since metrics in a BPM system do change over time, firms need to establish the process for accommodating this change.
Figure 1. Metrics identification and refinement process flowchart (Jensen & Sage, 2000)
Kaplan & Norton (2001) make the argument that BSCs are superior to other forms of performance measurements, such as key process indicators (KPI scorecards) and stakeholder scorecards. BSCs link performance measurement to the strategy, and cause and effect mapping (strategy maps) helps firms identify the indirect links between business activities and overall performance. They warn that BSCs should not just be a collection of financial and nonfinancial measures organized into three to five perspectives. Metric selection should be driven by the firm’s strategy. While formulating a strategy is an art form, describing the strategy (which BSCs do) should not be (Kaplan & Norton, 2001).
Causal and correlative measurement
A key attribute for the BSC and other BPM frameworks is the support for identifying and communicating causal linkages between components of the business that fulfill the strategy. Identifying the causal linkages may be a bottleneck. It might take firms too much time to statistically prove causal linkages between performance metrics, thus severely impairing the causal diagnostic benefit that BPM systems are supposed to supply (Maisel, 2000).
While some firms have begun establishing closed loop measurement approaches, which by their nature seek to understand causality in a deeper fashion, many firms struggle with the more subjective correlative measurement that might provide insight with less effort and in less time. Nail (2002) points out that marketing departments need to master correlative measurement, not just closed loop measurement. Complexity in integrated marketing campaigns is making closed loop measurement harder to do. However, correlative measurement requires a more complex data store and higher level of analytical skills than causal analysis (Nail, 2002).
Successful violations of accepted BPM principals
The widely accepted model of performance measurement adheres to three basic principles: performance should be clearly defined; performance should be accurately measured; rewards should be contingent upon measured performance (Austin & Gittell, 2002). However, continued acceptance on these three principles may be misplaced. Austin and Gittell document cases where firms deliberately violated one or more of these three principles but still exhibited high levels of performance relative to competitors. The widely accepted model of performance is based on compliance and extrinsic motivation. Following Austin & Gittell, these approaches tend to give rise to undesirable outcomes including the distortion of information quality, the displacement of employee effort from real organizational goals to ones that can be more easily pre-specified and measured and a deterioration of performance. In contrast, the firms in the anomalous cases used a performance measurement approach that was based on ambiguity and intrinsic motivation. In these approaches, information quality tends to be high and require dialog across levels in the organization to determine what is the right thing to do in a particular context. Moreover, since individuals are frequently not identified in specific performance situations, the “information-reducing” effect of fear is minimized.
Intrinsic motivations have other benefits. Intrinsic motivation is often required to transfer tacit information. Intrinsic motivation enhances learning and is needed for creativity. Intrinsic motivation can help overcome those situations where it is impossible to specify all the relevant employee behaviors and outcomes needed. Intrinsic motivation is also not without its faults, however. Changing intrinsic motivation is hard and the outcomes are less certain. Intrinsic motivation has its ugly side as well. Envy, vengeance and the desire to dominate are intrinsic motivations. Also, manipulating extrinsic motivations enables more flexible firm behavior (Osterloh & Frey, 2002).
Discriminatory power and coherence
Almost all measures lose their ability over time to discriminate between good and bad performance (Meyer, 2002). Causes include: improved performance, learning how to meet the measure without improving the performance that is sought (perverse learning or gaming); replacing low performers with high performers (selection); and withholding performance data when differences persist (suppression). This requires firms to change measures and search for new measures that can discriminate better.
Activity-based approaches might provide measurement coherence. Meyer (2002) suggest that we think of a firm as a bundling of activities which incur costs and may or may not add value to the customer. “The problem for the firm is finding those activities that add value for the customer and generate revenues in excess of costs, extending those activities, and reducing or eliminating those activities that incur only costs,” (Meyer, 2002). He has proposed activity-centric performance measurement as an approach that can aid in establishing the right measures and improving lateral and horizontal measurement coherency in the organization. His notion of activity-centric performance measurement is derived from activity-based costing, and is referred to as activity-based revenue (ABR). ABR renders individual accountability independent of organization design. This approach is not without problems either. Gathering activity-based measures can be difficult. ABR is also more suitable for complex situations such as when a firm supplies many products to many customers and the product specifications adding value to the customer are not fully understood.
Bounded Rationality
Technology and process considerations aside, decision-making based on measurement data is fraught with individual biases, depending on how the measurement data and problem is presented within the relevant decision-making context. Most managers make decisions about future hypothesized choices that make little sense from the utility theory perspective of a rational assessment of probabilities (Kahneman & Tversky 2000). Prospect theory attempts to explain why people make what appear to be unsound decisions, especially under uncertainty and risk. How decision problems are described (framed) can lead to decision outcomes that deviate from standard decision-making theory (utility theory). These issues around framing and biases can affect decision-making based on performance measurement data. Managers consistently exhibit unwarranted risk aversion and a propensity to look at decisions in narrow terms often isolated from future or past decisions (narrow framing), quite possibly leading to, in aggregate, incorrect management choices. This narrow framing and excessive risk aversion may be unintended consequences of excessive insistence on measurable short-term successes (Kahneman & Lovallo, 1993). A performance measurement system project that proceeds unaware of the framing issues, the heuristics people employ when making judgments under uncertainty and the cognitive biases that even statistical experts possess and employ, could be headed for little impact on the business, or worse still, accelerated faulty decision-making.
When making decisions, many organizations are often overly optimistic (the optimism bias). Despite the use of measurement, analysis of data and worst-case scenarios, decision-makers paint rosy pictures that give an ungrounded illusion of control (Kahneman & Lovallo, 1993). Realism, however, may have its costs. Kahneman and Lovallo (1993) point to research that indicates “the deeply disturbing conclusion that optimistic self-delusion is both a diagnostic indication of mental health and well-being, and a positive causal factor that contributes to successful coping with the challenges of life.”
While the work of Kahnamen, Tversky and others in the prospect theory strain have contributed much to understanding judgments and decision-making, some of the biases and heuristics that prospect theory “uncovers” may be an artifact how information in these probabilistic reasoning tasks is represented (Gigerenzer, 2000). Some of the biases that prospect theory exposes disappear when the information in the problem is represented as a frequency problem rather than a probability problem. Gigerenzer (2000) argues: “While the standard probability format has become a common way to communicate information, … it is only one of many mathematically equivalent ways of representing information.”
Gigerenzer shows how the dramatic removal of one such fallacy, the conjunction fallacy, is achieved with a frequentist representation of the information. Following Gigerenzer (2000), the original conjunction fallacy problem is shown below:
Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student she was deeply concerned with issues f discrimination and social justice, and also participated in antinuclear demonstrations.
Participants are asked with of the two alternatives was more probable:
1) Linda is a bank teller
2) Linda is a bank teller and is active in the feminist movement
His transformed problem asks participants in the study the following:
There are 100 persons who fit the description above (i.e., Linda’s). How many of them are:
a) bank tellers
b) bank tellers and active in the feminist movement
In Gigerenzer’s (2000) research, the conjunction fallacy largely disappears with this reframing.
While the classic depiction of the fallacy above is constrained and not usually naturally found, as is, in BPM environments, this type of reasoning employed by people may in fact be behind many decision-making problems. The implications for business performance measurement is that information representation can have a significant impact in the number of decision errors made as a result of common cognitive limitations.
Use of Metaphor
A further source of errors in reasoning may lie in how the human brain actually processes information and reasons. Assuming that to reason is to construct and manipulate symbolic representations in a language of thought may be wrong. Based on our biology and evolution, our use of visual, spatial and kinesthetic cognitive abilities to process information may be the basis for reason and language (Potts, 2001). The role of metaphor (image schemas and perceptuo-motor mappings) is to provide a structural environment for this reasoning.
Citing Lakoff & Johnson (1999), Potts enumerates some metaphors and associated abstract concepts:
|
Metaphor |
Abstract Schema |
Concrete Schema |
|
Important is big |
Significance in situation |
Seen/felt size |
|
Difficulties are burdens |
Obstacle to intention |
Felt weight |
|
More is up |
Quantity or degree |
Seen or felt elevation |
|
Categories are containers |
Classification |
Seen or felt containment |
|
Similarity is closeness |
Diagnostic/predictive similarity |
Seen or felt proximity |
|
Help is support |
Assistance toward intention |
Felt firmness underneath |
|
Time is motion |
Passage of time |
Seen/felt movement |
|
States are locations |
Situational equivalence |
Seen/felt place |
|
Change is motion |
Variation over time |
Seen/felt movement |
|
Action is self-propulsion |
Autonomous activity |
Intentional movement |
|
Purposes are desired objects |
Intention |
Reinforcing object |
|
Causes are physical forces |
Causes and origins |
Felt pressure and weight |
|
Relationships are enclosures |
Relational dependency |
Seen/felt enclosure |
|
Controlling is being above |
Causal dependency |
Vertical alignment |
|
Seeing is understanding |
Knowledge |
Objects seen |
|
Understanding is grasping |
Knowledge and comprehension |
Objects actively felt |
Potts (2001) suggests that “if all cognition, from everyday preconscious thinking to abstract professionalized mathematical reasoning is grounded in perceptual and performance schemas rather than arbitrary symbolic codes, perhaps we should consider the ways in such schemas can be used in professional descriptions of desired and actual computing and information artifacts.”
Indeed. While Potts was considering the use of metaphor to enrich requirements specification of computing systems, the approach is especially noteworthy with regard to BPM systems. Visualization and metaphor, if is the basis for reasoning skills, may be the best form of further reducing errors in decision-making due to problems of cognition.
Others have classified metaphors differently. Nesbitt (2000) classifies metaphors into five categories:
- Spatial metaphors. These relate to scale, location and structure and can carry quantitative information. Relationships can be described by a position on a map or a two or three-dimensional grid. Structures such as tree maps or data maps can carry broader overview information.
- Temporal metaphors. These are concerned with how data changes over time. It includes concepts of movement, animation, rhythms and cycles.
- Sight metaphors. These use direct mappings from information to the attributes of sight. These include color, light, shape and surface texture. Icons are an example of how abstract shapes can be used to convey information using intuitive symbols.
- Sound metaphors. These deal with direct mappings of typical sound properties such as pitch, amplitude, timbre and also more musical qualities such as rhythm and melody. Auditory symbols are less common, but good examples exist, such as the Geiger counter.
- Touch metaphors. These relate to tactile properties such as force, inertia and vibration. Other properties such as weight, density, hardness and surface texture can be used to encode information.
With abstract data (data that does not conform to the three-dimensional world, such as business information), metaphor classifications would first help guide the analysis and transformation of the data (Nesbitt, 2000).
Defensiveness and Organizational Learning
More danger lurks still. Cognitive limitations and problem framing is just one component of the overall issues in BPM. How organizations deal with threatening information is another substantial concern. It is one thing to have the problem framed appropriately and have the performance data represented properly. It is another to get human beings to discuss the information and its implications and take appropriate action. Argyris and Schön (1996) examine the role defensiveness plays in organizational learning. Organizational inquiry (the intertwining of thought and action that proceeds from doubt to the resolution of doubt) can produce a change in thinking and acting within an organization. This learning, however, has two distinct “flavors” – single-loop and double loop learning. Single loop learning produces a change in behavior, but not a change in the underlying assumptions within the organization. For example, normal product process improvement inquiry may yield changes in the manufacturing process, but not impact basic assumptions about the organization. Double-loop learning on the other hand results in a change in the strategies and assumptions governing the activities (Argyris and Schön, 1996). The relationship between the two forms of learning is depicted in Figure 2.
Figure 2. Double-loop and Single-loop learning (adapted from Argyris, 1999).
Argyris and Schön describe two models of theories in action. These are theories that organizations actually put in to use (theories of action put to use, or action theory), versus theories that organizations say they put into action (theories of action espoused, or espoused theory). The first model, (called Model-I), is one they claim to have observed in many organizations in many environments throughout the world. Model-I theory in use has four main governing variables or values and the accompanying action strategies:
|
1 |
Define goals and try to achieve them |
Design and manage the environment unilaterally (be persuasive, appeal to larger goals. |
|
2 |
Maximize winning and minimize losing |
Own and control the task (claim ownership of the task, be guardian of the definition and execution of the task). |
|
3 |
Minimize generating or expressing negative feelings |
Unilaterally protect yourself (speak in inferred categories with little or no directly observable data, be blind to impact on others and to incongruity; use defensive actions such as blaming, stereotyping, suppressing feelings, intellectualizing) |
|
4 |
Be rational |
Unilaterally protect others from being hurt (withhold information, create rules to censor information and behavior, hold private meetings) |
An organization with this learning model (which Argyris and Schön contend is a great many organizations) is “highly unlikely to alter its governing variables, norms and assumptions,” (Argyris and Schön, 1996). Argyris and Schön offer Model-II theory in use has three main governing variables and accompanying action strategies:
|
1 |
Valid information |
Design situations where participants can be origins of action and experience high personal causation. |
|
2 |
Free and informed choice |
Task is jointly controlled |
|
3 |
Internal commitment to the choice and constant monitoring of its implementation |
Protection of self is a joint enterprise and oriented toward growth (speak in directly observable categories, seek to reduce blindness about own inconsistency and incongruity). Bilateral protection of others |
If this model is followed, the degree of defensiveness between individuals and groups will tend to decrease and double-loop learning will be enhanced (Argyris and Schön, 1996). Argyris contrasts the organizational reasoning associated with these two models as defensive reasoning and productive reasoning. He points out the productive reasoning in dynamic environments is especially difficult for people because it requires them to “reexamine their basic assumptions and test their judgments against changing conditions” (Argyris, 1997). Interestingly, he comments on the need to move beyond a static conception of the world that is reflected in deterministic causality to forms of probabilistic reasoning. “Because the world of action is dynamic and uncertain, probabilistic reasoning is more realistic and accurate in assessing the likelihood of accomplishing our intended result,” (Argyris, 1997). Action can more easily follow from probabilistic reasoning for Model-II organizations. Organizations holding to defensive reasoning is more likely to dismiss probabilistic evidence that challenges the organization’s espoused theory.
For BPM systems, organizational defensiveness has profound implications. While cognitive biases can be overcome by reframing and representing problems so they are more intuitively understood, organizational biases due to defensive behaviors are much harder to root out and change. Argryis points to some hope in the form of management information systems. He states several benefits for these systems (Argyris, 1997):
- Technology allows the design of information practices that support individual and organizational learning
- Storing and retrieving relevant actual performance information is relatively easy and timely
- Individuals can use information technology tools to record and discover discrepancies between stated goals and actual performance in a nonthreatening setting.
- All members of the organization can have access to confirming and disconfirming data, lowering the cloak of secrecy and control
- By linking accurate, timely information to the sense of stewardship among decision makers, the likelihood of learning increases. As organizations begin to change their practices, individuals within them will feel less threatened and more willing to correct their mismatches between intent and action as part of an ongoing development process.
To-date, most BPM systems excel in assisting single-loop learning, that is, correcting specific processes so that they meet stated goals. BPM systems are currently not designed specifically or solely to help manage the double-loop learning problem. Since organizational environments (markets and competitive situations) can rapidly change and extinguish even the largest or seemingly durable entity, enhanced double-loop learning is critical for long-term survival.
Competing Views of Strategic Intent, Strategic Control and Knowledge
In actual practice, companies craft their strategies differently. In some cases, strategy is a planned and deliberate process. In others it is unplanned and emergent. Where one finds strategy crafted in a top-down fashion, one can also examples of firms that created their strategy bottom, up, using grass-roots efforts (Mintzberg, 1987). How a firm actually crafts, executes and controls its strategy can have a significant effect on its BPM system. How firms construct their strategic BPM systems (ones that measure the strategy) will most likely need to mirror how they constructed the strategy itself. Is it conceivable that a top-down strategy in which the details of the strategy are known to a few at the top of the firm can be measured with a BPM system built using bottom-up approaches that involve asking managers and directors closer to the front-lines how to measure the strategy? Emergent strategies will have provisional BPM measurement that must evolve over time, either using top-down and/or bottom-up conversations in the firm to describe how formulate the strategy and how to best measure it
Business environments rarely change on a regular and orderly basis. Firms (and BPM systems) are obviated when the environment shifts in subtle or profound ways. Turbulent changes are easy to detect Some argue that subtle discontinuities are not and can only be dealt with by minds that are in tune with existing patterns yet still able to perceive breaks in them (Mintzberg, 1987). With most knowledge of the business environment today codified in digital data and presented as digital information, personal and direct observation or trusted anecdote via word-of-mouth might be insufficient for detecting this change. Perhaps BPM systems can play a significant role in not just monitoring changes in internal states, but also understanding changes in the external environment as well.
Knowledge as a social phenomenon is emerging as a significant contribution to knowledge management (Thomas, Kellogg & Erickson, 2001). Knowing what comprises the social and cognitive context that contributes to individuals understanding things is critical for successfully managing knowledge in that organization. Some social techniques for managing or creating knowledge include Bohm dialogs, systematic use of metaphor, strategy mapping, story-telling and narrative, expressive communication. Thomas, Kellogg & Erickson (2001) go so far as to re-label knowledge management as knowledge socialization. Viewed in the light of the need for enhanced organizational learning, it is not surprising the social nature of knowledge viewpoint is challenging more traditional thinking.
Stacey (2000) criticizes what he calls mainstream thinking on knowledge management for many oversimplifications or inadequate explications. The key concepts in mainstream thinking – double-loop learning, tacit and explicit knowledge, systems dynamics, sender-receiver models of knowledge transmission from information theory, dialog as a special form of communication – have the following problems: they treat individual learning different than organizational learning and hence require not one, but two theories of how learning takes place, they fail to account for how new knowledge is created and they cannot explain the unpredictable patterns of knowledge that may emerge outside of the control of the managers. Stacey argues that “systems, databases, stored and written artifacts” are simply “records that can only become knowledge when people use them as tools in their process of gesturing and responding to each other.”
Stacey (2001) goes further, arguing that knowledge is not designed but emerges from the conscious and unconscious interactions and gestures between individuals, and as such can be thought of as a complex adaptive system. Systems thinking, which attempts to understand things by examining and reducing the interactions between components of the overall system, is inadequate for understanding how knowledge actually is created and diffuses. Stacey concludes that “from this perspective, it becomes impossible to think of designing such a process and it makes no sense to think of managing it.” In this view, knowledge is intractable.
What this means for BPM is that the art of identifying, linking and gathering data for a BPM system is only part of the problem and not the thorny problem at that. Getting knowledge regarding the performance measurement data diffused and used throughout an organization is at the core of what knowledge management and BPM is all about. That, according to Stacey, may very well be an intractable problem.
Success and Fail Factors
Business performance measurement literature abounds with do’s and don’ts of measurement. Some classic problems include the use of too many metrics; the use of primarily or exclusively financial and cost metrics; the use of only short-term focused metrics; a lack of alignment between operational metrics and organizational objectives and strategies; and metrics that drive the wrong behaviors and performance (Van Aken & Coleman, 2002). In their literature review and industry research, Bititci, Carrie and Turner (2002) identified the following aspects companies look for a performance measure system to address:
· Reflect stakeholder requirements to maximize their satisfaction
- Reflect the external/competitive position of an organization
- Focus on competitive criteria of the organization’s markets in order facilitate strategies and actions to improve the organization’s competitive position
- Provide an input into strategy development
- To help ensure alignment if strategy, actions and measures
- Differentiate between control and improvement measures
- Focus on critical areas of the business
- Use locally meaningful terminology to encourage understanding and ownership
- Facilitate resource bargaining
- Promote proactive measurement by focusing on leading measures
- Accommodate both quantitative and qualitative measures
- Measure organizational capability and learning
- Ensure that measures are used at the correct levels
- Promote understanding of the causal relationships between various measures
- Facilitate simple reporting
- Be dynamic and change in response to the internal and external environments
Based on their experience, Kaplan and Norton (2001) identify the following key pitfalls for BSC performance measurement systems:
· Changes in executive leadership due to mergers and acquisitions in which BSC loses executive sponsorship or priority
- Design problems in which a poor BSC is built
- Too few measures (two to three) per perspective
- Too many measures without clear identification of the critical few measures
- Lack of linkage to the BPM system and sustainable competitive advantage
- Process failures in implementing a BSC
- Lack of senior management commitment
- Too few individuals involved
- Keeping the scorecard at the top of the hierarchy and not distributing it
- Too long of a development process, not incremental
- Treating the BSC as a systems project
- Hiring inexperienced consultants
- Introducing the BSC for compensation purposes only
Based on the current issues regarding BPM systems discussed above and elsewhere in this paper, key success/fail factors that have been cited in the literature are summarized in Table 1.
|
|
|
|
|
|
|
|
8. Problems related to lack of reference models and uniqueness of strategies must be overcome |
|
Table 1. BPM critical success factors
Since today performance measurement data is almost always derived from or communicated with computers and information system technology, it is not surprising that some of the critical success factors are essentially technology problems.
A Framework for Evaluating Performance Measurement Systems
Over the last 20 years or so, established frameworks and reference models, along with specific, lesser- known measurement approaches, APL, IPMS, OPTIMAS (Jensen & Sage, 2000) and others, have populated the business world. Companies have choices when building a BPM. Despite the large amount written about how to measure, much less has been written on what are the key attributes that are desirable in a BPM system. De Haas & Kleingeld (1999) mention seven pre-existing measurement system criteria from other studies and add their concept of coherence (discussed below) to the list to make eight: