Warning: is_readable(): open_basedir restriction in effect. File(D:\InetPub\vhosts\mikaaho-763.package\pm-index.com\wwwroot/wp-content/plugins/http://pm-index.com/wp-content/plugins/slideshare/languages/slideshare-en_US.mo) is not within the allowed path(s): (D:/InetPub/vhosts/mikaaho-763.package\;C:\Windows\Temp\) in D:\InetPub\vhosts\mikaaho-763.package\pm-index.com\wwwroot\wp-includes\l10n.php on line 476
The study behind PM-Index |

The study behind PM-Index

Research Design

The model was first presented in the Mika Aho’s doctoral dissertation, published in November, 2011. As the dissertation is written in Finnish, this website attempts to summarize the most important points in English for a wider audience.

The overall research approach can be classified as a constructive case study. The constructive research approach is a research procedure for producing constructs which, according to Kasanen and others (1993), ‘produce solutions to explicit problems”. The authors also state that the functionality and innovativeness of any such solution has to be demonstrated by implementing it in practice.

The empirical evidence was collected between 2009 and 2011 from five large Finnish manufacturing companies using different data collection methods such as direct observation, interviews and questionnaires. The five case studies were chosen for practical reasons. The case studies represent business intelligence and performance management projects in which the author participated as a consultant. The number of employees in the companies studied ranged from approximately 300 to over 11000 employees and total annual revenues ranged from 75 million to over 2.6 billion euros, so the case studies represent a range of different organisational structures. However, the concept of performance management was studied in a corporate level in each case company. The paper also includes an extended literature review which was conducted in order to assess the current state of performance management and to look at existing maturity models in the field, as well as the issues and challenges highlighted through their implementation.

Process for identifying the components of performance management

The data was collected from the participant observations, interviews, questionnaires, and from the relevant literature. Yin (2009) stresses that analysing the data is the most difficult phase of any case study. In this study, the evidence was collected and analysed in parallel. The whole process can be classified as evidence-based reasoning. In this kind of reasoning, some of the components, such as technology, were introduced into the model as being already known, i.e. they are already a central element of any modern performance management process.

A total of 457 analytical units in the form of words, sentences and quotations were gathered from the empirical evidence and the literature. Various methods – such as quantifying, theming, typing and content analysis – were used to analyse this evidence, which was finally reduced to 56 analytical units. From this set, the researcher began to discover patterns, recurring themes and expressions which could be grouped in the same category. This was the most inductive – data-driven reasoning – phase in the study. Eventually, after several iterations, the author was able to identify five main components which comprise performance management. These are i) information, ii) intangible assets, iii) performance, iv) strategy and business, and v) technology. These five main components are each divided into three sub-components. Moreover, four supporting components were identified that can be used together with any of the main components. The four supporting components are i) management and responsibility, ii) communication, iii) methods and tools, and iv) scale and scope. These allow performance managent to be analyzed from a number of different perspectives.

Process for identifying the maturity levels and their content

A five-level maturity model was created to denote the organization’s performance management maturity in terms of the above components. These five maturity levels draw on the core concepts of the Software Engineering Institute’s Capability Maturity Model, but the focus is solely on performance management. The process for identifying the maturity levels was the most deductive phase in constructing the model, as the reasoning was directed by existing models and theories.

A total of five maturity levels were identified, and these represent the development process of an organization as it proceeds from one stage to another in its PM initiative. The maturity levels are i) information silos, ii) understanding the value, iii) fact-based decision-making, iv) analytical business, and v) strategic tool.

In order to add content for each component and maturity level combination, the author examined several existing maturity models in the area (e.g. Brudan, 2010; Balanced Scorecard Institute, 2011; Moncla, 2004; Hostmann, 2007; Luftman & Kempaiah, 2007; Eckerson, 2007; Hagerty, 2006; Davenport & Harris, 2007; Daveport & Harris, 2010; Wettstein & Kueng, 2002; Curtis et al., 2009). There were some contradictions between the existing models. For example, Eckerson (2007) suggest that data warehouses should be categorised as level three, whereas Davenport and Harris (2007; 2010) see them as level four. In addition, there were found to be some inconsistencies between individual models: Hostmann et al. (2007;2010) suggest that a competence centre already exists at level three, whereas in their previous studies (Hostmann et al., 2006) the competence centre was only in the process of being formed at that level. However, this could also indicate that the model has been developed over time.

Despite the inconsistencies, the most important elements of each model were collected and placed in the construct at the appropriate levels. A total of 759 analytical units were gathered from the empirical evidence and the literature. These units were in the form of words and sentences. On closer examination, common characteristics such as scorecards, top management support and data warehouses started to emerge.  Once the matrix was finished, the researcher wrote a complete description for each maturity level. At the end of the process these maturity levels were cross-checked for errors and inconsistencies. For example, a strong metric strategy link cannot be emphasized unless this metric is used at the same maturity level elsewhere. After this construct validation phase, some changes were also made to the matrix based on interviews and feedback from representatives of the case companies.

Process for validating the construct

In order to test the validity of the model, a Performance Management Index (PMI) was created. This was based on the above mentioned components and maturity levels, and consists of a total of 283 statements divided equally between the identified components. The statements also have different weights, as they are linked to different maturity levels. On the whole, the development of the index followed the process of scale development described by MacKenzie et al. (2011) beginning with construct conceptualization and culminating in the development of the norms for the scale.

The questionnaire was sent to key employees in the five case companies, and the results were later discussed with the respondents in the form of a face-to-face interview to find out whether the given PMI score corresponded to the beliefs of the respondents. Following this, the index was further refined. The PMI score represents the organisation’s maturity level for each performance management component, or alternatively it aggregates the score for the organization’s performance management as a whole. By looking at the finer points, the respondent can see which components need more attention and where the organization is already performing well.

Comments are closed.