SciELO - Scientific Electronic Library Online

vol.8 suppl.1Análisis de la accesibilidad de los sitios web de las universidades ecuatorianas de excelenciaImplementación de una base de datos relacional difusa. Caso práctico: tutoría académica índice de autoresíndice de materiabúsqueda de artículos
Home Pagelista alfabética de revistas  

Servicios Personalizados




Links relacionados

  • No hay articulos similaresSimilares en SciELO


Enfoque UTE

versión On-line ISSN 1390-6542versión impresa ISSN 1390-9363

Enfoque UTE vol.8  supl.1 Quito feb. 2017 

Ingeniería General

Assessment of the Quality of Leadership in the Government of Technologies and Information Systems

Evaluación de la Calidad de Liderazgo en el Gobierno de Tecnologías y Sistemas de Información

Vicente Merchán1 

Rocío Rodríguez2 

1 Universidad de las Fuerzas Armadas ESPE, Sangolquí - Ecuador

2 Universidad Nacional de La Matanza, Buenos Aires - Argentina


Nowadays, leadership and the style of governing have become one of the quality criteria that challenges the Government of Technologies and Information Systems (GoTIS) value-based on public domain. In this aspect, the excellence models and GoTIS standards play a vital role in the success of the organizations due to their usefulness as systems that help the improvement towards the organizational excellence. Two of them are the Ibero-american Model of Excellence in Public Administration and the GoTIS standard ISO/IEC 38500. This paper takes them into account to create a new assessment model of GoTSI, the same one that is validated through the opinion of leaders of technologies and information systems of high-level. The work provides two sub-criteria, eleven dimensions and five components of assessment. This model will allow GoTIS assess itself as a principle of quality and, from there, make an improvement and an organizational change of the processes and GoTIS team decisions. Similarly, it will also help to open the way to the development of new quality criteria from a specific and comprehensive view.

Keywords: TIS government; leadership; measurement and quality assessment; technology and information systems.


En la actualidad, el liderazgo y estilo de gobierno se han convertido en uno de los criterios de calidad que desafían al Gobierno de las Tecnologías y Sistemas de Información (GoTSI) basado en valor en el dominio público. En este aspecto, el conocimiento sobre los modelos de excelencia y los estándares GoTSI juegan un papel vital en el éxito de las organizaciones debido a su utilidad como sistemas que ayudan a mejorar la excelencia organizacional. Dos de estos son el Modelo Iberoamericano de Excelencia en la Administración Pública y el estándar GoTSI ISO/IEC 38500. Este trabajo los toma en cuenta para crear un nuevo modelo de evaluación de GoTSI, el mismo que es validado a través de la opinión de líderes de tecnologías y sistemas de información de nivel directivo. En términos generales, el nuevo modelo se compone de 2 sub-criterios, 11 dimensiones y 5 componentes de evaluación. Este permitirá que el GoTSI se evalúe como principio de calidad y, a partir de ahí, realice una mejora y un cambio organizativo de los procesos y decisiones del equipo GoTSI. Del mismo modo, también ayudará a abrir el camino al desarrollo de nuevos criterios de calidad con una visión específica y completa.

Palabras clave: Gobierno de TSI; liderazgo; medición y evaluación de calidad; tecnología y sistemas de información.

1. Introduction

Quality is essentially one of the factors that allows the organizations of Technologies and Information Systems (TIS) to become competitive in current environment. For decades, quality has become the main strategic aim of many organizations because its survival depends on the quality of services provided to internal and external users.

Before proceeding and in order to avoid confusions, we will use the term "TIS" as synonymous of "IT" or "TIC" based on the definition of the ISO / IEC 38500 standard (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008), which defines this as: "the necessary resources to acquire, process, store and disseminate information".

According to (Pérez Juste, López Rupérez, Peralta Ortiz, & Municio Fernández, 2014), quality is defined as " … quality, as a complex reality, must be understood as an inclusive harmonization of these different elements: effectiveness achieving a service, product or excellent object, through efficient processes, that are satisfactory to the direct and indirect recipients, and to the staff of the organization responsible for achieving it”.

In a definitional sense, the scientific and cultural revolution of the organizations in the twenty-first century demands a new paradigm, which is called "inclusive culture" (Pérez Juste, López Rupérez, Peralta Ortiz, & Municio Fernández, 2014), because it belongs to a new conception of the world and life. The inclusive culture includes historical aspects of quality in order to search for a better solution to the problems of the organizations in all of their aspects. The quality stops to focus on the good or on the product and becomes a requirement of the entire organization through its leaders. The role of the management and the employee involvement are the essential factors of change.

In the case of leadership that is understood as a quality criterion, it has been written about the influence of this aspect not only in general, but also in specific and exhaustive of the subject. Among them: Nwabueza (Nwabueze, 2011), Bäcktröm (Bäcktröm, Wiklund, & Ingelsson, 2011), Doeleman (Doeleman, Have, & Ahaus, 2012), Ooi (Ooi, 2012) and González (González Rosas, Carrión García, & Acosta Uribe, 2014) have made studies to analyze it. The leadership has been translated into models of excellence and quality for the evaluation of organizations through sub-criteria, dimensions and components (Benavides Velasco & Quintana García, 2003) (Martínez-Vilanova y Martínez, 2008) (Fundación Iberoamericana para la Gestión de la Calidad [FUNDIBEQ], 2015). Similarly, it has been translated into the principles of ISO / IEC 38500 (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008) standard for the evaluation of the Government of Technology and Information Systems (GoTIS).

The organization directing, evaluating and controling regardless of its quality criterias. The use of TIS should also be directed and controlled, under the support of a GoTIS value-based leadership. In this regard, the guidance provided by the ISO / IEC 38500 (Toomey, 2009) (Fernández Martínez & Llorens Largo, 2012) (International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC), 2008) standard is appropriate. However, there are related works demonstrating the need for innovation from the perspective of quality assessment in the GoTIS (Sánchez Peña, 2013) (Merchán & Rodriguez, Towards Value-Based Information Technology Leadership Excellence, 2016).

According to directing perspective, there are quality criteria that are covered by a full understanding of the principles of the standard. However, these are not an aim in which the standard is focused. Consequently, this allows reviewing the concepts of "GoTIS quality" and specially the leadership concepts proposed in other studies. This also allows proposing in our context a set of sub-criteria, dimensions and components that guide the evaluation processes of the quality of leadership seen as a criterion. The new model is validated through a research survey (Pfleeger & Kitchenham, 2001), that reflects the opinion of technology and information systems leaders.

This paper has four more sections. The second section provides the background about assessment models from the conceptual point of view, emphasizing leadership. The third section, GoTIS value-based assessment model of leadership is proposed. The fourth section, the proposal is validated. Finally, the conclusions and future works are stated in fifth section.

2. Assessment models

Some evaluation models are described in the context of quality:

Model MBNQA (BNQP - Baldrige National Quality Program, 2015): Is responsible for evaluating and improving the performance through 7 basic criteria: leadership, strategic planning, customer and market focus, information and analysis, the focus of human resources, process management and business results. The program provides a total score of 1000 points in the final evaluation. The model defines leadership as the rank that guides and keeps the organization. It includes the government system, the ethical and social responsibilities, and key communities.

EFQM (European Foundation for Quality Management [EFQM], 2013): It is responsible for improving the business performance by understanding the strengths and weaknesses that need to be improved through the assessment of 5 enablers or "criteria" such as: leadership, strategy, people, suppliers and resources and, processes, products and services; and 4 agents of results that are: customers, people, society and key. The model defines leadership as the model for its value and ethics, being a person who inspires reliability. It looks at the future and makes things happen. Flexible people allow the organization to anticipate and react in time to ensure its success.

Ibero-american Model (Fundación Iberoamericana para la Gestión de la Calidad [FUNDIBEQ], 2015): The model is aimed to the public administration. It consists of 5 facilitators criteria and 4 criteria results. Each criterion has its respective sub-criteria. The facilitators criteria are: Leadership and government style, strategy, people development, resources and partners, and processes and customers-citizens. The criteria results are: Client-citizens, people development, society and globals. All of them assign a total score of 1000 points in the final evaluation. This model defines leadership as the process that directing and guiding the activities of the members of a group influencing on the process to route their efforts towards achieving a goal or specific goals.

There are some works of TIS that have used these assessment models with relevant purposes and that are now taken into account. Such is the case of Municio (Municio & Red Universitaria de Evaluación de la Calidad RUEC, 1998) who led to the creation of the high education quality assessment model with the definition of variables based on TIS. Sanchez (Sánchez Peña, 2013) created two models to assess the quality of GoTIS based on ITIL and COBIT; Merchán (Merchán & Rodríguez, Análisis de los modelos de Gobierno de Tecnologías de la Información y sus relaciones con el Modelo de Excelencia Iberoamericano, 2015) analyzed the relationship between the ibero-american model and ISO/IEC 38500.

3. Proposed quality model

If you do not have an evaluation model and works as described above, it would be an arduous task to determine the GoTIS leadership as a quality criterion. Thus, based on this information, a critical analysis of the criterion of leadership is performed and a description of the sub-criteria, dimensions and components of quality assessment is proposed. This proposal received suggestions from three experts on government. Finally, the operational validation of the proposal took place by a group of twenty-seven leaders of TIS that are on leadership positions and / or Chief Information Officers public and private organizations in Ecuador.

In addition, this work is not about looking adoptions nor adaptations to current dysfunctions. What is sought is an approach to assessing the quality of GoTIS leadership for the local environment in the following terms (Merchán & Rodriguez, Towards Value-Based Information Technology Leadership Excellence, 2016):

1) Identification of basic quality principles on which the proposal is based: A first principle of leadership is in the framework of actions, the systematic support and the commitment of the senior management to quality through the process of GoTIS decision making. In the process of governance, the strategy, mission and vision are formulated, the objectives are expressed, the plans and projects are approved, the resources are organized, the technologies are selected, and the processes and results are evaluated. The leader who supports quality does it consciously and coherently. This creates a culture in which every action and decision is at the service of quality. The evaluation measure is the second leadership principle, which translates the criteria value into control elements that can be evaluated in terms of their level of compliance.

2) Evaluation model: Figure 1 shows the structure of the proposed evaluation model, which includes two quality sub-criteria: Culture of value-based excellence, value-based TIS-Enterprise Architecture; and eleven dimensions (which will be detailed later in this section).

Fig. 1: Structure of the evaluation model 

Each sub-criteria includes characteristics of TIS organization considered necessary to get good results through the leadership criteria and the defined governance style. The content of the sub-criteria meets the quality principles described in the previous subsection, and captures the essence of the reviewed principles of quality models. Both sub-criteria are defined in terms of 11 assigned dimensions.

The hierarchy of the evaluation model to guide the classification of sub-criteria, dimensions and evidence is appreciated in Figure 2.

Fig. 2: Hierarchy of the evaluation model 

The previous model defines the Value at the top of the hierarchy acting as a facilitator that is related directly to the effects of the business. Leadership and its effects are consistent with the perception of value that the business will have. Likewise, the sub-criteria influence on the definition of the dimensions, and these influence on the evidence. It is important to state that the evaluation model provides feedback for quality criteria. The dimensions are designed to measure the achievement of each of the sub-criteria. The dimensions have the characteristic to define the most significant aspects of the sub-criteria. Finally, the evidences, representing quantitative or qualitative data, facilitate the value judgment of the accomplishment of the dimensions on GoTIS team. Table 1 describes the components that make up the leadership criterion.

Table 1 The components that make up leadership 

Each sub-criterion is defined by dimensions that represent general aspects of what is included in the model. These in turn are evaluated by five components that make up the evidences: Importance, Diffusion, Utilization, Monitoring and Perceived value. Tables 2 and 3 show the dimensions and effects of the fifth component: Perceived value.

Table 2 Topics to be assessed in the culture of excellence 

Table 3 Topics to be assessed in TIS-Enterprise Architecture 

3) Method of measurement and assessment: The quality is introduced following a systematic hierarchical process for each sub-criterion. This process allows to get orderly, systematically and consistently the subjective elements of the evaluators and then incorporate quantitative elements estimating the value. The general rating scale shown on Table 4 consists of three hierarchical and independent levels of evaluation that have to be analyzed properly.

Table 4 General rating scale 

The first level measures the implementation of GoTIS quality system. The measurement of the implementation is performed on the dimensions that define each sub-criterion. Each dimension is measured in terms of two Implementation components: Importance and Diffusion. The second level measures the application of GoTIS quality system. The Application measurement is performed on the dimensions that define each sub-criterion. Each dimension is measured in terms of two components of Application: Utilization and Monitoring. The third level includes the measurement of GoTIS quality system effects according to the perceived value by the shareholders, executives and/or employees. Finally, the score assigned to each level corresponds to a criterion of importance that it represents.

Each dimension is assessed in terms of its components and each component is valued according to a Likert rating scale with five scores as it is shown on Table 5.

Table 5 Scoring 

The concepts of successions and series (Allen R., 2008) are used to develop equations that allow to calculate the components, dimensions, sub-criterion and criterion of the model.

Equations (1) and (2) allow to evaluate the Importance I[i] and Diffusion D[i], respectively; of the i-th dimension to a set of k evaluators.




score (1) and (2) represent the value assigned by the assessor to the component according to the rating scale.

The level of Implementation of the sub-criterion IMSC of the i dimensions set is measured by (1) and (2) as shown in (3).



Weight represents the highest value of the scoring.

Equations (4) and (5) allow to measure the Use U; and Monitoring S, respectively of the i-th dimension for a set of k evaluators.




score (4) and (5) represent the value assigned by the assessor to the component according to scoring.

The Application level of the APSC sub-criterion of the i dimensions set is measured through (4) and (5), as shown in (6).



weight of (6) represents the highest value of the scoring.

Equation (7) allows to measure the Perceived Value V[i] of the i-th dimension to a set of k evaluators.



score (7) represents the value assigned by the assessor to the component according to the scoring.

The level of Effect of the EFSC sub-criterion of the i dimensions set is measured through (7) as shown in (8).



weight of (8) represents the highest value of the scoring.

Equation (9) determines the total measure of a sub-criterion SC.



IMSC, APSC and EFSC represent the implementation measures, application and effect of the respectively sub-criterion; and, PIM, PAP and PEF represent the weighting values assigned to the implementation level, application and effect respectively.

Equation (10) determines the total measure of the CR criterion and therefore of GoTIS leadership quality system.



n represents the number of sub-criteria associated with CR criterion.

The total measurement of CR criteria will be evaluated according to quality parameters, which were also defined by TIS leading experts. Table 6 shows the level of leadership quality by percentage. This table shows a range of proportional measure between the number of levels that are estimated to have the same importance.

Table 6 Level of leadership quality 

The evaluation process should be supported in the use of questionnaires and/or interviews, tables and graphs for each sub-criterion, as part of the work of the evaluation team. The use of tools that support the process to replace the data of leaders avoiding the possible influence of an evaluator and that can influence the process is not excluded.

Finally, the self-evaluation process follows three steps: Awareness, Planning, Implementation, Learned Lessons and Final Report.

4. Validation of the proposed quality model

The validation of the model featured the participation of 27 leading managers of TIS properly chosen by the nature of the research project. A questionnaire of 15 questions was delivered: 4 related to demographic data and 11 related to leadership criteria data.

To collect the opinion of the experienced people, the Likert score was used in the answers on a scale of 1 to 5. Where:






The demographics data of the selected TIS leaders are analyzed on Table 7.

Table 7 Demographic data of surveys 

Analyzing the demographic results it can be seen that the percentage of leaders based on the primary sector to which they belong is higher in services by 48.1%. This shows that the services sector excels in the use of TIS. The other fact is the gender whose majority participation is by the male gender with 77.8% compared with 22.2% of the female gender. This difference may be circumstantial although it is possible that the male gender actually has greater access to technology for other aspects. The studies of the leaders reflect that most have master's level with 74.1%. Finally, there is the age of the leaders where mostly of them are found in the group of 36-45 years old by 59.2% of the participants.

Meanwhile, the support for the creation of the evaluation model through the importance of the dimensions that define the sub-criteria is evidenced by noting that the percentage of importance (Moderate + Complete) given to each of the dimensions exceeds 90% except for the dimension 2.a (Figure 3 and Figure 4).

Figure 3 ilustrates the importance that the leaders give to the dimensions that define the sub-criterion of culture of excellence, where it is observed that the dimensions with "complete importance" are 1.a and 1.b. The active involvement of the leaders of TIS dominates in these dimensions and in the definitions of key strategic objectives generators of value in the organization. Otherwise, stands the percentage of "complete importance" in the other dimensions.

Fig. 3: Level of importance of the excellence culture sub-criterion 

Figure 4 presents the importance that leaders provide to the dimensions that define TIS-enterprise architecture sub-criterion where it is noted that the dimensions with "completa importancia" are the 2.c and 2.d. The monitoring and control in the delivering value in TIS organization predominates on these dimensions. On the other hand, it should be noted the variety in the level of importance that the leaders provide to the 2.a dimension. According to the expert´s opinion, the definition of GoTIS by the senior executives would have let various interpretations; for example, the participation of the CEO and shareholders that are not exactly part of the senior executives. In any case it is a matter of GoTIS experience. In conclusion, IT stands the percentage of "complete importance" in all dimensions.

Fig. 4: Level of importance of TIS-Enterprise architecture sub-criterion 

With Alpha (α) of Cronbach (Cronbach, September 1951) it is determined the reliability or consistency of the results obtained in the comprehensive opinion survey. For this we proceeded with the analysis using the method of variance of the items of the questionnaire, obtaining a value of 0.885. This result is considered optimal or high reliability in view of the fact that it is close to 1 (Aragón & Sánchez, 2012), interpreted as efficiency in the test.

5. Conclusions and Future Works

The work presented in this paper was supported on related works; therefore, the premise that was used on this proposal was based on using leadership and government style as one of the quality criteria that generates effectiveness and competitiveness to an organization of TIS.

The methodology used on this paper is based on critical analysis of leadership that emphasize the assessment models and related works, which leads to the definition of two sub-criteria, eleven dimensions and five components that received suggestions from experts. Finally, the validation of the proposal took place by a group of twenty-seven leaders of TIS that are on leadership positions and/or on high-level on public and private organizations in Ecuador.

The main reflections of the work are:

The proposal highlights the leadership and government style as a criterion of quality management with directing perspective, as it is the conception of models of excellence that supports them.

The criterion of leadership is known for its commitment to culture of excellence, whose strategic development is supported in TIS-enterprise architecture.

The quality of leadership is given by a set of sub-criteria and dimensions. The dimensions are evaluated by components that stand out on the importance of TIS investigated leaders.

In general, it stands out the "complete importance" that the leaders provide to the dimensions 1.a and 1.b, with 96.3% and 81.5%, respectively. This aspect is explained in the active involvement of the leaders of TIS and in the definitions of the key strategic objectives that generate value in the organization.

Dimensions: 1.c, 1.d and 1.e; stand out by the average percentage of 97% of "complete importance" and "moderate importance”, confirming the active participation of the leaders in GoTIS commitments.

The 2.a dimension brings up attention because of the "divided" opinions; nevertheless, it highlights the most important with 77%. This aspect may be due to the diversity of GoTIS experiences in Ecuador.

Dimensions: 2.b, 2.c, 2.d, 2.e and 2.f show an interesting opinion on where stands out the "complete importance" of the behavior that must have the leaders through the decision making for both control and GoTIS monitoring team.

From the point of view of the results, it is possible to evaluate the leadership not only seeing the satisfaction effect, as it is done, but also seeing the perceived value effect.

The assessment process allows getting in an orderly, systematically and consistently manner the subjective elements of the evaluators and then incorporate quantitative elements in the estimation of the value.

The work highlights the importance of expanding the set of criteria that will lead to a comprehensive solution characterized by the existence and value delivery which meet the national need and the public administration policy issued by the national government. For this reason, the next step of this research focuses on the advancement of a more complete and easy definition, based on this methodology of measurement and assessment from the perspective of direction and/or GoTIS, which will be discussed in future works.


Allen R., A. (2008). Álgebra Intermedia (Séptima ed.). México: Pearson Educación. [ Links ]

Aragón, A., & Sánchez, G. (2012). Metodología de la investigación. (Universidad de Murcia) Retrieved 2016, from ]

Bäcktröm, I., Wiklund, H., & Ingelsson, P. (2011). Learning from others to adapt quality management to the future. Total Quality Management & Business Excellence, 22(2), 187-196. [ Links ]

Benavides Velasco, C., & Quintana García, C. (2003). Gestión del Conocimiento y Calidad Total. Madrid: Díaz de Santos. [ Links ]

BNQP - Baldrige National Quality Program. (2015). Criteria for Performance Excellence. Retrieved from http://www.nist.govLinks ]

Cronbach, L. (September 1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297-334. [ Links ]

Doeleman, H., Have, S., & Ahaus, K. (2012). The moderating role of leadership in the relationship between management control and business excellence. Total Quality Management & Business Excellence , 23(5-6), 591-611. [ Links ]

European Foundation for Quality Management. (2013). EFQM Excellence Model. Recuperado el 2015, de ]

Fernández Martínez, A., & Llorens Largo, F. (2012). Gobierno de las TI para las universidades. Madrid: CRUE. Retrieved 01 22, 2014, from ]

Fundación Iberoamericana para la Gestión de la Calidad (FUNDIBEQ). (2015). FUNDIBEQ. Recuperado el 2015, de http://www.fundibeq.orgLinks ]

González Rosas, E., Carrión García, A., & Acosta Uribe, B. (2014). Comparativa conceptual de los criterios de liderazgo en modelos de calidad total. Congreso internacional de contaduría, administración e informática. México D.F. [ Links ]

International Organization for Standardization (ISO) and the International Electrotechnical Comission (IEC). (2008). ISO/IEC 38500:2008. Corporate Governance of Information Technology. Retrieved 07 18, 2014, from ]

Martínez, Vilanova y Martínez, A. (2008). Modelo de Evaluación y Diagnósis de Excelencia en la Gestión, basado en el Cuadro de Mando Integral y el Modelo EFQM de Excelencia. Aplicación a las Cajas Rurales. Tesis doctoral, Universidad Politécnica de Valencia, Organización de Empresas, Economía Financiera y Contabilidad, Valencia. [ Links ]

Merchán, V., & Rodríguez, R. (2015). Análisis de los modelos de Gobierno de Tecnologías de la Información y sus relaciones con el Modelo de Excelencia Iberoamericano. XXI Congreso Argentino de Ciencias de la Computación-CACIC 2015. Buenos Aires - Argentina. [ Links ]

Merchán, V., & Rodriguez, R. (2016). Towards Value-Based Information Technology Leadership Excellence. In C. Collazos (Ed.), IEEE 11CCC 2016. Popayán: IEEE Colombia, 2016. [ Links ]

Municio P & Red Universitaria de Evaluación de la Calidad RUEC. (1998). Evaluación de la Calidad de la Educación Superior. Un modelo de autoevaluación institucional. (1 ed.). (W. K. Educación, Ed.) Madrid-España: Red Universitaria de Evaluación de la Calidad. [ Links ]

Nwabueze, U. (2011). Implementing TQM in healthcare: The critical leadership traits. Total Quality Management & Business Excellence , 22(3), 331-343. [ Links ]

Ooi, K.-B. (2012). The effectiveness of TQM: A simulator for knowledge distribution? Total Quality Management & Business Excellence , 23(5-6), 653-671. [ Links ]

Pérez Juste, R., López Rupérez, F., Peralta Ortiz, M., & Municio Fernández, P. (2014). Hacia una educación de calidad. Gestión, Instrumentos y Evaluación. (5ta). Madrid-España. [ Links ]

Pfleeger, S., & Kitchenham, B. (2001). Principles of Survey Research part 1: Turning lemons into Lemonade. ACM SIGSOFT: Software Engineering Notes, 26(6), 18-20. [ Links ]

Sánchez Peña, J. J. (2013). Evaluación de la calidad en el ámbito del Gobierno y la Gestión de Servicios de TI. Tesis Doctoral Universidad de Alcalá, Ciencias de la Computación, Alcalá de Henares. [ Links ]

Toomey, M. (2009). Waltzing with the Elephant: Una guía exhaustiva para la dirección y el control de la tecnología de la información. (J. Pardo Martínez, Trans.) Australia: Infonomics Pty Ltd. [ Links ]

Received: January 09, 2017; Accepted: February 24, 2017

Creative Commons License This is an open-access article distributed under the terms of the Creative Commons Attribution License