The seminar “Conceptual approaches to the development and use of Key National Indicators System” was organized during the Second meeting of the INTOSAI Working Group on Key National Indicators (KNI). Main questions of the seminar were:

1. Рrogress definition and measurement;

2. Рrogress measurement in the activities of Supreme Audit Institutions (SAI), regional and international organizations;

3. Methodological approaches to use of progress measurement indicators by SAIs;

4. Use of the KNI system in the conditions of global financial and economic crisis.

The aim of the seminar was to promote dialogue, discussion and strategic proposals on how to reinforce the use of KNI in SAIs activity.

1. Mr. Peng Xinlin, Deputy Director of Law Department of the NAO of China.

Relevant measurement indicators for scientific development of China as well as its application in auditing.

Mr. Peng Xinlin in his presentation pointed out that 9 main targets of economic and social development and 22 main quantitative indicators are set in the 11th Five-Year Plan (2006-2010). Indicators are divided on 14 anticipated and 8 obligatory. Also there are special-purpose development outlines by Ministries based on 11th Five-Year Plan. They nail down specific development goals and put forward more concrete and detailed measuring indicators.

Mr. Peng Xinlin highlights the evolution of government auditing. Now auditors not only check but also examine performance of government and other financial revenues and expenditures. Society and eco-system are now not less important than economic benefits. Institutions in China collect measuring indicators from different channels.

He emphasizes important tasks for Auditors:

a) Contrast and analysis between economic and environmental indicators.

b) To prevent and control water pollution.

c) Environmental control, protection and improvement.

2. Mr.Teemu Kalijärv, Principal performance auditor of NAO of Finland.

Mr. Ville Vehkasalo, Principal performance auditor of NAO of Finland.

What do key national indicators indicate and how should we audit them?

The speaker underlines interrelation of all factors. It is impossible to define Government activity success in any sphere by one indicator. It is necessary to take into account interrelations of all factors. Finland doesn’t have the System of KNI. But Finland has a developed statistic system that’s why there is a background for development the System of KNI. Not all statistic data reflex the condition, for example GDP, that’s why there are non-official indicators. The growth of GDP doesn’t mean prosperity. SAI of Finland doesn’t actively participate in indicators development, because active participation in it can influence its independence and objectivity. SAIs could use existing indicators in audit but it’s necessary to be sure in their confidence. That’s why SAIs shouldn’t develop indicators but they should control them. Key National Indicators should based on national features.

Concluding remarks of the presentation:

• Reliable indicators (= statistics) are always needed 
• ‘Key national indicators’ should be based on national circumstances 
• International comparisons are difficult and possibly biased 
• SAIs should audit whether indicators are in fact valid and reliable 
• SAIs could also utilize indicators when analysing policy effectiveness

3. Mr. Jorge Pedro Castolo Dominguez, General Director, Performance Audit Office on Economic Development of Mexico.

Experience and progress in Mexico.

Mr. Jorge Pedro Castolo Dominguez pointed out main regulatory documents on global and institutional levels.

The speaker specified that public Management has evolved, through the last decades, from a traditional bureaucratic model to a outcome-oriented model.

The new model focuses on the impacts of public actions implemented for the benefit of society.

The most important aspect to consider is the assessment on the compliance of goals, outcomes, performance indicators and benchmarking standards.

There are indicators which are used as part of the outcomes assessment in Mexico:

• Outcome-oriented or strategic indicators: effectiveness
• Management or process indicators: efficiency.
• Cost indicators: economy.
• Quality indicators.
• Citizen–user’s satisfaction indicators.
• Indicator on the actors’ competence.

4. Mr. Rolf Elm-Larsen, Director, SAI of Denmark

SAIs use of progress measurement indicators

This presentation is about auditing of policy processes, implementing government programs, feedback on the implementing the programs.

Mr. Rolf Elm-Larsen in his presentation raised some key questions concerning SAI and KNI which should be discussed.

Mr. Rolf Elm-Larsen speaks about Key National Indicators and financial, compliance and performance audit.

The speaker gives some definitions of KNI:

• a indispensable part of the regulatory system
• it is basic tool for making corrective in implemting policies
• a precondition for making the the accountble responsible.

The speaker highlights that all standards on financial audit is focusing on the financial statement and Key National indicators are not a part of the financial statement.

• KNI is a precondition for fulfilling the purpose of public sector audit;
• KNI don’t fit with into the normal conceptual framework of financial audit;
• KNI is highly important for performance audit.

Questions which a SAI has to raise:

• Do the government has KNI for all materiel programs?
• Has government defined KNI so the measurement is valid, reliable and relevant?
• Do the government use KNI as a corrective mechanism for improving the implementation of the program?

There is no KNI system in Denmark but there is a good statistic infrastructure to develop indicators when necessary. To develop KNI is not a government task that’s why it’s difficult to make KNI accountable.

There are some final remarks of the presentation:

• KNI is a useful tool in performance audit of efficiency and impact of government programs;
• Government by reflection say it is too difficult to produce KNI;
• Legalistic bureaucrats assert that only if the KNI is required by law the government has to produce KNI even though it is good governance;
• Because it is postponing the responsibility and accountability.

5. Mr. Piskunov, auditor of the Accounts Chamber of the Russian Federation, Head of the Secretariat of the INTOSAI Working Group on KNI.

About approaches to progress measurement with use of KNI.

Mr. Piskunov says about the importance of the audit of national resources use. Solution of this task means presence of a measuring system including indicators. There are some international projects. Basic principal is taking into account all countries rights on choice of goals and ways of development. Not only potentials and processes of Socio-Economic development should be audited but the system of development. The task of SAI is to assist Government in harmonization relations between business and society. Growing disbalance and negative tendencies of world economic processes testifies about the necessity of coordination of goals and development strategies. KNI should include indicators of threats an risks. Coordination of development scenarios needs transparency of national metrics. There is a problem of total evaluation and visualization of performance audit outcomes by KNI. One of the goals of our project is a reference model development.

6. Mr. Mindaugas Macijauskas, Deputy Director, National Audit Office of Lithuania.

System of indicators used to evaluate performance and progress of NAO of Lithuania.

In the beginning of the presentation Mr. Mindaugas Macijauskas raises a question: Why do we need evaluation?

• To know where we are
• To measure the progress
• Show the public outputs achieved using budgetary money

Evaluation is a method to give an information about how budget funds are expended. The goal of audit is accountability. The program of departments audit is prepared every year: compliance and performance audit, Strategic operational Plan 2008 -2010, Public audit strategy 2008-2020. There is a system of indicators and criteria. It is necessary to gather information for each indicator before it development.

7. Mrs. Gulmira Jumagulova, Head of main analytical department of SAI of Kyrgyzstan.

Key National Indicators of Kyrgyzstan (2002-2008)

Mrs. Gulmira Jumagulova tells that Kyrgyzstan has 173 indicators including 6 key indicators. Also she characterized the situation in Kyrgyzstan from 2002 to 2008.

8. Mr. Jon Hall, Chief of the OECD Global project on measuring the progress of societies.

The Global Project on Measuring the Progress of Societies

Mr. Jon Hall says about the necessity of progress measurement and gives information about the Global project:

• The Global Project on aims to foster the development of sets of key economic, social and environmental indicators to provide a comprehensive picture of how the well-being of a society is evolving and seeks to encourage each society to consider in an informed way the crucial question: is life getting better?
• Brings together the large and growing movement that seeks to understand and measure progress.
• The Global Project is the first systematic global effort to go “beyond GDP” by enabling and promoting new ways to measure societal progress.

Partners: WB, UNDP, UNICEF, IADB, AfDB, EC, INTOSAI, UNESCWA, UNESCAP

In the framework of OECD and INTOSAI cooperation INTOSAI is invited to The 3rd OECD World Forum “Charting Progress, Building Visions, Improving Life” (27-30 October 2009 in Busan, South Korea) in order to organise a special workshop for the Working group on KNI.

The Global Project wants to assist INTOSAI and working group members that want to develop KNIs

• Run training courses for KNI projects
• Advise on how to run a national roundtable to select KNIs
• Apply the Quality Framework for Sets of Indicators to new initiatives
• Suggest new ICT tools to communicate KNIs to a wide audience
• Willing to consider other suggestions

9. Final conclusion.

We can see the growing actuality of the theme of KNI System development. It is evident that not all countries have the System of KNI but all countries have their own indicators and a good statistical data base for development the System of KNI taking into account national features.