Account Chamber of the Russian Federation

MINUTES OF THE 2nd MEETING OF THE WORKING GROUP

April 21 – 22, 2009

Chaired by The Chair of the Working Group, Head of the Russian Federation Accounts Chamber Sergey Stepashin, D.Sc.

 

Present:

China	Mr. Liu Jiayi Mr. Dong Dasheng Mr. Peng Huazhang Mr. Zhou Weipei Mr. Peng Xinlin Mr. Luo Quan
Denmark	Mr. Rolf ELM-LARSEN
Finland	Mr. Teemu KALIJJÄRVI Mr. Ville VENKASALO
Indonesia	Dr. Daeng M. NAZIER
Kazakhstan	Mr. Altay ZEINELGABDIN Mr.Rollan SERIKBAYEV
Hungary	Dr. Gusztav Bage
Latvia	Ms. Liene RUGAJA
Lithuania	Mr. Mindaugas MACIJAUSKAS
Mexico	Mr. Jorge Pedro Castolo Domínguez, P.A. Mr. Mario Zhao
Pakistan	Mr. Imran IQBAL
Poland	Mr. Wojciech MISIĄG
Russian Federation	Mr. Sergey STEPASHIN Mr. Alexander PISKUNOV Ms. Tatiana YARYGINA Ms. Maria PISKUNOVA Ms.Yulia POROSKOVA Mr. Vladimir KULESHOV Mr. Valery KHRIPUN
Slovakia	Mr. Juraj KOLAROVIČ
Ukraine	Mr. Valentyn SYMONENKO Mr. Taras PRYTULA
USA	Ms. Bernice STEINHARDT
OBSERVERS
Kyrgyzstan	Ms. Gulmira JUMAGULOVA
OECD	Mr. Jon HALL

  

Welcome remarks:

Sergey Stepashin, Chairman of the Accounts Chamber of the Russian Federation, Chairman of the INTOSAI Working Group on Key National Indicators.

Mr. Stepashin greeted the participants of the meeting and spoke about great work which had been done by the Working Group since the first meeting. First, he mentioned the signing of the Memorandum of Understanding between OECD and INTOSAI and expressed approval of the Working Group activity report by the Governing Board (November 2008, Vienna). Mr. Stepashin spoke about items which had been planned to be discussed at the meeting and wished a fruitful work.

Liu Jiayi, Auditor General of National Audit Office of People ́s Republic of China.

Mr. Liu Jiayi greeted the participants of the meeting and spoke about the importance of the Working Group activity in the field of knowledge sharing. CNAO shares its experience with foreign colleagues and learns some better practices from other SAIs. The research in the field of KNI shall lead to greater success. Mr. Liu Jiayi brought up the issue of the world financial crisis and the necessity of KNI development. The speaker also highlighted the indispensable role of auditing supervision. Mr. Liu Jiayi wished the meeting great success.

Presentation of a new member.

Presentation of Daeng Nazier, Principal Director – Planning, Evaluation, Research, SAI of Indonesia.

Mr. Daeng Nazier spoke about the history, Mandates & Position and Organization Structure of the Audit Board. The speaker enumerated the Audit Board international activities. The important part of the presentation was about Indonesian KNI. There is a Long-term Development Plan (2005-2025) which consists of vision and mission, and direction-phase-priority and Medium-term Development Plan (2004-2009). The speaker spoke about the future activity of the Audit Board as well. Apart from that, Mr. Daeng Nazier expressed some expectations:

• Learning experiences from other countries in developing national indicators;
• Participating in the Working Group in developing and using key national indicators;
• Reviewing Indonesian national indicators by comparing or benchmarking them to others;
• Designing and implementing a pilot project to implement the guideline;
• Developing a framework or approach to audit government performance in achieving national indicators;
• Developing relevant audit guidelines.

Presentation of a new observer.

Presentation of G.Jumagulovа, Head of the Main Analytical Department of Accounts Chamber of the Kyrgyz Republic.

«Accounts Chamber as a Supreme Auditing Institution of the Kyrgyz Republic».

Ms. G.Jumagulovа in her presentation spoke about main objectives and tasks, and organizational structure of the Accounts Chamber of the Kyrgyz Republic. Ms. G.Jumagulovа defined the main objective of the Accounts Chamber as conduction of audits and performance audits. The Kyrgyz Republic formed the system of strategic management (Development Strategy of the country for 2007-2010) and program budgeting requirement control over the results of development. In this connection, main efforts of the Accounts Chamber of the Kyrgyz Republic in 2008 were focused on reforming audit activities. Accounts Chamber of the Kyrgyz Republic establishes and maintains international relations with government units of foreign states and their international associations. The Accounts Chamber is a member of INTOSAI, ASOSAI, ECOSAI and Council of Heads of the Supreme Auditing Institutions of the CIS; it also maintains mutual relations with Supreme Auditing Institutions of other countries on the bilateral basis.

Presentation of Dr. Sergey Stepashin, Chairman of the INTOSAI Working Group on Key National Indicators, Chairman of the Accounts Chamber of the Russian Federation.

«Report on the INTOSAI Working group on KNI activity (May 2008 – April 2009)»

In his report, Sergey STEPASHIN underlined the importance of Key National Indicators development and the use in the context of present world financial crisis. Mr. STEPASHIN pointed at the most important events in the activities of the INTOSAI Working Group on Key National Indicators in the last year. Among them he mentioned the increase of the Working group membership, initiation and coordination of the number of related sub-projects, the development of relations with the Organization of Economic Cooperation and Development, including the Memorandum of Understanding and Cooperation between the OECD and the INTOSAI on the issues of measuring the social progress and Working group participation in the Third International OECD Forum “Statistics, Knowledge, and Strategy”, in Bussan (Korea). Mr. STEPASHIN also focused on White Book of Recommendations on the development and use of Key National Indicators in the activities of the Supreme Bodies of Financial Control as a final document of the group activities and all parts of the Knowledge base that it should be based on. In conclusion, the Chair of the Working Group said that, in his opinion the 2nd Meeting of the Working group and the Seminar would be an important step in future work of the Working group and in the development of common approaches to the issue of Key National Indicators.

Presentation of Valentyn Symonenko, the Chairman of the Accounts Chamber of Ukraine.

«Usage of Key National Indicators by Supreme Audit Institutions while assessing government anticrisis measures»

Mr. Symonenko thanked Mr. Stepashin and Mr. Liu Jiayi for excellent organization of the meeting. Mr. Symonenko started his speech with the issue of crisis and said that the development of the methodology of KNI application is very timely. The speaker stressed that KNI should be used not only for evaluation of current situation but for evaluation of efficiency of anti-crisis measures undertaken by governments, for estimating how timely, adequate and efficient these measures are.

Mr. Symonenko pointed out what indicators are used in Ukraine and spoke about the necessity to see interaction of indicators.

But he noticed that budgetary resources deficit can raise a question about expediency and possibility of developing and implementation of KNI system. Also Mr. Symonenko said that limits and corridors for KNI values must be calculated. Conformity with them shows that government keeps the right path.

Presentations on realization of the subprojects.

Bernice Steinhardt, Director, Strategic issues, SAI of the USA

«Glossary on key national indicators»

In her report Ms. STEINHARDT underlined the coordination of the activities on Glossary development between INTOSAI and OECD and pointed that it will be a Guide, rather than a traditional Glossary. Ms. STEINHARDT also gave an overview of the Guide that consisted of 4 parts: “What is measured? What are indicators? What are indicator systems? and What is data quality” and gave full explanation about each item. At the end of the presentation Ms. STEINHARDT underlined that the Draft guide is available at the working group website and proposed that some next steps should be discussed:

• Draft guide posted on KNI WG Wiki
• Comments/additions by July 1, 2009
• What form will guide take?

1. Where to locate the guide

2. Discussion of the draft guide

3. Problem of translation

Gustav Bager, Senior advisor to the President of SAI of Hungary.

«The Pilot Project in the field of key national indicators describing the processes of knowledge-based economy and society»

Mr. BAGER started his report with conceptual outline of The Pilot project that included information about the aims of the pilot project, the participants of the project, the net step of the pilot project and the Implementation of the pilot project in 2009-2010. Then Mr. BAGER focused on the conceptual framework of knowledge-based economy and society (KES) and gave some background information on the main focus areas of the KES and basic characteristic features of a narrower concept of KES.

The speaker pointed out next steps of the pilot project:

• Establishing partnerships between the country SAIs and the international organisations • Finalisation of the conceptual outline of the pilot project. Implementation of the pilot project in 2009-2010.

Mr. BAGER also made a brief summary of the activities of such international organisations as OECD, the World Bank, the United Nations and the European Union. In conclusion, Mr. BAGER highlighted the role of Hungary in the light of the European Innovation Scoreboard indicators.

Liene Rugaja, Senior Desk Officer of the Audit, Methodology, Analyses and Development Department of SAI of Latvia.

«Development and use of key national indicators – review of international experience»

At the beginning of the report, Ms. Rugaja focused on the review of international experience. Ms. Rugaja told the audience about 46 country responses and publicly available information that the review was based on, and pointed out the main questions addressed by the survey. As for the survey results, Ms. Rugaja highlighted general issues of planning and performance measurement and the practices of national indicators development and use in different countries. At the end of the repot Ms. Rugaja presented general conclusions of the Survey:

•   Concepts of national indicators and system of KNI varies from country to country
•   Policy planning and performance measurement depends on administrative and political structure
•   Small countries with centralized governments tend to have simpler planning systems
•   Complex, overlapping systems, often due to various international commitments, requiring national strategies

Presentation of Alexander Piskunov, Auditor of the Accounts Chamber of the Russian Federation.

«Regional CIS Key National Indicators pilot project»

At the beginning of the presentation Mr. Piskunov pointed out main reasons for setting the issue of developing and implementing the key national indicators in the CIS countries as a separate pilot project and creation of the Expert Group on the Key National Indicators. Mr. Piskunov said that the first meeting of the Expert Group was held in Moscow (Russia) at the end of 2008. Also at the beginning of the current year the Secretariat held a number of working meetings with the representatives of the Supreme Bodies of Financial Control of CIS. The meetings confirmed the relevancy of studying the KNI issue in the CIS and the interest of the financial control bodies in the joint work on devising standards for the control-and-analytical activities. The works already conducted by the Expert group and planned for the years 2009-2010 should allow to achieve the following results:

• To devise a frame logical model;
• To create conditions for the network communication (via the web-site);
• To conduct a number of seminars and trainings for the Supreme Bodies of Financial Control representatives.

Mr. Piskunov dwelled upon future plans of the Expert Group to report to the Board of Chief Officers of the Supreme Bodies of Financial Control of the CIS countries on the results on the activities of the Expert Group including the Draught Methodological Recommendations on organizing and conducting the performance control (audit) and application of the Key National Indicators.

Tatiana Yarygina, Deputy Director of the State Research Institute of System Analysis.

Working plan 2009 and organizational activities.

SAI of Latvia proposed to define coordinators for Plan item concerning “Preparation of Chairman report on the Working Group activity to the INTOSAI Governing board. Providing materials on subprojects” and to define the date of subprojects realization. Working plan 2009 was approved with amendments of SAI of Latvia.

Coordinators of subprojects were asked to elaborate detailed activity plans and to submit information on progress of the subprojects by September 10.

All WG members were invited to comment and contribute to development of the on-line knowledge base by adding information to the WG website.

Ms.Yarygina spoke about the structure of the Working Group knowledge base.

Presentation of Altay Zeinelgabdin, Member of SAI of Kazakhstan

Mr. Zeinelgabdin declared that the Accounts Committee is the supreme body of the state financial control, carrying out external control over execution of republican budget, directly subordinate and accountable to the President of the Republic of Kazakhstan. Mr. Zeinelgabdin mentioned that in accordance with the Constitution of the Republic of Kazakhstan the President appoints two members for the term of 5 years and each of the Chambers of the Parliament of the Republic of Kazakhstan (Senate and Mazhilis) appoints independently three members of the Accounts Committee for control over execution of the republican budget.

Mr. Zeinelgabdin also highlighted the main functions of the Accounts Committee and its international activity.

Besides this Mr. Zeinelgabdin underlined that within the framework of the further effective development of the state financial control system and conducting of measures on minimization of the financial violations in 2007 there was established the Republican state enterprise “Centre for research of financial violations” under the Accounts Committee. At the end of the presentation Mr. Zeinelgabdin proposed to organize the 3rd meeting of the INTOSAI Working group on KNI in Astana, the capital of the Republic of Kazakhstan. The 3rd Meeting will be held in the period of March-May April 2010г.

Conclusion remarks

Dr. Sergey Stepashin, Chairman of the INTOSAI Working Group on Key National Indicators, Chairman of the Accounts Chamber of the Russian Federation.

Summing up the results Dr. STEPASHIN pointed out two items: Firstly, the 2nd meeting of the Working group demonstrated not only the active participation of Working group members, but also a potential of their cooperation; Secondly, the discussion over the documents, mentioned in Agenda shows that there is much work to be done in future.

Taking into account these conclusions Dr. STEPASHIN asked all project coordinators and the Secretariat to prepare all final documents of the 2nd Meeting in two upcoming weeks and to distribute them among all Working group members.

Dr. STEPASHIN also underlined that in November, 2009 it would be necessary to present the first results of the Working group activity to the Governing board of INTOSAI.

At the end of the speech Dr. STEPASHIN thanked all participants of the meeting, and especially colleagues from China for excellent organization of the 2nd Meeting of INTOSAI Working group on KNI.

 

April 22, 2009

General remarks of the Seminar.

The seminar “Conceptual approaches to the development and use of Key National Indicators System” was organized during the Second meeting of the INTOSAI Working Group on Key National Indicators (KNI). Main questions of the seminar were:

1. Рrogress definition and measurement;

2. Рrogress measurement in the activities of Supreme Audit Institutions (SAI), regional and international organizations;

3. Methodological approaches to use of progress measurement indicators by SAIs;

4. Use of the KNI system in the conditions of global financial and economic crisis.

The aim of the seminar was to promote dialogue, discussion and strategic proposals on how to reinforce the use of KNI in SAIs activity.

1. Mr. Peng Xinlin, Deputy Director of Law Department of the NAO of China. “Relevant measurement indicators for scientific development of China as well as its application in auditing”.

Mr. Peng Xinlin in his presentation pointed out that 9 main targets of economic and social development and 22 main quantitative indicators have been included in the 11th Five-Year Plan (2006-2010).

Indicators are divided into 14 anticipated and 8 obligatory.

Also there are special-purpose development outlines by Ministries based on 11th Five-Year Plan. They nail down specific development goals and put forward more concrete and detailed measuring indicators.

Mr. Peng Xinlin highlights the evolution of government auditing. Now auditors not only check but also examine performance of government as well as financial revenues and expenditures. Society and eco-system are now not less important than economic benefits.

Institutions in China collect measuring indicators from different sources. He emphasizes important tasks for Auditors:

a) Contrast and analysis between economic and environmental indicators.

b) Prevention and control of water pollution.

c) Environmental control, protection and improvement.

2. Mr.Teemu Kalijärv, Principal performance auditor of NAO of Finland.

Mr. Ville Vehkasalo, Principal performance auditor of NAO of Finland.

”What do key national indicators indicate and how should we audit them?”

The speaker underlines interrelation of all factors. It is impossible to define Government activity success in any sphere by one indicator. It is necessary to take into account interrelations of all factors. Finland doesn’t have the System of KNI. But Finland has a developed statistics system that’s why there is a background for development the System of KNI. Not all statistical data reflect the condition, for example GDP, that’s why there are non-official indicators. The growth of GDP doesn’t mean prosperity. SAI of Finland doesn’t actively participate in indicators development, because active participation in it can influence its independence and objectivity. SAIs could use existing indicators in audit but it’s necessary to be sure in their reliability. That’s why SAIs shouldn’t develop indicators but they should control them. Key National Indicators should be based on national features.

Concluding remarks of the presentation

• Reliable indicators (= statistics) are always needed

• ‘Key national indicators’ should be based on national circumstances

• International comparisons are difficult and possibly biased

• SAIs should audit whether indicators are in fact valid and reliable

• SAIs could also utilize indicators when analysing policy effectiveness

3. Mr. Jorge Pedro Castolo Dominguez, General Director, Performance Audit Office on Economic Development of Mexico.

Experience and progress in Mexico.

Mr. Jorge Pedro Castolo Dominguez pointed out main regulatory documents on global and institutional levels.

The speaker specified that public Management has evolved, through the last decades, from a traditional bureaucratic model to an outcome-oriented model.

The new model focuses on the impacts of public actions implemented for the benefit of society.

The most important aspect to consider is the assessment of the compliance of goals, outcome, performance indicators and benchmarking standards.

There are indicators which are used as part of the outcome assessment in Mexico:

•   Outcome-oriented or strategic indicators: effectiveness.
•   Management or process indicators: efficiency.
•   Cost indicators: economy.
•   Quality indicators.
•   Citizen–users’ satisfaction indicators.
•   Indicator on the actors’ competence.

4. Mr. Rolf Elm-Larsen, Director, SAI of Denmark

SAIs use of progress measurement indicators

This presentation is about auditing of policy processes, implementing government programs, feedback on the implementation of the programs.

Mr. Rolf Elm-Larsen in his presentation raised some key questions concerning SAI and KNI which should be discussed.

Mr. Rolf Elm-Larsen spoke about Key National Indicators and financial, compliance and performance audit.

The speaker gave some definitions of KNI:

•   a indispensable part of the regulatory system
•   it is a basic tool for editing in implemented policies
•   a precondition for making the accountble responsible.

The speaker highlighted that all standards on financial audit is focused on the financial statement and Key National indicators are not a part of the financial statement.

• KNI is a precondition for fulfilling the purpose of public sector audit;

• KNI do not fit into the normal conceptual framework of financial audit;

• KNI is highly important for performance audit.

Questions which a SAI has to raise:

• Does the government have KNI for all materiel programs?
• Has government defined KNI so the measurement is valid, reliable and relevant?
• Does the government use KNI as a corrective mechanism for improving the implementation of the program?

There is no KNI system in Denmark but there is a good statistical infrastructure to develop indicators when necessary. To develop KNI is not a government task that’s why it’s difficult to make KNI accountable.

There are some final remarks of the presentation:

• KNI is a useful tool in performance audit of efficiency and impact of government programs;

• Government by reflection says it is too difficult to produce KNI;

• Legalistic bureaucrats assert that only if the KNI is required by law the government has to produce KNI even though it is good governance, because it is postponing the responsibility and accountability.

5. Mr. Piskunov, auditor of the Accounts Chamber of the Russian Federation, Head of the Secretariat of the INTOSAI Working Group on KNI.

About approaches to progress measurement with use of KNI.

Mr. Piskunov spoke about the importance of the use of national resources audit. Solution of this task means presence of a measuring system including indicators. There are some international projects. The basic principle lies within taking into account all countries’ rights on choice of goals and ways of development. Not only potentials and processes of Socio-Economic development should be audited but the system of development. The task of SAI is to assist Government in harmonization relations between business and society. Growing disbalance and negative tendencies of world economic processes testifies the necessity of coordination of goals and development strategies. KNI should include indicators of threats and risks. Coordination of development scenarios needs transparency of national metrics. There is a problem of total evaluation and visualization of performance audit outcomes by KNI. One of the goals of our project is a reference model development.

6. Mr. Mindaugas Macijauskas, Deputy Director, National Audit Office of Lithuania.

System of indicators used to evaluate performance and progress of NAO of Lithuania

At the beginning of the presentation Mr. Mindaugas Macijauskas raised a question: Why do we need evaluation?

• To know where we are
• To measure the progress
• Show the public outputs achieved using budgetary money

Evaluation is a method to give information about how budget funds are expended. The goal of audit is accountability. The program of departments audit is prepared every year: compliance and performance audit, Strategic operational Plan 2008 -2010, Public audit strategy 2008-2020. There is a system of indicators and criteria. It is necessary to gather information for each indicator before its development.

7. Ms. Gulmira Jumagulova, Head of main analytical department of SAI of Kyrgyzstan.

Key National Indicators of Kyrgyzstan (2002-2008) Ms. Gulmira Jumagulova stated that Kyrgyzstan has 173 indicators including 6 key indicators. Also she characterized the situation in Kyrgyzstan from 2002 to 2008.

8. Mr. Jon Hall, Chief of the OECD Global project on measuring the progress of societies.

The Global Project on Measuring the Progress of Societies

Mr. Jon Hall spoke about the necessity of progress measurement and gave information about the Global project:

• The Global Project aims at fostering the development of sets of key economic, social and environmental indicators to provide a comprehensive picture of how the well-being of a society is evolving and seeks to encourage each society to consider in an informed way the crucial question: is the life getting better?

• Brings together the large and growing movement that seeks to understand and measure progress.

• The Global Project is the first systematic global effort to go “beyond GDP” by enabling and promoting new ways to measure societal progress.

Partners: WB, UNDP, UNICEF, IADB, AFDB, EC, INTOSAI, UNESCWA, UNESCAP

In the framework of OECD and INTOSAI cooperation INTOSAI has been invited to The 3rd OECD World Forum “Charting Progress, Building Visions, Improving Life” (27-30 October 2009 in Busan, South Korea) in order to organise a special workshop for the Working group on KNI.

The Global Project wants to assist INTOSAI and working group members who want to develop KNIs

• Run training courses for KNI projects
• Advise on how to run a national roundtable to select KNIs
• Apply the Quality Framework for Sets of Indicators to new initiatives
• Suggest new ICT tools to communicate KNIs to a wide audience • Willing to consider other suggestions

9. Final conclusion.

There is the growing actuality of the theme of KNI System development. It is evident that not all countries have the System of KNI but all countries have their own indicators and a good statistical database for development the System of KNI, taking national features into account.