METHODOLOGY

The TPPR Methodology is envisioned to be a comprehensive methodology for assessing public procurement legislations (PPLs) with the ultimate goal of identifying the strengths and weaknesses of legal frameworks and their enforcement around the globe. Experts assess the legislations once in two years, whereas the implementation of the law is assessed every year.

Structure and Logic

The Methodology is composed of 64 indicators, each of which holds a similar degree of importance. A number of these indicators are further broken down into scoring components. 

The Methodology covers all the major components of any public procurement system, from the nature of the legislation to the complaint review process, with focus on the transparency of public procurement systems.

The selection of indicators for the Methodology was largely based on international best practice, international standards and aspects of other existing methodologies in the sphere of public procurement, such as:

Several indicators have been directly taken from one of the sources listed above as examples of best international practice. All of these indicators have been properly referenced.

During the indicator selection and elaboration process, an effort was made to ensure that the Methodology can be used to assess many different kinds of public procurement systems, but at the same time set high standards.

The indicators are separated into 5 groups (benchmark indicators) that represent the key characteristics (values) of a well-functioning, transparent and accountable public procurement system:

  1. Uniformity of the Legislative Framework – 14 indicators
  2. Efficiency – 10 indicators
  3. Transparency – 18 indicators
  4. Accountability and Integrity – 7 indicators
  5. Competitiveness and Impartiality – 10 indicators

The methodology also includes 5 indicators that are used to assess legal components that are not directly part of the public procurement legislation but are crucial in terms of creating a transparent environment necessary for a proper functioning of any public procurement system. These indicators are grouped separately under ‘Transparency Environment’.

Indicators are also arranged according to the procurement process:

  1. Pre-tendering Phase – procurement processes leading up to the publication of a notice of intended procurement.
  2. Tendering Phase – procurement processes between publication of a notice of intended procurement and selection of a tender winner.
  3. Post-tendering Phase – procurement processes after the selection of a tender winner.

These two arrangements allow for both process and value-based assessments of public procurement legislations.

Limitations

Public procurement systems vary significantly by country. The TPPR Methodology is intended to be applicable on a global scale, meaning that the indicators cannot be too specific and cannot cover all the possible variations and exceptions.

For the same reason, the TPPR Methodology can only be used to assess public procurement legislations on the national level, and it does not include indicators for any industry specific rules.

Scoring System

Each indicator included in the TPPR Methodology is granted equal weight and receives a maximum of 1 point. With a total of 64 indicators, public procurement legislations are rated on the scale of 0 to 64 (converted to percentages for easier understanding and visualization).

Indicators that are further broken down into scoring components are nevertheless worth 1 point. Each scoring component is evaluated separately.

The Methodology uses two ways to distribute points among scoring components of an indicator:

  1. ‘Scoring method’ is used when scoring components of an indicator overlap (meaning they do not add up as points) or have an unequal distribution of points.
  1. ‘Point distribution’ is used when each scoring component of an indicator is given an equal share of the overall point granted to that indicator.

This scoring system and the structure of the Methodology allow for the results to be presented in three ways:

  1. Country Overall Results and Rating – country overall results on the scale of 0-64 are converted into and presented using percentages (0-100%). Countries are then ranked by their overall results.
  1. Results by Benchmark Indicators – results for benchmark indicators are presented only through percentages, since benchmark indicators include an unequal number of indicators (and, therefore, unequal number of maximum points).
  1. Results by Procurement Process – results for different phases of the procurement process are presented only through percentages, due to unequal number of indicators included in each phase.

Countries are also ranked and compared by their results for each benchmark indicator.

Full Document: