Evaluation of transport policy implementation in Krakow (Poland)

first published 09/10/2017

As part of its Transport Policy 2015-2020, Krakow developed a transport policy evaluation mechanism, the Conformity Assessment Procedure. It gathers evaluation information to be used throughout the entire SUMP process, and using it involves little financial outlay.

Developed as part of the CH4LLENGE project (an EU-funded SUMP project), it is the city’s first evaluation tool. The procedure focuses on policy implementation in nine thematic areas, with external experts carrying out many of the evaluations. The tool will be integrated into the upcoming monitoring and evaluation (M&E) plan for Krakow's SUMP.

Context 

Krakow was the first Polish city to adopt a sustainable transport policy in 1993. The city's current transport plan is based around the "Transport Policy Krakow 2015-2025", which acts as the framework for a comprehensive master plan incorporating the city's overall concept of public transport, project management, and road and public transport infrastructure investment. Krakow was a partner city in the Ch4llenge project, and the Transport Policy was devised using the SUMP approach.

However, it does not yet include specific sets of measures, nor a comprehensive M&E plan. These will form part of the SUMP currently being developed, whose approval is expected in June 2018. The transport policy evaluation mechanism described below will be part of Krakow’s SUMP, and can be an example for cities with little or no experience in evaluating mobility policy.

In action 

Nine topics are evaluated, ranging from spatial planning and public transport to environmental protection and communication with residents. Each topic has between 5 and 15 indicators. These were selected by the municipality’s Urban Infrastructure Office and an advisory board consisting of professors from technical universities. Some of the indicators include:

• Maintaining a comfortable density of pedestrian crossings, ensuring adequate wide sidewalks and crosswalks;
• Attracting private capital into public investment;
• Informing citizens of planned transport investments by creating information platforms.

External experts mostly conduct the evaluations, whilst some are carried out by municipal policy officers. The experts' areas of knowledge are related to the indicators and cover topics such as urban planning, architecture, environmental science, and communications. The private sector is also called upon, with people from planning institutes and design companies used.

The experts receive documentation and detailed presentations on what has been done in the 9 different topics over the past year(s). They then conduct an independent assessment of the indicators. Firstly, the implementation of all policy topics is rated as low, average, or high. Then, the implementation of individual indicators is measured on a scale between 0-3.

The cumulative value of the ratings relating to the implementation of the indicators is compared to the potential maximum value, leading to a percentage that reflects the realised implementation. The cumulative value of all 9 policy topics determines the level of implementation of the overall transport policy.

Drawing on external perspectives provides the municipality with a broader view of its work. Outcomes of the evaluations are incorporated into transport policy development.

Results 

The Ch4llenge project enabled Krakow to implement a transport policy evaluation mechanism for the first time. The Conformity Assessment Procedure acts as the starting point for a more comprehensive M&E strategy that will form part of the upcoming SUMP. A simple approach, it can also form the basis of more in-depth evaluation studies. In the current Transport Policy, evaluation is foreseen in 2020 and 2025. However, the SUMP M&E plan would propose to make it an annual activity, and to streamline the process better.

Since 2013, the Conformity Assessment Procedure has been undertaken twice. The many external evaluators have helped to ensure more balanced and unbiased policy evaluation. The trials showed that scores given by external experts were generally lower than those from internal municipal policy officers. The outcomes were presented to the City Council and have proven useful for strategic planning and policymaking, whilst also facilitating the SUMP development process.

Challenges, opportunities and transferability 

Linking the degree of implementation to high-level SUMP goals is challenging. Although the weightings assigned to the indicators provide some information on how they contribute to different aspects of urban development, there is no direct connection with factors like CO2 or congestion reduction. The M&E plan should take such things into account, with further impact evaluation measures also required.

Although it remains a work in progress, Mr Lukasz Szewczyk, Director of The Municipal Services Department of Krakow Municipality, states that “this new evaluation method is a big step forward, granting us more insight into our achievements in sustainable urban mobility. When the new SUMP and its Monitoring and Evaluation Plan is approved, we will be ready for the future.”

In Depth 

• To read the CH4LLENGE Monitoring & Evaluation kit, click here:
• To read the CH4LLENGE local monitoring and evaluation programmes, click here.
• To read an introduction to the Transport Policy Krakow, click here (in Polish).
• To download the whole policy document, click here.

For more information, contact Mr Tomasz Zwolinski -Tomasz [dot] Zwolinskium [dot] krakow [dot] pl (Tomasz[dot]Zwolinski[at]um[dot]krakow[dot]pl).

Image credit: © ELTIS/Harry Schiffer