Stage 4: Commission and Assessment

Strategic Actions associated with this stage. This step signifies the hand over point from the innovation provider to the Agency. This step aims to optimise the overall performance of the innovation through a period of calibration and fine tuning. This stage includes the overall innovation’s commissioning, its calibration and fine tuning and subsequent assessment in accordance with the predefined assessment framework. Through this stage it is unlikely that a pilot will be terminated however unforeseen results may become apparent with any pilot. The business case should be reviewed to ensure that assumptions made are still valid. The continuous monitoring and assessment of the pilot is an integral element of this stage and any adverse occurrences should be rapidly identified. Response to such situations, particularly those in connection with safety and the environments, must be swiftly implemented and appropriate consideration given to the termination or modification of the pilot. Typical outputs that might be expected from this stage. Stage 3 Stage 2 Stage 5 Stage 5 Yes No

Step 4.3 - Assessment

NB This step must only commence following the successful completion of all steps within stage 3.

Once the scheme is open to traffic, the gathering and interpretation of monitoring data would initially concentrate on identifying any untoward adverse effects (especially risky situations or even additional accidents), in order that these can be corrected without delay, or if this is not possible, consideration can be given to abandoning the pilot. The emphasis would then shift progressively to establishing the various impacts of the pilot with increasing accuracy and levels of confidence as data accumulates over time.

A monitoring and evaluation report must be prepared summarising the various results from the assessment and drawing conclusions from the pilot. Several formats for the final report may be required to ensure that all the identifiable stakeholders receive the appropriate summary information. The report should provide information to assess pilot scheme impacts at three levels of timescale:

The latter two impacts will inform any subsequent applications of the technique and will establish the various impacts of the pilot with increasing accuracy and levels of confidence as data accumulate over time. It should be noted that different parties may be responsible for the different assessment levels.

While the evaluation experience from other pilots has highlighted the necessity of managing expectation, there is a need to ensure that when early results and reports are published appropriate caveats are included to ensure more robust results established over the longer term are not undermined.

Contingency plans should be developed that consider alternative strategies to be invoked should there be any untoward risk to the pilot users, for example, a serious accident or obvious adverse impact. In conjunction with this there should be continual monitoring of press reports for any publicity which may adversely affect the pilot.