Stage 2: Innovation and Assessment Requirements
Step 2.2.2 - Define assessment framework
Having determined the potential impacts and the range of indicators to assess them consideration must be given to how these impacts are to be quantified and measured. It is possible at this stage that the likely impact of some indicators will be minimal particularly in comparison to the size of the pilot being undertaken. Judgement will be required to assess the risk of eliminating the quantification of some of these potential indicators.
Consideration should also be given here to the potential of combining the measurement of two or more indicators. It is essential at this stage that all the assumptions made in the assessment are recorded. These will prove particularly useful should the completion of the pilot lead to the development and adoption of new standards.
A document to determine the methodology to be adopted to quantify the impacts must be produced. This is a particularly important document which must be updated if new indicators are identified or the overall methodology is subsequently modified.
The design of the methodology must enable a truly independent assessment of the innovation to be determined. To avoid bias, an independent party may be needed to carry out the assessment. It is possible that the requirements for the innovation itself may provide an opportunity, with only minimal changes, for collecting and presenting essential assessment data and information. Close liaison between the assessment stream and the innovation requirements stream is essential at this stage.
A guide and a comprehensive checklist issued by the European Commission in 1998 provides detailed guidance on this [EC 1998B]. It provides a detailed guide and checklist for assessment of transport telematics applications.
The Assessment Strategy developed for the Active Traffic Management Pilot [HA/TRL 2004] and the M1 HOV [TRL 2005B] provides a good example of the considerations that should be made in the development of a pilot’s assessment framework.
An important aspect to consider is to ensure that sufficient representative ‘before data’ is collected. Whilst data collection cannot commence until the pilot has been reviewed and progressed to Stage 3 - Implementation it is strongly recommended that preparation for before data collection is started at the earliest opportunity.