BECA IQA Att 1 to Tony Randle.pdf

Attachment 1

Report
Bus Rapid Transit Indicative Business Case
Independent Quality Assurance Review 2015
Prepared for the NZ Transport Agency
Prepared by Beca Ltd (Beca)
2 July 2015

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Revision History
Revision Nº
Prepared By
Description
Date
1
Graham Spargo and

2 July 2015
Alan Kerr
2

3

4

5

Document Acceptance
Action
Name
Signed
Date
Prepared by
Graham Spargo

Reviewed by
Alan Kerr

Approved by
Stephen Hewett

on behalf of
Beca Ltd

The statements and opinions expressed in this Independent Quality Assurance Review are made in good faith, and on the basis that all
information relied upon is accurate in all material respects, and not misleading by reason of omission or otherwise.
The statements and opinions expressed are based on information available as at the date of this Independent Quality Assurance
review. Beca reserve the right, but is under no obligation, to review or amend our IQA review, in the event that any additional
information, which was in existence on the date of this review, was not brought to our attention, or subsequently comes to light.
© Beca 2015 (unless Beca has expressly agreed otherwise with the Client in writing).
This report has been prepared by Beca on the specific instructions of our Client. It is solely for our Client’s use for the purpose for which
it is intended in accordance with the agreed scope of work. Any use or reliance by any person contrary to the above, to which Beca has
not given its prior written consent, is at that person's own risk.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // i

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Contents
1  Overview
1
2  Bus Rapid Transit Independent Quality Assurance Review Findings
2015
5
2.1  Completeness
5
2.2  Convincing
7
2.3  Consulted
9
2.4  Clear and concise
9
3  NZ Transport Agency IBC Guidance Questions
10

Appendices
Appendix A
Economic Evaluation Assessment

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // i

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
1  Overview
Our overall opinion is that sufficiency for decision–making purposes is achieved by the draft Indicative
Business Case (IBC) for Wellington’s Bus Rapid Transit (BRT) initiative dated 29 June 2015.
This Independent Quality Assurance (IQA) review is informed by both the NZ Treasury ‘better business case’
guidance and that of the NZ Transport Agency. The Agency framework is founded in the Treasury approach
with tailoring to meet Agency needs.
Both sources of business case guidance are relatively recent (within the last four years). As a consequence
the draft IBC has been developed in circumstances of Business Case practice being in transition in response
to the new guidance. This guidance continues to evolve. Our conclusions and comments take this dynamic
into account.
As business cases are a ‘thinking tool’ and a key input to decision-making, we take the view that from an IQA
perspective the focus should be on ensuring the principles, key concepts and processes employed are
sound in terms of business case development. Because of this, there wil  always be variability in approach
and process adopted. Our review has attempted to look past this to help determine whether the approach
taken tel s a clear story, and provides suitable evidence to arrive at a conclusion.
We note that in comparison with prior transportation evaluation processes that decision-makers and officials /
officers are very familiar with, the business case framework obliges a ‘re-calibration’ in terms expectations
and understandings of what detail and analysis an Indicative Business Case wil  cover. We commend to the
reader the explanation provided in the introductory sections of the draft IBC on the focus and scope of an
IBC (in particular please see from Page xv of the draft IBC).
The IQA itself is framed in terms of NZ Treasury and State Services Commission recommendations for
coverage. We also incorporate quality assurance considerations from the NZ Transport Agency on-line
guidance. The IQA structure has four dimensions:
  Completeness - Is al  the required information for an IBC of this type appropriately included?
  Convincing – Is the analytical framework that has been employed, and the level and type of analysis that
has been undertaken appropriate for an IBC?
  Consulted – in the overall context of the development of the Bus Rapid Transit initiative and current draft
IBC phase in particular, is the approach to consultation and engagement satisfactory?
  Clear and Concise - The final criterion relates to the presentation of material in the IBC.
Completeness
a.
Developing business cases requires judgements to be made as to relevant context and how this
shapes scope and coverage against business case ‘good practice’ guidance.  The Wellington Public
Transport Spine Study (PTSS) is very important context for the BRT IBC.  The partner organisations
have agreed it as the foundation stone upon which the further Business Case steps are to be built. A
relevant consideration is that the PTSS work goes to a level of detail beyond that which a Strategic or
Programme Business Case would normally canvass. As a consequence our position is that the
current IBC stage can and has appropriately drawn directly on significant elements of this prior
material without the need for additional transport modelling and some elements of evaluation.
Inclusion and further interpretation of this previous information, plus additional evaluative steps under
the current IBC process, mean that for Indicative Business Case decision-making purposes it is our
view that the information and evaluation base is sufficiently complete.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 1

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
b.
Further to the point above, a significant amount of transport modelling was undertaken for the PTSS.
This has been relied on to provide input to the draft IBC and an interpolation process has been used
for options that were not investigated as part of the PTSS. The aim of the Business Case process is to
use the right level of detail at the right time to enable a preferred option to be identified. Therefore,
given the comprehensiveness of work that has already been undertaken, we consider it is appropriate
to have used the existing model ing to reduce the number of options for consideration into the DBC
phase. It is anticipated that more modelling wil  be undertaken during the DBC phase using both meso
and micro scale tools.
c.
In terms of the form of the draft IBC (that is the layout and specific coverage or reference to particular
matters) the document is not fully aligned to NZ Transport Agency IBC guidance (see
http://hip.nzta.govt.nz/processes/project-development/indicative-business-case for detail).   From a
completeness perspective the draft IBC does however in our view satisfy general y accepted practice
under the NZ Treasury ‘Better Business Case’ guidance. Also, in terms of material considerations, it
does cover the main elements expected under the NZ Transport Agency guidance. The Agency
makes clear that its guidance templates are not to be slavish followed, however we consider it prudent
to check with senior NZTA Planning & Investment officials as to the level of comfort with the framing of
the draft BRT IBC as it stands from a completeness perspective.
Convincing
This criterion relates to the analytical framework that has been employed, and the level and type of analysis
that has been undertaken.
a.
Our opinion is that the draft IBC’s preferred options make sense from the evaluation work carried out
and have a sufficiently clear logic trail.
b.
The economic assessment has been undertaken using 2013 EEM procedure and is considered
appropriate. We identify some areas where further explanation or provision of a range in estimates
would be helpful, and the materiality arising from this. This can help provide confidence to decision-
makers and enable sensitivity to these changes across a range to be better understood. From
discussion with the IBC authors this has been considered and largely incorporated in  the draft IBC.
c.
The spreadsheet economic model is considered suitable and fit for purpose for the BRT IBC and as a
basis for further Business Case steps. Spreadsheet tables and linkages are logical and can be traced
through.  Appendix A to our IQA sets out matters of detail in relation to our evaluation of these
aspects.
d.
Our discussion with the IBC authors has included canvassing “walking benefits”. The EEM has been
followed; however we have encouraged exploring a range of values ascribed to this element to reflect
uncertainties and to help clarify what sensitivity there is to this factor. We understand and it is our
view that inclusion of a suggested range wil  not materially alter the overall conclusions, albeit it wil
marginal y reduce the overall Benefit / Cost Ratio, particularly for the higher cost options.
e.
For the commercial case, the draft IBC makes recommendations on procurement. A matter that we
would recommend for particular attention in the Detailed Business Case phase is whether the same
procurement options apply uniformly across the two preferred options. Some segments may warrant a
different approach, for example to address different risk profiles.
f.
From the draft IBC evaluation we note that we see scope for the decision-makers to explore the
potential for making a judgment to take only one option forward to DBC if they wish.  The draft IBC
satisfies the objective of providing decision-makers with an early indication of the preferred
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 2

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
investment, however in recommending taking forward two options to DBC it does increase the
investment in evaluation needed. This is, of course, offset by providing broader optionality at DBC
stage. The two recommended options identified are sufficiently different to warrant further multi-criteria
evaluation during the DBC (acknowledging that one of the drivers behind the decision wil  be political
considerations).
g.
The IQA scope included undertaking specific evaluations for the following matters:
  A review of the Multi Criteria Analysis and options evaluation. This included a check of:
a.  The process used to determine the criteria
b.  The chosen criteria
c.  The process used to evaluate the options
d.  The results of the options evaluation
e.  Check of the sensitivity analysis
We consider the MCA development and evaluation approach to be suitable for the IBC and it
provides a clear and logical process for alighting on the two preferred options.
  A review of the range of options identified to determine whether there is a sufficient range. This
exercise has relied heavily on the PTSS. The business case process would typically require the
development of a Programme Business Case (PBC) where a range of alternative programmes with
intervention options would be investigated. These would typically be wide ranging. The PTSS has,
in our view, been used in lieu of a PBC to determine the preferred programme, although some
components of the PBC have been incorporated within the IBC. We are comfortable with this
approach given the genesis of the PTSS work and its comprehensive nature, and we therefore
consider the range of options identified to be suitable.
  A review of the Cost Benefit calculations, including:
a.  Review the cost benefit profiles to determine whether there are any errors
b.  A cell by cel  review of the excel data model, checking for errors.
c.  The tools ability to be dynamic and ease of use in subsequent business casing steps.
No material errors have been identified.  We consider the tools ability to be used in subsequent
stages to be appropriate.
Consulted
a.
The NZ Treasury and NZTA guidance promotes the idea of an IBC not just stating what consultation
has been undertaken, but also explaining the nature of any issues raised or views expressed by
stakeholders, and how these have been taken into account in the development of the IBC.  This
aspect of the draft IBC is limited currently and may merit further attention in our view, especially given
the political nature of the project and the range of views held.
b.
The DBC phase encourages a broad engagement and consultation base and is a potential means of
addressing the limited consultation and engagement of the current IBC phase.  We think this would be
acceptable given the prior level of extensive interaction for the PTSS. A suitably comprehensive and
robust consultation strategy for the DBC stage wil  be needed.
c.
As a significant body of work and related engagement and consultation processes have occurred
relatively recently for the PTSS, we acknowledge there is a justifiable basis for having undertaken
targeted consultation for the IBC phase.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 3

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Clear and Concise
This criterion relates to the presentation of material in the IBC.
a.
The architecture of the report and layout of the draft IBC is clear.
b.
Good use has been made of various table formats.  Colour coding assists in distil ing out key
considerations in the MCA evaluation sections.
c.
To assist with understanding and clarity the description of the options would be assisted by the use of
cross section images, or concept sketches. We understand this is under consideration already by the
IBC authors.
d.
The draft IBC is suitably concise given the subject matter and required coverage.  Our suggested
amendments and revisions wil  not materially impact this in our view.
e.
The overal  IBC narrative is pitched appropriately in our opinion.  The ‘case for change’ usefully draws
out essential elements of the story from the body of the IBC.
Recommendations
In summary, following our review, we recommend the following:
1.  That the Wellington Bus Rapid Transit Project partners note the Independent Quality Assurance review
findings that there is a sufficient and appropriate basis provided by the draft Indicative Business Case to
support recommended Options 3 and 5 being taken forward to Detailed Business Case evaluation.
2.  That matters identified in the Independent Quality Assurance be actioned to strengthen the draft
Indicative Business Case, or where identified, given particular focus at the next stage of the Detailed
Business Case.
3.  In recognition of the transitional phase of new Business Case methodology being applied for projects
such as Bus Rapid Transit, that the Project partners consider having a targeted discussion to confirm a
common understanding and expectations as to scope and the level of technical analysis meeting
Indicative and Detailed Business Cases international best practice guidance.

The sections which follow expand upon the specific matters addressed in summary form in the preceding
sections.

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 4

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
2  Bus Rapid Transit Independent Quality Assurance
Review Findings 2015
Purpose of an Indicative Business Case:
“Confirm the way forward with short-listed options for further analysis”1
“The primary purpose of the indicative business case is to provide
decision-makers with an early indication of the preferred investment”2

Dimensions Assessed For The Indicative Business Case
2.1
Completeness
  Is the appropriate level of information for an Indicative Business Case included? (See NZ Treasury’s IBC guidance
and NZTA’s version)
  Are al  substantive elements of each option included to the level expected under IBC guidance?
  Have al  substantive economic, social and environmental impacts been identified (and quantified where feasible) in
accordance with IBC guidance?
  Is there satisfactory information to form a view on value for money of options for IBC purposes?
Reviewer’s opinion:
We are general y satisfied that the draft IBC has an appropriate level of information, providing sufficient detail on the
substantive elements of each option, and provides satisfactory information to form a view on ‘value for money’ for
Indicative Business Case decision-making purposes.
Insofar as identification and quantification of al  substantive economic, social and environmental impacts in our opinion
the multi-criteria assessment information satisfactorily covers this off in terms of ‘better business case’ guidance.  For NZ
Transport Agency guidance it is, however, debatable that al  aspects are achieved to a level of detail encouraged (for
example setting out of Environmental Scan considerations, consenting risk factors, and inclusion of a position on
strategic fit and effectiveness).  Given the previous Wel ington Public Transport Spine Study (PTSS) evaluation work
which did incorporate analysis of most of these matters, plus the ability to traverse these at Detailed Business Case
stage, it is our view that this matter of ‘completeness form’ is not material.
Other specific observations relevant to ‘completeness’ are:
a.
Developing business cases requires judgements as to relevant context and how this shapes scope and coverage
against business case ‘good practice’ guidance.  The PTSS is very important in this respect for the BRT IBC. The
BRT partner organisations have agreed the PTSS as the foundation stone upon which the further Business Case
steps are to be built.  A relevant consideration is that the PTSS work goes to a level of detail beyond what a
Strategic or Programme Business Case would normal y canvass.  This reflects the transition process underway

1  NZ Treasury Guidance http://www.treasury.govt.nz/statesector/investmentmanagement/plan/bbc/guidance
2 NZ Transport Agency Indicative Business Case guidance page 2 Indicative Business Case, Strategic Case Review &
Scoping 27 June 2013
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 5

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Dimensions Assessed For The Indicative Business Case
between historic and new evaluative regimes.  As a consequence our position is that the current IBC stage can
and has appropriately drawn directly on significant elements of this prior PTSS material without the need for
additional transport modeling to be undertaken and with reliance on some elements of PTSS evaluation. Inclusion
and further interpretation of the PTSS previous information, plus additional evaluative steps under the current IBC
process mean that for Indicative Business Case decision-making purposes it is our view that the information and
evaluation base is sufficiently complete.
b.
Further to the point above, a significant amount of transport modeling was undertaken for the PTSS. This has
been relied on to provide input to the IBC and an interpolation process has been used for options that were not
investigated as part of the PTSS. The aim of the Business Case process is to the use the right level of detail at
the right time to enable a preferred option to be identified. Therefore, given the comprehensiveness of work that
has already been undertaken, we consider it is appropriate to use the existing modeling to reduce the number of
options for consideration into the DBC phase. It is anticipated that more modeling wil  be undertaken during the
DBC phase using both meso and micro scale tools.
c.
In terms of the form of the draft IBC (that is the layout and specific coverage or reference to particular matters) the
document is not ful y aligned to NZ Transport Agency IBC guidance (see http://hip.nzta.govt.nz/processes/project-
development/indicative-business-case for detail).   From a completeness perspective the draft IBC does however
in our view satisfy generally accepted practice under the NZ Treasury ‘Better Business Case’ guidance.  Also, in
terms of material considerations, it does cover the main elements expected under the NZ Transport Agency
guidance.  The Agency makes clear that the templates are not to be slavish fol owed, however we consider it
prudent to check with senior NZTA Planning & Investment officials the level of comfort with the framing of the draft
IBC as it stands from a completeness perspective.
d.
The draft IBC assumes prior knowledge and uses the PTSS as a partial PBC (i.e. the PTSS makes the
recommendation that BRT is the preferred programme). Although we consider it appropriate that the IBC does not
re-litigate this, we suggest some additional brief preamble in the introductory section is warranted as to the key
reliance on the PTSS and in particular about the other programmes considered and why they were disregarded.
The bottom of page ii  or the top of page iv of the draft IBC could useful y include this information.
The NZTA IBC guidance (http://hip.nzta.govt.nz/processes/project-development/indicative-business-case) promotes
inclusion of elements that are minimal y referenced or absent in the draft IBC.  The Agency makes clear that the
templates are not to be slavish fol owed, however we consider it may be prudent to check with senior NZTA Planning &
Investment officials if they are comfortable with framing of the draft IBC as it stands.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 6

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Dimensions Assessed For The Indicative Business Case
2.2
Convincing
  Are the status quo, refined Problem Definition and any cited evidence presented in an accurate and balanced way?
  Does the evaluation response and options relate logical y to, and suitably address, the problem definition consistent
with IBC practice? (See NZ Treasury’s IBC guidance and NZTA’s version)
  Do the options offer a proportionate, wel -targeted response to the problem/s?
  Is the level and type of analysis provided commensurate with the size and complexity of the problem/s and the
magnitude of the impacts and risks of the options?
  Is the nature and robustness of the cited evidence commensurate with the size and complexity of the problem and
the magnitude of the impacts and risks of the options?
  Do the conclusions relate logical y and consistently to the IBC analysis of the options?
  Are risks, sensitivity analysis, and assessment profile suitably articulated?
Reviewer’s opinion:
The status quo / reference case, Problem Definition and evidence have in our opinion been presented in an accurate and
suitably balanced way. This is also the case for the evaluation response and the options developed relate logical y to,
and suitably address, the problem definition and benefits sought.
The options explored draw on the previous PTSS work and we consider this a reasonable and appropriate position to
have taken for IBC purposes.  The options explored offer a proportionate and suitably targeted response to the problems.
The level and nature of analysis and interpolation is in our view commensurate with the size and complexity of the
problems.  The interventions also respond appropriately in our view to the magnitude of the impacts and risks of the
options.
For IBC purposes it is our view that the nature and robustness of the cited evidence is commensurate with the size and
complexity of the problem and the magnitude of the impacts and risks of the options.  We are comfortable with the
drawing through of analysis and assessment from the PTSS process, and how it has been further interpreted and
augmented to shed additional light and understanding.
We consider that the conclusions reached relate logical y and consistently to the IBC analysis of the options.  The
consideration of risks, sensitivity analysis, and assessment profile are general y suitably articulated in keeping with
‘Better Business Case’ guidance.  In terms of NZ Transport Agency guidance the level of detail on matters such as
statutory process risks, or community engagement and consultation aspects is thin.  This can however be picked up at
the Detailed Business Case stage and we do not consider this material for the purposes of current IBC decision-making.
a.
The economic assessment has been undertaken using 2013 EEM procedure and is considered appropriate. We
identify some areas where further explanation or provision of a range in estimates would be helpful, and the
materiality arising from this.  This can help provide confidence to decision-makers and enable sensitivity to these
changes across a range to be better understood. From discussion with the IBC authors this has been considered
and the draft IBC modified to include this.
b.
The spreadsheet economic model is considered suitable and fit for purpose for the BRT IBC and as a basis for
further Business Case steps.  Spreadsheet tables and linkages are logical and can be traced through.
c.
Our discussion with the IBC authors has included canvassing “walking benefits”. The EEM has been fol owed;
however we have encouraged exploring a range of values ascribed to this element to reflect uncertainties and to
help clarify what sensitivity there is to this factor.  We understand and it is our view that inclusion of a suggested
range wil  not material y alter the overal  conclusions, albeit it wil  marginal y reduce the overal  Benefit / Cost
Ratio, particularly for the higher cost options.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 7

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Dimensions Assessed For The Indicative Business Case
d.
For the commercial case, the draft IBC makes recommendations on procurement.  A matter that we would
recommend for particular attention in the Detailed Business Case phase is whether the same procurement
options apply uniformly across the two preferred options.  Some segments may warrant a different approach, for
example to address different risk profiles.
e.
From the draft IBC evaluation we note that we see scope for the decision-makers to explore the potential for
making a judgment to take only one option forward to DBC if they wish.  The draft IBC satisfies the objective of
providing decision-makers with an early indication of the preferred investment, however in recommending taking
forward two options to DBC it does increase the investment in evaluation needed.  This is, of course, offset by
providing broader optionality at DBC stage. The two recommended options identified are sufficiently different to
warrant further multi-criteria evaluation during the DBC (acknowledging that one of the drivers behind the decision
wil  be political considerations).
f.
The IQA scope included undertaking specific evaluations for the fol owing matters:
  A review of the Multi Criteria Analysis and options evaluation.  This included a check of:
–  The process used to determine the criteria
–  The chosen criteria
–  The process used to evaluate the options
–  The results of the options evaluation
–  Check of the sensitivity analysis
We consider the MCA development and evaluation approach to be suitable for the IBC and it provides a clear and
logical process for alighting on the two preferred options.
  A review of the range of options identified to determine whether there is a sufficient range.
–  This exercise has relied heavily on the PTSS.
–  The business case process would typical y require the development of a Programme Business Case
(PBC) where a range of alternative programmes with intervention options would be investigated. These
would typical y be wide ranging.
–  The PTSS has, in our view, been used in lieu of a PBC to determine the preferred programme, although
some components of the PBC have been incorporated within the IBC.
–  We are comfortable with this approach given the genesis of the PTSS work and its comprehensive nature,
and we therefore consider the range of options identified to be suitable.
  A review of the Cost Benefit calculations, including:
–  Review the cost benefit profiles to determine whether there are any errors
–  A cel  by cel  review of the excel data model, checking for errors.
–  The tools ability to be dynamic and ease of use in subsequent business casing steps.
No material errors have been identified during our IQA.  We consider the spreadsheet tool’s ability to be used in
subsequent stages wil  be appropriate.

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 8

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Dimensions Assessed For The Indicative Business Case
2.3
Consulted
  Does the draft IBC canvass the approach to consultation for IBC purposes (see IBC guidance and NZTA’s
version)?
  Are relevant stakeholders and relevant experts identified and their key considerations articulated?
  Does the IBC explain how any issues raised in consultation have been addressed or dealt with?
Reviewer’s opinion:
Our understanding is that a conscious decision has been taken to focus consultation and engagement within the key
partners group.  This reflects the process to date, and in particular the significant body of work and related engagement
and consultation process for the PTSS. We acknowledge that a justification therefore exists for using targeted
consultation for the IBC phase. We note however that guidance for best practice from the NZ Treasury and NZ Transport
Agency would suggest some wider consultation would stil  be encouraged.  This is however ultimately a judgement cal
for the Project Governance and / or Steering Group based on the specific circumstances of the BRT initiative.
In terms of specific considerations we note that:
a.
The NZ Treasury and NZTA guidance promotes the idea of an IBC not just stating what consultation has been
undertaken, but also explaining the nature of any issues raised or views expressed by stakeholders, and how
these have been taken into account in the development of the IBC.  This aspect of the draft IBC is limited and
may merit further attention in our view.
b.
In the later DBC phase, achieving good practice wil  necessitate achieving broad engagement and consultation.
Given the approach to the IBC phase there is therefore potential for the DBC to put an enhanced focus on
achieving excel ence in consultation and engagement.    Ensuring a suitably comprehensive and robust strategy
for the DBC stage wil  be needed.
2.4
Clear and concise
  Is the material communicated in plain English, with minimal use of jargon and any technical terms explained?
  Is the material structured in a way that is helpful to the reader?
  Is the material concisely presented, with minimal duplication, appropriate use of tables and diagrams, and references
to more detailed source material, to help manage the length?
Reviewer’s opinion:
Our view is that the document is wel  structured and easy to fol ow. In summary:
a.
The architecture of the report and layout of the draft IBC is clear.
b.
Good use has been made of various Table formats.  Colour coding assists in distil ing out key considerations in
the MCA evaluation sections.
c.
To assist with understanding and clarity the description of the options would be assisted by the use of cross
section images, or concept sketches. We understand this is under consideration already by the IBC authors.
d.
The draft IBC is suitably concise given the subject matter and required coverage.  Our suggested amendments
revisions wil  not material y impact this in our view.
e.
The overall IBC narrative is pitched appropriately in our opinion.  The ‘case for change’ usefully draws out
essential elements of the story from the body of the IBC.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 9

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
3  NZ Transport Agency IBC Guidance Questions
In addition to the IQA assessment matters in the preceding sections, the NZ Transport Agency has available
on-line specific guidance for transportation related initiatives. For Indicative Business Cases the guidance
encourages achieving or traversing the matters set out in the Table below 3.
The draft BRT IBC in our view satisfies the intent of the clear majority of matters identified.
1. Is the option going to alleviate the perceived transport problems and / or maximise potential opportunities
identified in the programme business case?
Reviewer's opinion:
The draft BRT IBC options identifies Options 3 and 5 as preferred to take to DBC stage.  Due to the transition phase
between evaluative regimes (i.e. the previous process leading to the PTSS) and the more recent Business Case
framework, the PTSS assumes the role of Programme business case for the purposes of IBC development.
We are general y satisfied that the draft IBC has an appropriate level of information, provides sufficient of the substantive
elements of each option, and provides satisfactory information to form a view as to the relative efficacy of the two
preferred options in al eviating the transportation problems identified.
2. Is the proposed solution the best way to respond to the problem and deliver the expected benefits?
Reviewer's opinion:
The two recommended options are underpinned by a proportionate and suitably targeted response to the problems.  The
level and nature of analysis and interpolation is in our view commensurate with the size and complexity of the problems.
The options also respond appropriately in our view to the magnitude of the impacts and risks.
We consider that the conclusions reached relate logical y and consistently to the IBC analysis of the options.
3. Does it tell the story as to why this is the preferred option/solution?
Reviewer's opinion:
The narrative developed and accompanying logic trail are clear and communicate the basis upon which the preferred
options are arrived at.  Relative advantages and disadvantages of options are set out through the MCA and other parts of
the cases, enabling informed judgments to be made.
4. Is the option consistent with established policy directives?
Reviewer's opinion:
The high level policy settings are set out at Section 1.4 (page 19) of the draft IBC.  These set out the key considerations
and enables alignment of the BRT initiative with these to be understood.

3 Guidance source:
http://hip.nzta.govt.nz/__data/assets/pdf_file/0016/51109/14203_NZTA_IndicativeBusinessCase_OverviewPosters_Final
.pdf  Version as at 16 June 2015
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 10

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
5. What are the likely scale and significance of any impacts of the option?
Reviewer's opinion:
The multi-criteria analysis results at Section 2.6 (page 43) and Cost Benefit analysis at Appendix C (page 89), plus other
elements of the five cases convey relativities and enable judgments as to significance and likely scale to be made.  We
consider these to have been satisfactorily addressed for the purposes of IBC decision-making.
6. Is the option likely to be acceptable to the public, affordable and feasible to construct and operate?
Reviewer's opinion:
Suitable and sufficient information for the IBC phase is available to inform decision-makers views on affordability and
feasibility in our view.  Public acceptability is not readily discernible as this wil  reflect a range of matters which shape
public attitudes.  Assessment of these matters is distributed across the MCA framework (e.g. 5.1 PT user satisfaction,
7.1 land take (and community response to this), 8.4 Rates impact (and acceptability of this)).  Understanding of these
considerations obtained via the PTSS provides insight into the public acceptability and this is a matter that would be a
focus under the Detailed Business Case.
7. Is there a clear rationale for the rejection of options on completion of the indicative business case?
Reviewer's opinion:
The synthesis of MCA, CBA and 5 case narratives provides a clear basis for why options other than 3 and 5 were
rejected.  We consider this aspect is suitably covered off.

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 11

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015

Appendix A
Economic Evaluation
Assessment
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 12

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
General Information (Section A)

Application for Funding:
New Zealand Transport Agency
Evaluation Date:
June 2015
Reviewer:
Bob Hu, Beca Ltd. Wel ington
Project Name:
Wel ington BRT Indicative Business Case
Problem Description:
The purpose of the IBC is to confirm the strategic rationale for BRT, to identify and
evaluate options for BRT and to develop a preferred way forward, and to provide a high-
level consideration of the financial, commercial and management options available.
Alternatives and Options
Five main options of bus priorities through Wel ington city have been considered. And
Considered:
variations on these options have also been tested.
Preferred Option:
The preferred options have not been confirmed as the report has not been finalised yet.  It
is considered appropriate to not finalise a preferred option at an IBC level, as the purpose
of the IBC is to develop a way forward, and the preferred option can be developed and
assessed in a DBC stage if it is needed.
Do Minimum:
The do minimum has included the likely future improvements, such as the WNCR
schemes, base PT improvements and minor network works.
Project Costs:
The capital cost for the construction of the project has been estimated as a range
between $58M (Bus Priority) to $207M (BRT)
Key Project Attributes:
The project proposes and assesses the possible BRT improvements around the major
routes in Wel ington city. The routes have been grouped into three “branches”, including:
  The Central branch: From the Railway Station to Basin Reserve;
  The Newtown branch: From Basin Reserve to Constable Street; and
  The Kilbirnie branch: From Basin Reserve to Kilbirnie Crescent.
Communications:
Discussions with PWC IBC authors

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 13

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Conclusions (Section B)

In general, the economic assessment has been undertaken using standard 2013 EEM
Conformity:
procedures. The spreadsheet economic model is considered to be acceptable and fit for
purpose of the high level BRT Indicative Business Case study.
The traffic performance values used for the economic assessment were based on the
previous Wel ington PT Spine Study model ing outputs with a number of high level
assumptions. There is no specific traffic model ing exercise being undertaken for the
current study. Although this is considered to be a risk for the traffic benefits estimation,
this risk can be minimised by sensitivity tests. Fol owing a review of the methodology of
the benefits interpolation, the reviewer agrees that the approach is appropriate
considering the current phase of the project is only an IBC level. The reviewer
recommends that a project specific model ing exercise (i.e. macroscopic and micro-
simulation) wil  need to be undertaken for any subsequent business case steps  in order
to capture the more precise transportation benefits.
The economic assessment has considered eight benefits / disbenefits components below.
Without detailed traffic assessment for the IBC, the larger number of components it
captures, the higher potential bias the results could have. However, since there is no
specific requirement regarding this, the current assessment is considered to be
acceptable.
  PT travel time benefits
  Additional PT user benefits;
  PT reliability benefits;
  Walking benefits;
  Emissions reduction benefits;
  Agglomeration benefits
  General traffic decongestion dis-benefits and
  General traffic VOC benefits.
Credibility:
The fol owing information has been provided to the reviewer for this economics peer
review.
  Wel ington PT Spine Study peer review report (18th Nov 2013);
  PwC BRT Indicative Business Case draft report (15th June 2015);
  PwC BRT economics interpolation extract methodology (16th June 2015);
  PwC BRT economics calculation spreadsheet s v21 (10th June 2015);
  GWRC estimation of BRT travel time benefits spreadsheet v2 (15th June 2015); and
  GWRC BRT IBC SATURN Model ing technical memo v2 (12th June 2015).
A review of the capital cost and maintenance cost estimates has not been undertaken as
part of this economic peer review as requested by the client. It is understood that the cost
estimates for the BRT IBC study were obtained from the PTSS work. The risk of cost
uncertainties wil  lie with the Road Control ing Authority.
Choice of Do Minimum:
A do minimum has been assumed rather than a do nothing scenario. The do minimum
has included the likely future improvements, such as the WNCR schemes, base PT
improvements and minor network works. The selection of do minimum is considered
appropriate.
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 14

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Identification of Options:
Five main options have been identified as part of the assessment including:
  Option 1, Targeted bus priority and other modes improvements;
  Option 2, Peak bus lanes and priority;
  Option 3, Targeted bus lanes and priority;
  Option 5, Ful  bus lanes and ful  priority; and
  Option 7, Physical y separated bus lanes and ful  priority.
Two variations on these options have also been considered based on the timing of the
construction and implementation; and a lower-quality solution, or no solution at al  for the
“Kilbirnie branch”. The option identification is considered to be appropriate.
Economic Efficiency
Refer to section D and E.
Evaluation:
Sensitivity and Risk
A number of sensitivity tests have been undertaken by the assessor as below. The
Analysis:
sensitivity tests are considered to be appropriate and added value to the economics
assessment.
  Sensitivity tests on project benefits periods and discounting rates; and
  Sensitivity tests on timing variants.
Further sensitivity tests are recommended by the reviewer as part of the DBC phase and
address:
  Benefits interpolation proportion;
  Travel time reliability benefits;
  Walking benefits.
Assessment Profile:
Assessor’s Profile
Reviewer’s Profile
Strategic Fit:
n/a
Strategic Fit:
n/a
Effectiveness:
n/a
Effectiveness:
n/a

Economic Efficiency:
n/a
Economic Efficiency:
n/a
Reviewer’s Comments:
We understand that NZTA would general y expect an indication of the overal  Assessment
Profile.

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 15

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Reviewer’s Recommendations (Section C)

3.1
The accident savings have not been included in the benefits analysis. It is understood that the public transport
project under an ICB phase does not require any crash analysis. However, by incorporating the possible
accident benefits may capture some additional differences between the Targeted bus lanes options versus the
Ful  bus lane options versus the Physical y separated bus lanes.
Significance Level: Comment only
3.2
The benefits and costs for each individual options have been interpolated between the Bus priority option
(Option 1, lower bound) and the BRT option (Option 7, higher bound) from the PTSS work. The methodology of
the interpolation is considered virtual critical to the economic analysis. This information (i.e. Appendix B) wil  be
requested by the reviewer.
Significance Level: Additional information received and concluded to be satisfactory
3.3
As discussed in 3.2, the benefits and costs have been interpolated between Option 1 and Option 7. It has been
noted that Option 1 (Bus priority) does not compatible with the future Basin Reserve improvements, however al
the other options include Basin Reserve improvements. Therefore Option 1 may not be appropriate to be
considered as a reference, or the consideration the assessor undertook has not been included in the report.
Significance Level: Additional information received and concluded to be satisfactory
The reported reliability benefits are exactly the same as the travel time benefits for al  options. Since the
3.4
reliability benefits normal y only contribute to 25% of the travel time benefits for a transport project, explanation
wil  be requested.
Significance Level: Explanation received and concluded to be satisfactory
The emission reduction benefits have been included in the benefits calculation, however, they are reported as
3.5
$0 crossing al  option.  The general expectation of the emission benefits is about 4% of the VOC benefits.
Explanations on the emission benefits wil  be requested.
Significance Level: Explanation received and concluded to be satisfactory
Under the variation tests, by including the Kilbirnie branch scheme, the benefits have almost double for al
3.6
options, however the costs stayed similar. Therefore, the incremental benefits to costs ratio seems high. The
reviewer wil  investigate further on the assumptions relating to these.
Significance Level: To be investigated further in DBC phase
Under the variation tests, the 40 years benefits periods for the project should always from the starting of the
3.7
first construction. The delay of the Central branch improvements mean, it will have less number of year’s
benefits for Central branch components. The reviewer wil  investigate further on the assumptions relating to the
discounting.
Significance Level: To be investigated further in DBC phase
The possible dis benefits during the construction of the project has not been included. However, with a wel -
3.8
developed construction management plan, the effects could be managed to a minimal level, and it can be
considered in the future DBC. It is considered appropriated.
Significance Level: Comment only
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 16

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Maintenance cost has not been included in the cost estimates. However, due to the current phase is only at an
3.9
IBC stage, it is considered appropriate.
Significance Level: Comment only
The specific values used in the economic assessment wil  be check by the reviewer, include:
3.10
  The version of EEM;
  Values of time and VOC; and
  Update factors. Etc.
Significance Level: To be revisited as part of DBC phase
3.11
Minor clarifications:
  The travel time reported in Table 5, was it in dol ar values or minutes or hours?;
  The travel time savings reported in Table 6, was it for Bus only or it includes general traffic as wel ?
  There was a few type errors in the report, for example, the reviewer believes the Table 10 and Table 11
have a row reporting “Total Cost” instead of “Total Benefits”.
Significance Level: Comment only

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 17

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Evaluator’s Economic Efficiency Analysis (Section D)

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 18

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Reviewer’s Economic Efficiency Analysis (Section E)

A full re-evaluation has not been undertaken, however an indicative benefit calculation is carried out (as
follows) on full BRT Option 7 with the Kilbirnie branch and where the Central spine is delivered immediately.
Benefits Elements
Opt 7b
Comments
Travel Time benefits
$35M
7 min/pax (TT saving based on SATURN model) *1/60 * 5000
Similar to section 4
pax/hr (based on WTSM) * 2 hr/day * 240 working days/yr *
$8 TT cost/hr * 15.5 (USPWF 40yrs, 6%) = $35M
Additional PT user
$3.5M
7 min/pax (as above) * 1/60 * 400 pax/hr (based on WTSM) *
benefits
Similar to section 4
2 hr/day * 240 working days/yr * $8 TT cost/hr * 15.5
(USPWF 40yrs, 6%) * 0.5 (rule of half) = $3.5M
Reliability benefits
$21M
4.8 min (EEM late ratio) * 3.5 min (base on GWRC survey) *
Canvassed with IBC
1/60 * (5000*0.5) pax/hr (only trips beyond Basin/Mt Vic) * 2
authors as different to
hr/day * 240 working days/yr * $24 TT cost /hr * 15.5
section 4
(USPWF 40yrs, 6%) = $125M
Based on GWRC survey (Table 6, page 9, BRT IBC draft
report), 17% of the al  stops get delayed. Therefore, the
benefits for passengers affected = $125M * 0.17 = $21M
Walking benefits
$2M*
The project is not considered as a walking / cycling project.
Canvassed with IBC
The associated cost / scheme should be clearly defined if the
authors as different to
Walking benefit is to be included.
section 4
The economics undertaken for the walking benefits is not EEM
standard. Please fol ow the EEM SP11 procedure if the
walking benefit is to be included.
Assuming al  the 400 pax/hr are new pedestrians and
everyone walks 250m,  the total Health and environment
benefits for walking facility would be as below.
$3 (EEM rate) * 0.25 km * 400 pax/hr * 2 hr/day * 240
working days/yr * 15.5 (USPWF 40yrs, 6%) = $2M
Emissions reductions
$0.7M
According EEM, 4% of the VOC benefit is assumed
benefits
Similar to section 4
Agglomeration benefits  $5M*
The assessor is required to provide the reference for the “15%
Canvassed with IBC
of TT benefits” used for estimating Agglomeration benefits.
authors and justification
provided and amendment
to text noted
Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 19

Attachment 1
Bus Rapid Transit Indicative Business Case Independent Quality Assurance Review 2015
Benefits Elements
Opt 7b
Comments
Decongestion
-$3.4M
New indicative benefit calculation is NOT carried out by the
disbenefits
reviewer.
Assessor’s methodology is considered appropriate, model ing
and calculation has not been checked.
VOC benefits
$18M
New indicative benefit calculation is NOT carried out by the
reviewer.
Assessor’s methodology is considered appropriate, modelling
and calculation has not been checked.
Total Benefits
$82M
This will give a BCR of 1.0 assuming there is no change
on the Costs.

Beca // 2 July 2015
3819906 // NZ1-10925599-2 0.2 // page 20