1 Thursday, 22 March 2018 2 (9.30 am) 3 CHAIR OF THE INQUIRY: Good morning. As you will be aware, 4 this is the first anniversary of the incident at 5 Westminster where several people lost their lives and 6 others were injured. I ask you all to stand for 7 a minute. 8 Mr Lake. 9 MR LAKE: My Lord, the first witness today is 10 Professor Flyvbjerg. 11 PROFESSOR BENT FLYVBJERG (sworn) 12 Examination by MR LAKE 13 CHAIR OF THE INQUIRY: You are going to be asked some 14 questions by Mr Lake, the Counsel to the Inquiry. If 15 you just listen to the question and answer it as 16 directly as possible, and if you don't understand the 17 question or wish clarification, just say so. 18 Can you speak clearly into the microphone so 19 everyone can hear your evidence, and slowly so that the 20 shorthand writers can record it. 21 A. I will do my best, thank you. 22 CHAIR OF THE INQUIRY: Thank you. 23 MR LAKE: My Lord. 24 Professor Flyvbjerg, could you state your full name, 25 please? 1 1 A. My full name is Bent Flyvbjerg. 2 Q. I think you are the BT Professor and inaugural Chair of 3 Major Programme Management at Oxford University, Said 4 Business School? 5 A. That's correct. 6 Q. And also a professorial fellow of St Ann's College, 7 Oxford? 8 A. That is correct. 9 Q. I think you have been instructed by the Inquiry to 10 provide a report into various issues of risk and 11 optimism bias? 12 A. Yes. 13 Q. I would like you to look at a document, please. It 14 should be in hard copy in front of you, and also shown 15 on the screen to your right. It's reference 16 TRI00000265. 17 Is that the report that you prepared for the 18 purposes of this Inquiry? 19 A. Yes, it is. 20 Q. Are you content that its contents be accepted as your 21 evidence given under oath on this Inquiry? 22 A. Yes. 23 Q. Thank you. 24 I don't intend to go through it page by page, but to 25 consider certain of the issues that arise. Perhaps at 2 1 the outset, we could look at page 41 of this. 2 By way of summary, we can see that your report also 3 includes your CV and details of your experience and 4 expertise? 5 A. Yes. 6 Q. If we go forward to page 53, under the heading "Advisory 7 Roles for DfT and HMT", the Department of Transport and 8 Treasury, we can see the first thing you have noted 9 there is the guidance document on optimism bias 10 procedures in 2004? 11 A. Yes. 12 Q. I think that's something that you consider in the body 13 of your report? 14 A. Yes, it is. 15 Q. If we put the report aside for the moment and take that 16 off the screen, before turning to the questions of 17 optimism bias, I would like to ask you a few questions 18 about risk more generally to inform the Inquiry. 19 First of all, could you explain what is a quantified 20 risk analysis? 21 A. Yes. A quantified risk analysis is basically a register 22 of risk, specific risks that might happen to a project 23 or an investment, and also probabilities of those risks 24 actually occurring and the impacts that the risks might 25 have. So a risk register, a list of probabilities, and 3 1 a list of the impacts that the probabilities might have 2 if they occur. That's basically a quantitative risk 3 assessment. Those are the basics of a quantitative risk 4 assessment. 5 Then on top of that, something that is called Monte 6 Carlo simulations are done based on the elements that 7 I just mentioned. A quantitative analysis called 8 a Monte Carlo simulation is done in order to assess the 9 likelihood at different levels, or the 90 per cent 10 likelihood that something might happen, or the 11 50 per cent likelihood that something might happen. So 12 that's a probability distribution that's generated by 13 this Monte Carlo simulator. 14 Q. If you can just break down some of the elements of that, 15 you started with identification of risks, and inclusion 16 of those in a schedule or a table. 17 How is that done? Who does that in a project? 18 A. The way it's typically done in the quantitative risk 19 assessment is by a panel of experts. So you basically 20 get a panel of experts in a room and they fill out 21 a questionnaire or several questionnaires about what 22 they think are the likely risks to happen to the project 23 or investment at hand, and what the probabilities are, 24 what they think the probabilities are that this would 25 have, what they think the impacts are going to be and so 4 1 on. So you get a range of experts to do this, and then 2 you aggregate the surveys that the experts have done, 3 the numbers that they have put in there, and on that 4 basis you calculate what the risks are according to 5 these experts. 6 CHAIR OF THE INQUIRY: Could I ask you, Professor, to speak 7 a little more slowly for the sake of the -- 8 A. Sorry about that. 9 CHAIR OF THE INQUIRY: Not at all. 10 MR LAKE: Presumably the advantage of having more than one 11 person identifying the risks is to try and minimise the 12 chance that a risk is overlooked? 13 A. Yes, it is. So you want a range of different experts 14 who cover different areas of expertise in relation to 15 the project that you are assessing. And like you said, 16 having a range of experts will give you a broader view 17 of what the risks might be. 18 Q. Although you have a broader view, is that approach in 19 any sense foolproof or is it in practice the case that 20 risks slip through the net? 21 A. Risks might slip through the net, but that actually is 22 not the major problem with QRA, quantitative risk 23 assessment. 24 Q. What is the major problem? 25 A. The major problem is that it has been documented that 5 1 when you assess risk like this, based on the subjective 2 valuation of experts, there will be a bias in the 3 outcome. 4 It turns out that everybody, including experts, are 5 born with biases or develop biases throughout their 6 life, and one basic bias is optimism. So we all tend to 7 be basically optimistic. That's a human thing, and 8 people sometimes think if somebody is an expert in 9 a area, then they are realistic, they are not biased. 10 They don't have the optimism bias. That's actually been 11 documented in research over and over not to be the case, 12 that experts are biased just as laypeople are biased. 13 So what you get is a biased outcome of the 14 quantitative risk assessment when you use this approach. 15 Q. How does that bias you describe there affect the 16 outcome? What's the practical effect? 17 A. So, for instance, if you envision the experts trying to 18 assess what it is going to cost to do a project, if you 19 are optimistic in doing a cost assessment, then you 20 assess that it's going to cost less than it actually 21 turns out to cost. That's optimism. If you are doing 22 a budget and you are optimistic, you will have a low 23 budget, and that will come back as a cost overrun later 24 in the project when it's being built. 25 Q. Looking at that, at the application of that bias in the 6 1 context of the QRA, if you have this panel of experts 2 sitting, how will that bias affect their prediction of 3 the possible financial consequence or time consequence 4 of risk? 5 A. So in each -- I was just using cost as an example, and 6 it can be anything. Like you said, it could also be the 7 timetable, or it can be environmental impact or safety 8 impact, whatever. 9 The way the optimism bias affects things is 10 generally that the costs and risks will be 11 underestimated, because people are optimistic about the 12 cost and the risks, whereas the benefits and the 13 opportunities will be overestimated because that's how 14 optimism plays into benefits and opportunities. We 15 think things are going to go better than they do in 16 general. 17 Q. You said for the quantified risk analysis, as well as 18 identification of the risks, and identification of the 19 possible consequences, there was consideration of the 20 probability of that risk arising. Is that too affected 21 by the optimism bias? 22 A. Yes. So something negative would be -- you would 23 underestimate the risk of that, and something positive, 24 you would overestimate the risk of that, according to 25 the optimism bias. So yes, it does affect the outcome 7 1 of probabilities. 2 Q. We've also heard in the evidence given to the Inquiry so 3 far that when there is a risk analysis, a risk schedule 4 for the project, mitigation measures are proposed and 5 the effectiveness of those is assessed. Is that too 6 something that's affected by optimism bias? 7 A. Yes, it is. Optimism bias is pervasive. It's something 8 that is hardwired into our brains, and it will therefore 9 affect any assessment, including that. 10 Q. Leaving that aside for one moment, you also mentioned 11 Monte Carlo software. Can we be clear that what 12 information is put into the Monte Carlo software, what 13 goes in? 14 A. The input to the Monte Carlo software are the -- are the 15 assessments of the experts in the QRA. So that is the 16 input. Those are the data that the Monte Carlo 17 simulation works on the basis of. 18 Q. What is the output from that software? 19 A. The output is a probabilities -- probability 20 distribution that has been simulated by the software on 21 the basis of that data. 22 Q. Is this software or programme similar to it in general 23 usage? 24 A. Please say again. 25 Q. Is that software used generally, it's accepted that it 8 1 works? 2 A. Yes, it is. It's used worldwide and it's used in the 3 financial sector as well as in project management, which 4 is what we are talking about now. It's a very 5 widespread methodology that is used in a lot of places. 6 Q. So you said it gives you the different probabilities of 7 risk situations arising. We've seen in your report 8 a number of times there's references to P values, P50, 9 P80, P90. What are they? 10 A. They are shorthand for how likely it is to stay within 11 budget or within schedule or within whatever the 12 variable is that you're looking at. So a P90 for the 13 budget means that you have 90 per cent likelihood of 14 staying within budget. And therefore only a 10 per cent 15 risk of going over budget. So that's what the P90. 16 If it's P50, it means that you have a 50 per cent 17 likelihood of staying within budget and a 50 per cent 18 likelihood of going over budget. 19 Q. Just to stay with that P50, just to be clear about what 20 it means, if you're using a P50, it's as likely to be 21 over budget as it is to be under budget? 22 A. Exactly. It's 50/50. That's also called the median. 23 Q. We see also reference to P mean. Is that different to 24 the median? 25 A. Yes. So the P mean is the average, and the average and 9 1 the mean might be the same or they might be different. 2 In this type of project they are typically different. 3 Q. Looking at what the P figure means, taking a higher P 4 number like P80, it's clear from what you said that that 5 still means there's a 20 per cent chance that that 6 budget figure would be exceeded, that risk figure would 7 be exceeded? 8 A. That's correct. 9 Q. Does the quantified risk analysis or the probability 10 software, Monte Carlo analysis, give you any idea by how 11 much it might be exceeded? 12 A. It gives you the probability that it would be exceeded 13 but you need to combine that with the actual size of the 14 budget to find out how much. 15 Q. If we have all these different probability figures, P50, 16 80, 90 or P mean, how does a project choose which is the 17 correct one to use or the appropriate one to use in 18 their circumstances? 19 A. A project decides that by deciding how willing they are 20 to take risks. 21 So if you are willing to take a higher risk, let's 22 say if you are a portfolio manager and you have many 23 projects, you win some, you lose some. Some will come 24 in over budget, some will come in under budget, and they 25 compensate for each other. They even each other out, 10 1 because you have a whole portfolio. 2 Then you would actually want to use a P50 and accept 3 the 50 per cent risk that some projects will go over 4 budget and some projects will go under. But it's up to 5 you to decide, and that's your question really. If 6 you're a very conservative management or owner of the 7 project and you -- maybe you are only doing this one 8 project and it's very important for you not to go over 9 budget, then you would absolutely not be satisfied with 10 a P50 that is a 50 per cent risk of going over budget. 11 You would want to go for P80. So that you have 12 80 per cent chance of staying within budget or even 13 a P09 or even P95. We see that. There are actually 14 projects that are, you know, billion pound projects 15 where it would be a disaster to go substantially over 16 budget; where the owner wants a P95. So 95 per cent 17 assurance to stay within budget. But that's really 18 a decision by the owner or the project team delivering 19 the project to decide what they want, and after they 20 have decided the Monte Carlo simulation, or whatever 21 probability of execution that it is that you refer to, 22 would then tell you what the budget is that you need to 23 have. 24 It's like buying insurance. Most people understand 25 what it means to buy insurance and they understand, if 11 1 you want a higher level of security in your insurance, 2 it costs you more. It's exactly the same with this. If 3 you want a high level of security, that you stay within 4 budget, you need a higher contingency, and therefore 5 your budget will be larger and it will cost you more. 6 Q. Obviously the prediction figure used, the probability 7 figure makes no difference to the actual outturn of the 8 project. It does make a difference to how much 9 confidence you can have that you're not going to exceed 10 your budget amount? 11 A. That is correct, but that in itself might actually also 12 affect the outturn of the project because -- 13 Q. In what way? 14 A. In the way that if you don't stay within budget, you 15 have political scandal, you'll be on the front page of 16 the media, and all of a sudden the project team that is 17 supposed to deliver the project will be preoccupied by 18 putting out fires in the media, and applying for more 19 funding or raising more funding for the project, because 20 they don't have enough money now evidently because 21 there's an overrun. And that will distract them from 22 what they are supposed to do, which is deliver the 23 project, and the quality of delivery would therefore 24 suffer, which in itself will affect the cost and we see 25 this vicious cycle happening with projects once that 12 1 starts. So it actually may affect the outcome of the 2 project in that way. 3 Q. That situation that you are describing there would arise 4 where there has been an insufficient allowance, or the 5 figure has not been set high enough to cope with the 6 probability? 7 A. Yes, exactly. 8 Q. In an earlier answer you gave to me, you contrasted on 9 the one hand an organisation with a portfolio of 10 projects and on the other a particular project promoter 11 who is really only dealing with one project. What's the 12 difference between these two situations in terms of 13 choice of probability? 14 A. So if you have a portfolio, you would want to choose 15 a P50 value basically. So you would accept the 50/50 16 risk of cost overruns because half your projects will go 17 over budget, half your projects will go under, and on 18 balance they will tend to cancel each other out. 19 Whereas if you have only one project, you will 20 typically want a much higher P value, meaning a much 21 higher likelihood of staying within budget because you 22 have one -- only one project. Often a large project 23 that can have negative impacts on your organisation if 24 it goes over budget. 25 So you choose a -- P80 is a typical value that we 13 1 see more conservative clients using, and like I said 2 before, even P90 and P95 for ultra-conservative clients. 3 Q. Presumably, as you move from P80 to P90 and beyond, 4 there are substantial increases in the allowance that 5 has to be made as you get higher and higher? 6 A. Yes. So we call that increasing marginal cost of 7 insurance, and that applies here. So it becomes more 8 and more expensive, the further towards 100 you get, and 9 it becomes very expensive. That's why we haven't seen 10 anybody going beyond P95 yet, because it would be 11 incredibly expensive to get the last 5 per cent. 12 Q. What you are describing there is very much the interests 13 of the person making the allowance for risk, what 14 their -- that body's individual appetite for risk is. 15 Is it likely to be a situation there where, within 16 a particular project, different parties will have 17 different appetites for risk and therefore use different 18 probabilities? 19 A. That's a possibility, and you might even want to do that 20 deliberately, give different actors within the delivery 21 of a project different budgets. 22 So the project owner at the top might want to have 23 reserves available that can be used in an emergency or 24 in a tight situation where there's not enough money or 25 something unexpected happens, so they don't have to go 14 1 back and ask for more money. So they have the 2 contingency. But they might not want to give that 3 contingency to the project director. So the director 4 has actually a lower budget. So we have one budget and 5 one contingency at the owner level, and then the project 6 director has another budget and another contingency that 7 is somewhat lower, and you might even have a third 8 budget which is what you give to the contractors. 9 You want to put pressure on the budget, so to speak, 10 because there is a tendency, if the money is there, it's 11 going to be spent, so you really need to manage the 12 contingency carefully, and this is a way to do that, 13 operating with three. We've even seen some projects 14 with five different contingencies and therefore five 15 different budgets allocated to different levels of the 16 delivery organisation. 17 Q. That brings up a particular issue I was going to ask you 18 about. 19 If a higher probability figure is used, and so 20 a greater allowance is made for risk, is there a danger 21 that once that money is there, it will end up being 22 spent on the project because everybody knows it's there? 23 A. That is a real risk. And you need to have measures in 24 place to avoid that happening. If you're not -- if you 25 don't carefully manage your contingency, the likelihood 15 1 is that it will be spent and you could actually have 2 a worse situation than if you had had no contingency, 3 because the contingency in itself becomes a carrot that 4 will drive spending. And you might end up having 5 a contingency, and then you need to put extra money on 6 top of that in the end, and the overall project becomes 7 more expensive than it would have otherwise been. 8 Q. What sort of measures would you commonly come across to 9 try and restrict the allowance -- avoid the situation 10 where the allowance ends up being spent, simply because 11 it's available? 12 A. So the most efficient way is to create an incentive 13 structure where the people who are delivering the 14 project are actually incentivised not to spend the 15 contingency. 16 You can do that by creating positive incentives that 17 if the -- if the builders are delivering to budget or 18 below budget, they actually get extra payment, they make 19 more money by delivering below budget or on budget. 20 That's a positive incentive. The other is the 21 stick. So you use both sticks and carrots. The first 22 was the carrot and the second is a stick that you punish 23 builders if they go over budget. That's the incentive 24 structure, and it's really important to think carefully 25 about that, and we find that good clients do that, you 16 1 know, mature clients who know how to deliver projects 2 and do it successfully, they have clear incentive 3 structures and partnerships that will incentivise 4 everybody involved to deliver on or below budget or on 5 or below schedule or at or above benefits, revenues, 6 whatever the relevant variables are. 7 So that's one way of doing it. 8 Another way is to make it very difficult to get 9 access to the contingency. So you don't just put it out 10 there. You actually put it with the Treasury, for 11 instance. So this is what the UK Government does for 12 very expensive projects that are funded out of the 13 Treasury. Or ministers. They will have part of the 14 contingency locked up in the Treasury, and you actually 15 have to go there and you have to meet certain criterion, 16 et cetera, in order for the contingency to be released. 17 So that's the second thing. Just to make it difficult 18 to get access to the contingency and to be clear about 19 what the criterion are for the contingency to be 20 released. 21 Q. Turning to the first of the -- returning to the first of 22 the two things there, the idea you said about 23 incentivisation, by sharing the benefit of cost 24 underruns and sharing the burden of cost overruns as 25 well. Would that mean that if the project went over 17 1 budget, the contractor had to bear part of the overrun 2 as well as the employer? 3 A. Yes. 4 Q. If it's under, they both share the benefits of that too? 5 A. Yes. 6 Q. That sort of approach seems slightly at odds with the -- 7 any desire to have a price that's absolutely fixed at 8 the outset, a fixed price contract? 9 A. Yes; correct. 10 Q. So putting that the other way round, is it fair to say 11 that if you have a contract where you insist on an 12 absolutely fixed price, it creates a situation where you 13 lose the incentive? 14 A. Very often with the -- yes, is the answer. And very 15 often with the fixed price you only use the stick. So 16 people get punished if they go over the budget, and so 17 that there might be fines, but also just the fact that 18 if the builder is actually carrying the whole risk, you 19 know, then it's going to hurt the builder that if they 20 go over budget because they actually have to cover that. 21 Q. Or if the way -- if the effect of the contract is it's 22 fixed price, does that in your experience act as 23 an incentive for the contractor to try and claim 24 variations, to drive the price up to get additional 25 money? 18 1 A. Yes, and I would actually say in general, in the vast 2 majority of cases, the notion of a fixed price contract 3 is a theory. It's an illusion that doesn't materialise 4 in reality. 5 It's called a fixed price contract, but it never 6 ends up being fixed price or almost never ends up being 7 fixed price. There are variations, and other things 8 that happen on big and complex projects like what we are 9 talking about. It's impossible to predict everything 10 that can happen, and therefore everything is not in the 11 contract. 12 So situations will arise where things happen where 13 it becomes an open question. Who is going to pay for 14 this? And a builder will say: it's not us; and maybe 15 the owner will say: it's not us; and then you have 16 a conflict, and often arbitration or even going to 17 court. 18 Q. Returning to the question of the appetite for risk from 19 the various parties, and looking in particular at 20 something like the Edinburgh Tram Project, where we have 21 a promoter, a local authority on the one hand, and 22 a funder, which is the Scottish Central Government on 23 the other, would you expect them to have the same or 24 different appetites for risk? 25 A. Probably different appetites for risk with the most 19 1 conservative stance taken the higher you go. 2 Q. If the project, instead of being carried out as 3 a one-off project by local authorities, is carried out, 4 and this would apply in Scotland or England, anywhere, 5 by a central government which is undertaking lots of 6 projects, does that make it easier to manage the risk 7 and to bear the risk of overruns? 8 A. Yes, it does. I mean, for one reason, there is a larger 9 budget to contain it within, but also there's more 10 experience in the sense of doing more projects and 11 having tried this before, and therefore having the 12 skills and the expertise that it takes to deliver 13 a project. 14 Q. Just sticking with the subject of experience, I think 15 you are involved now in a project, the academy of 16 project management, which is training people in the 17 various aspects of delivering these very large projects; 18 is that correct? 19 A. That's correct. It's called the Major Projects 20 Leadership Academy. 21 Q. What does that seek to do? 22 A. That seeks to train UK civil servants in how to deliver 23 major projects. 24 Q. And how many civil servants -- when did that start? 25 A. It started in 2012. 20 1 Q. How many people have been trained so far? 2 A. Approximately 500. 3 Q. Does that include people from Scotland -- 4 A. It does, yes. 5 Q. You referred earlier to the question of when you get to 6 very high probabilities that might be desired, something 7 over 95 per cent, the risk goes up dramatically -- the 8 cost of that goes up dramatically. 9 I think that might lead to something you were also 10 making reference to in your report, a black swan. What 11 is a black swan? 12 A. So a black swan used in this context is a metaphor for 13 a statistical outlier. So it's a very high value of 14 something. So if we talk about a cost black swan, it 15 means a project that had a very large cost overrun. 16 A very high percentage cost overrun. 17 Q. I think you have referred in your report to the fact, 18 you explain it in more detail, that the overrun has to 19 be more than 1.5 times the interquartile range of the 20 project assessments? 21 A. That is correct, and that's just a statistical method, 22 you know. That's how a statistician calculates what 23 a statistical outlier is, and then those are the ones 24 that are called black swans. 25 Q. Because you make the point in your report, after 21 1 analysing the fact, that the Edinburgh Tram Project, 2 over budget as it was, does not qualify for black swan? 3 A. That is correct. So when you use the statistical method 4 that you just referred to, you find that for light rail 5 schemes, given the data that we have, the black swan, so 6 the statistical outliers, are projects that had 7 143 per cent or above in cost overruns in real terms, so 8 no inflation included, and for the Edinburgh tram, the 9 cost overrun is 52 per cent in real terms, and obviously 10 that's much lower than 143 per cent. So the Edinburgh 11 tram is not a statistical outlier in that sense. 12 Q. When you've done that analysis, it's not of all projects 13 generally. That is specifically of light rail projects? 14 A. That is specifically light rail, and 23 per cent of 15 light rail projects have higher cost overrun than the 16 Edinburgh tram. 17 So the median is 25 per cent. So Edinburgh, the 18 Edinburgh tram had twice as big a cost overrun as the 19 median of projects. So the median is the middle value. 20 Q. But the average cost overrun is 25 per cent cost 21 overrun? 22 A. Not the average, the median. So 50 per cent is above 23 the median, and 50 per cent are below the median. 24 Edinburgh tram is clearly above the median. It is 25 twice as high as the median. Median 25 per cent, 22 1 Edinburgh tram 52 per cent. But the outliers don't 2 start until 143 per cent, and 23 per cent of all light 3 rail projects for which we have data are above 143 -- 4 no, are higher than the Edinburgh tram. 5 Q. The question of black swans, you explain there, that's 6 assessed purely by the size of the cost overrun. But 7 does that also -- is part of the definition of a black 8 swan dependent on the probability or the likelihood of 9 that occurring? Or is it purely cost related? 10 A. No. You could define your black swan like that, but 11 that's not how it was defined in the calculation we just 12 talked about. 13 Q. Presumably black swans are sufficiently rare, however, 14 that they are not something that would ever be caught by 15 a risk analysis at P50, P80 or even P90? 16 A. Yes. It's true that the black swans are in the very top 17 of the distribution, or in the right-hand side of the 18 tail, as we also call it sometimes, and they would be 19 above 90 or 95 per cent, that would be where you find 20 them. It depends on the specific distribution. But you 21 would not protect yourself against black swans by a P50 22 or P80, that is correct. You would need something that 23 we call black swan management, where you actually go 24 specifically for eliminating the black swans in order to 25 protect yourselves against them. 23 1 Q. What does that involve? What does black swan management 2 involve? 3 A. That involves identifying what are the specific risks 4 that could spin a project so much out of control that it 5 becomes a black swan, and how do you protect yourself 6 against those risks. 7 Q. If that sort of exercise is undertaken, black swan 8 management, does that have an overall effect on risk? 9 A. It does have -- it has an overall effect in the sense 10 that it also affects the median and the averages, but 11 not so much. A little. It has the -- the biggest 12 effect is actually clipping the tail, as we call it, 13 avoiding the very extreme outcomes, and that's the 14 specific purpose of doing it that way. 15 Q. What I'm really coming to with that question is: had 16 black swan management been employed in the Edinburgh 17 project, which wasn't a black swan, is it the sort of 18 thing that might have made a difference to the cost 19 outcome or would it not -- 20 A. It would have had some effect on the cost outcome, and 21 lowered it somewhat. And it would have protected the 22 Edinburgh tram from becoming an extreme outlier. 23 But with the benefit of hindsight, looking at what 24 happened in Edinburgh, in the Edinburgh tram, it would 25 have been more relevant to protect around the median and 24 1 the average than protecting against the extreme tail, 2 because the extreme tail didn't materialise here. 3 Q. In terms of protecting about the median and the average, 4 does that involve anything other than what might be 5 termed the standard risk management techniques? 6 A. Yes. So there's another technique called optimism bias 7 uplifts that has been developed by the Treasury and the 8 Department for Transport in the UK, and that's an 9 additional technique you use, and that is a specific 10 technique you use to protect yourself, not against the 11 tail, but against the mid-range and sort of upper 12 mid-range risks. 13 Q. I now want to turn to ask you more about optimism bias 14 uplifts or the adjustments that might be made. 15 You refer in your report to the inside view and the 16 outside view. Could you first of all explain what you 17 mean by the inside view? 18 A. So the inside view is understanding a project from the 19 inside. You understand the individual elements that the 20 project is made up of, the different parts that make the 21 project, and how they come together to form the overall 22 project. 23 If you're talking about risk assessment, like we 24 talked about before, the QRA, so the quantitative risk 25 assessment methodology is actually a typical example of 25 1 the inside view. You try to understand what are the 2 specific risks that apply to this project, given that 3 the project looks the way it does, and what are the 4 probabilities and impacts, and you add all this up to 5 get an overall estimate of the risks for the project. 6 That's a typical example of the inside view, so 7 understanding the project on its own terms. That's the 8 inside view. 9 Q. Now, from my understanding of your answers earlier, the 10 difficulty with that is the inside view involves 11 a number of exercises of judgement, and all those 12 exercises of judgements are likely to be tainted slightly 13 by optimism bias. 14 A. Correct. 15 Q. So what's the outside view? 16 A. The outside view is, as the term says, looking at the 17 project from the outside. So you basically say: okay, 18 let's find some other projects that look like this 19 project. So in the case of the Edinburgh tram, let's 20 find other trams. Let's find a bunch of trams that have 21 already been built and where we know what actually 22 happens. And then let's assume that, you know, pretty 23 much the same is going to happen to the Edinburgh tram. 24 We take the average. So what happened to these other 25 projects that have already been completed and we'll 26 1 assume that that has given us the best assessment of 2 what is likely to happen to the Edinburgh tram or 3 whatever project it is that you are planning. 4 So you take a reference group of projects that have 5 already been completed, and for which you therefore have 6 historical data. You don't have to assume things. You 7 actually know what happened. The only assumption you 8 make is that this is a good estimate for what's going to 9 happen to the project that we are now planning. That's 10 the outside view. 11 Q. I was going to ask you about that assumption you just 12 mentioned there. The assumption must be that the cost 13 overrun in any project is in statistical terms likely to 14 be the same as or informed by overruns in projects of 15 a similar nature. 16 A. That is the assumption or even better, the assumption is 17 that the best estimate you can get is what already 18 happened to previous projects historically, and you 19 can't better that by thinking that you are clever enough 20 to figure out what is actually going to happen to this 21 specific project that you are looking at. 22 You will not be better at predicting the future than 23 the historical data for the project that have already 24 happened. That's the -- that's -- a body of research 25 that has been done that has proven that, and that is 27 1 therefore -- that is the result that you take into this 2 methodology. 3 Q. That's what I was going to ask. Although I think both 4 you and I have described that as an assumption, it's not 5 just an assumption, it's been verified by research 6 carried out by many people, including yourself? 7 A. Including myself, but also including a Nobel Prize 8 winner in economics called Daniel Kahneman. So it's 9 very solid research. 10 Q. So using the outside view, optimism bias, how is 11 an allowance made using that approach? 12 A. Basically, you look at those previous projects that have 13 already been completed, and as a minimum we recommend 14 that you look at 20 to 30 projects, but the more you can 15 include, the better, because the more reliable your 16 estimate is. 17 But let's say you are looking at cost and cost 18 overrun, you just say for those 20 to 30 projects, for 19 each project, what was the cost overrun. You add that 20 up and divide by the number of projects you have in 21 average. That is your best estimate of the cost overrun 22 for the project that you are planning, unless you have 23 really good reason to think that your project is 24 different from the other projects. 25 That of course is exactly where you have to be 28 1 careful, because that judgement, whether you're better or 2 worse than the projects in your reference class, is 3 where optimism could re-enter your assessment. 4 Q. That sounds like once again an exercise of judgement by 5 the people reviewing a particular project. 6 A. Correct. 7 Q. Whereas if you're simply looking at an analysis, 8 published figures of what the cost overrun has been on 9 similar tram projects, from what you say, there appears 10 to be little or no judgement involved in that. That's 11 the whole point? 12 A. Correct. 13 CHAIR OF THE INQUIRY: When you're doing the calculation of 14 cost overrun and you achieve the average, would you be 15 including in the data that you are going to average 16 black swans? 17 A. Yes. We would. 18 MR LAKE: That comes to a point I was going to raise now. 19 We have heard that some things might not be included in 20 a QRA, and one of the things you mentioned is a black 21 swan. 22 We've also heard that a QRA wouldn't normally look, 23 for example, at a risk that the contract failed to 24 transfer financial risk to the contractor, or failed to 25 properly administer something that had been intended. 29 1 Would that be caught by reference class forecasting, 2 the extent -- the outside view? 3 A. Yes, it would. 4 Q. Because you just look at the overruns in all these other 5 projects and they would make up the statistical averages 6 that would then be used? 7 A. Basically anything that has happened to the projects in 8 the reference class, the historical projects, will be 9 included in the estimate of cost overrun, including 10 a risk that were not taken into account, and actually 11 including the very famous unknown unknowns. 12 So anything that happens to those historical 13 projects would be included in the numbers. So you don't 14 have to imagine it or visualise it. It's already there, 15 and therefore it's also taken into account for the 16 project that you are planning for. 17 Q. Turning for a moment to imagining a fight back against 18 that by people who use the risk approach, they might 19 say: there's a lot of detail goes into their QRA and 20 they get it checked by external people, therefore, it is 21 reliable, and it's not necessary to use other projects, 22 they are better with their own data. What is your 23 response to that? 24 A. My response is that this is correct regarding the 25 detail. So it is an advantage to have some level of 30 1 detail, but having things checked by outside experts 2 doesn't help because like I mentioned earlier, experts, 3 including outside experts, are subject to the optimism 4 bias. 5 The only way it would help to bring in outside 6 experts, if they use a method that is debiased, like 7 reference class forecasting where the optimism bias 8 doesn't have a chance to re-enter, then it would be 9 a good thing. But you could also do that internally. 10 You don't actually need to bring in outside experts for 11 doing that. 12 Q. Presumably, even if you're using the optimism bias 13 reference class approach, it's still appropriate to have 14 a risk analysis and a schedule of risk -- 15 A. Yes. 16 Q. -- mitigations and all these things, because that 17 controls the underlying costs? 18 A. Of course. You need to know that kind of detail when 19 you're actually hands on managing a project. So what 20 I recommend, and what my team recommends, is 21 a combination of the two, that you do your conventional 22 stuff, because it gives you this disaggregate detailed 23 idea of what the risks are. I think it was 219 24 different risks in the case of the Edinburgh tram, and 25 that's very useful to have that kind of overview. But 31 1 then you actually adjust that by the optimism bias 2 uplift. You know what is the empirical bias and this 3 kind of thing from your historical data, and then you 4 simply adjust what's in the conventional QRA. 5 That's the combination that really works. 6 Q. Would the QRA be of any use to identify which of the 7 risks is best to focus on, the most important risks, the 8 largest risks, or is that again something where it's too 9 affected by bias to be of value? 10 A. It would help you to give an indication of what to look 11 for. So the risks with the highest probabilities and 12 the highest impacts is what you would look for. But you 13 would have to remember that from the QRA, that would 14 still be subjective risk. So you would want to 15 double-check that with empirical data in order to verify 16 whether they actually are the highest probabilities, 17 whether they actually are the highest impacts. 18 Q. We heard -- we were discussing QRA in the Monte Carlo 19 software that it produces a range of probabilities. 20 A. Yes. 21 Q. Is it possible to get different probabilities from 22 reference classes? 23 A. Yes. The outcome of a Monte Carlo simulation and the 24 outcome of a reference class forecast are both 25 probabilities distribution, and actually if you fit in 32 1 the data from the reference class forecast into a Monte 2 Carlo simulation, there should be on principle the same. 3 The problem with Monte Carlo simulation is that they 4 don't do that. They use the subjective data, instead of 5 historical data, typically, and that's why they look 6 different. But the Monte Carlo simulation is typically 7 underestimating risk, whereas reference class 8 forecasting comes out with higher and more realistic 9 risk because they are based on historical data. 10 Q. So just to be sure I'm understanding that, the Monte 11 Carlo software can be used for either the reference 12 class data or the QRA data. The point is that it's the 13 underlying data that's more reliable with reference 14 class forecasting, less affected by bias? 15 A. Yes. So here the same garbage in, garbage out applies 16 as everywhere. So you -- if you put garbage in to 17 a Monte Carlo simulation, you get garbage out. 18 Unfortunately that's what happens in the majority of 19 cases. 20 Q. What I would like to do now is look at your -- return to 21 look at your report and a particular passage in this. 22 The reference is TRI00000265. If we could look at 23 page 12 of that. 24 If we could enlarge the upper half of the screen, 25 please. I want to ask you about the second paragraph, 33 1 first of all. You refer there to research by Kahneman 2 and Tversky. I think that's Kahneman you mentioned 3 earlier was a Nobel Prize winner for his work in these 4 fields: 5 "... argue that the prevalent tendency to 6 underweight or ignore distributional information is 7 perhaps the major source of error in forecasting. 8 Planners should therefore make every effort to frame the 9 forecasting problem so as to facilitate utilising all 10 the distributional information that is available." 11 What do you mean by "distributional information"? 12 A. So distributional information means variation. It means 13 uncertainty basically. So when you plan a project, you 14 are doing something that will happen in the future, and 15 things in the future don't happen with certainty. 16 There's an uncertainty. More things can happen than 17 will actually happen. So there's a range of things that 18 could happen, but only one of them will happen in each 19 instance. 20 That's what uncertainty is, and distribution 21 information is information about that uncertainty, and 22 what Kahneman and Tversky say here is that -- 23 Q. Could I ask you to slow down a little bit, because it's 24 going to become a bit difficult. 25 A. What Kahneman and Tversky say here is that that 34 1 distributional information is often, even typically, not 2 taken into account, and the Monte Carlo simulations that 3 we talked about earlier is such an example where all 4 distributional information is not taken into account, 5 and that's where the results are unreliable or even 6 misleading. 7 Q. How does the reference class forecasting enable that 8 information to be taken into account, or does it take it 9 into account? 10 A. It does take it into account, and this is because it 11 takes into account anything that has happened to the 12 historical projects. So anything that happens is taken 13 into account, and if you have a group of, you know, 60 14 plus light rail projects like we do in our report, then 15 you know what happened to each of these, and if we look 16 at schedule or cost or whatever it is that we want to 17 look at, we know what the variation is. You know, 18 sometimes, the costs come in at this level. Sometimes 19 it comes in at that level, and there's a whole range of 20 values. 21 Instead of us assuming or guessing at what would the 22 value be, we have the whole range of value. 23 So that's what Kahneman and Tversky mean when they 24 talk about all the distributional information available, 25 and that is what reference class forecasting delivers 35 1 you. 2 That's why it's no coincidence that the term 3 "reference class forecasting" was first coined by Daniel 4 Kahneman, actually. 5 Q. That's the point we discussed earlier. It is capturing 6 things which might not or will not be included in the 7 QRA? 8 A. Exactly. 9 Q. It's capturing a broader spectrum of information? 10 A. Yes. 11 CHAIR OF THE INQUIRY: Does it capture the uncertainty that 12 you spoke about in relation to the future, so that right 13 at the start of a project, making certain assumptions 14 about what will happen in the future, there's 15 uncertainty about that, but once the project is 16 concluded, then that -- what actually happened is taken 17 into the calculation of the ultimate cost of the 18 project? 19 A. It gives you the best estimate of the uncertainty that 20 you can get, but you might actually have a situation 21 where the project that you are doing is such an extreme 22 outlier that it's not captured by the distribution in 23 the reference data. So you have even larger uncertainty 24 that materialised. But that still doesn't mean that the 25 estimate that you get from the reference class would not 36 1 still be the most reliable estimate of uncertainty you 2 can get. 3 So this is what has been proven, is that the 4 estimate of uncertainty that you get by using the 5 historical data from the reference class is the most 6 accurate estimate of uncertainty that you can get, as 7 long as you have data from a minimum of 20 to 30 8 projects. 9 CHAIR OF THE INQUIRY: You spoke about an extreme outlier. 10 Is that the black swan or is that something else? 11 A. No, same thing. 12 CHAIR OF THE INQUIRY: Yes. 13 MR LAKE: If I can just pick up on something in your answer 14 to Lord Hardie's question, you said it's the best, most 15 reliable source of predicting, if you've got data from 16 20 to 30 projects. Does the reliability get better as 17 the number of projects increases? 18 A. Yes. So that's simple statistics. The more 19 observations you have, the more reliable an outcome you 20 get. 21 So yes. 22 Q. Of course projects have been done all the time that 23 could be added to your reference class, be it heavy 24 rail, light rail, motorways, hospitals? 25 A. Correct. 37 1 Q. So, for example, every ten years, would it be useful to 2 look again at the reference classes and recalculate 3 them? 4 A. Yes. 5 Q. I think you do in fact, at the Said Business School, 6 have -- compiled data in the form of reference classes? 7 A. Correct. 8 Q. That is a business in which you are involved through the 9 university? 10 A. Yes. That is a business. 11 Q. Just turning now to the foot of page 12 that we see on 12 screen, if we can look to just above the numbered 13 paragraphs indented, you note there that: 14 "Reference class forecasting follows three steps: 15 "1. Identify a sample of past, similar projects - 16 typically a minimum of 20 to 30 projects are enough to 17 get started, but the more projects the better ..." 18 Obviously we see there the figure of 20 to 30 we 19 have just been talking about. 20 What I want to ask you is about the similarity of 21 the projects. The assumption or the research has shown 22 that projects of a similar nature tend to produce 23 a similar outcome. But what is it has to be similar? 24 Is it the nature of the work undertaken? The problems? 25 The funding structures? What similarity do you look for 38 1 in this context? 2 A. So first of all, there is the same type of project. So 3 light rail projects could be compared to each other. 4 That's the first criterion. 5 But also that they are actually delivered under 6 similar regulatory regimes, including the type of law 7 and regulations that the projects have to adhere by. 8 So you probably wouldn't take a light rail scheme 9 from India if we were planning a light rail scheme here 10 in Scotland. You would say most likely that is not 11 comparable. We would be suspicious of that, and we 12 would actually do statistical tests in order to verify 13 whether that should be included or not. But most 14 likely, we would just make a decision that it's not 15 a comparable geography. The laws, the economy, the 16 logistics, the level of corruption and so on are 17 different. And therefore we should not use data from 18 that kind of setting for this kind of setting. 19 So those are the kinds of considerations that you go 20 through when you establish the reference class, and then 21 you do very rigorous statistical analysis. It's 22 actually quite easy by rigorous statistical analysis to 23 decide whether projects are similar or not. So that's 24 how you handle the concept of similarity in establishing 25 the reference class. 39 1 Q. It may seem like an obvious thing to say, but if you do 2 not choose the correct class of projects, you're not 3 going to get the correct optimism bias uplift? 4 A. That is correct. And that -- many people think that 5 that means that you have to be careful about what you 6 include. And that is true, you have to be careful about 7 that. But Daniel Kahneman has pointed out you have to 8 be even more careful about -- you throw out, because we 9 again, humans, have another bias in addition to the 10 optimism bias that is called uniqueness bias, that we 11 tend to think of projects as more unique than they 12 actually are. 13 That means that we are discarding data from other 14 projects that could actually be highly useful for us. 15 We say: no, this is not comparable because our project 16 is unique and therefore we can't compare to these other 17 projects. So Kahneman has argued that we are actually 18 erring on the side of not including enough in the 19 reference class, instead of including too much. 20 So that's the balance you need to strike when 21 you're -- in the considerations that people go through 22 when they are establishing reference classes. 23 CHAIR OF THE INQUIRY: What about the type of project that's 24 in a reference class, I'm thinking of planning or 25 regulations or use classes -- you have a series of 40 1 different uses that are within the same use class. 2 You've mentioned that you would compare a light rail 3 project with a light rail project, assuming the 4 circumstances were similar. Would you look at other 5 projects outside light rail projects, for instance, 6 heavy rail projects, to -- as part of the same class or 7 would they be excluded? 8 A. So we actually tested for that, and it turns out, 9 and I think this is mentioned in the report, that heavy 10 rail projects, as you mentioned, are statistically 11 similar to light rail projects, and therefore, 12 a statistician would tell you that these ought to be 13 included in the reference class to give you more 14 reliable results. So you should actually pool light 15 rail and heavy rail in this case. 16 We even had an example, an extreme example that we 17 had one statistician tell us: look at your data, opera 18 houses are performing the same way as light rail 19 projects, so you should include opera houses. We 20 said: no, we are not going there because we won't be 21 able to justify that. But from a statistician's point 22 of view, if things are similar statistically, they 23 should be proved. 24 CHAIR OF THE INQUIRY: I think you mentioned to Mr Lake that 25 you would advocate or favour a review every ten years or 41 1 perhaps even sooner of the material that you have 2 available. 3 Would that be information that should inform the 4 Treasury about its guidance to other public bodies? 5 A. Yes. 6 CHAIR OF THE INQUIRY: So does that mean that you would 7 envisage the Treasury reviewing its guidance on 8 a regular basis and if so, how often? 9 A. So obviously it's not my -- I'm not in a position to 10 tell the Treasury what to do. But hypothetically, if 11 I was asked, I would say yes, the data needs to be 12 revised from time to time. Preferably continuously 13 actually, because every time a new project has been 14 completed, we have additional useful data that should go 15 into the reference class. But I would say at a minimum 16 every ten years, preferably every five years. 17 CHAIR OF THE INQUIRY: Thank you. 18 MR LAKE: Turning back to your report now, and looking at 19 the second paragraph, the second step in reference class 20 forecasting, you say: 21 "Establish the risk of the variable in question 22 based on these projects - eg identify the cost overruns 23 of these projects." 24 Is that essentially taking the data from the 25 reference class? 42 1 A. It is, yes, establishing -- you basically look at the 2 risk of each project and you plot them in 3 a distribution. That's the second stage. 4 Q. Then the third is: 5 "Adjust the current estimate - through an uplift or by 6 asking whether the project at hand is more or less risky 7 than projects in the reference class, resulting in an 8 adjusted uplift." 9 Now, two parts to that. Firstly, simply applying 10 the uplift, that's essentially arithmetical? 11 A. Yes. 12 Q. And no judgement involved there at all? 13 A. Correct. 14 Q. Now, the second part of that clearly does involve some 15 judgement because someone might ask themselves, is in 16 fact our project less risky than the reference class? 17 A. Correct. 18 Q. Now, that seems to be coming back to what you have 19 described as the uniqueness bias. 20 A. Yes, it could be. That's where the uniqueness bias and 21 optimism could re-enter the whole exercise. 22 Q. I think you go on to deal with that in the following 23 paragraph, where you say: 24 "The final step in the reference class forecasting 25 process considers whether the project at hand is more or 43 1 less risky than the projects in the reference class. It 2 should be stressed that unless this consideration is 3 based on objective evidence, optimism bias might be 4 re-introduced into estimates." 5 If we scroll further down the page: 6 "In addition, planners might consider downward 7 adjustments of optimism bias uplifts based on capability 8 assessment, eg ability to identify and address risks 9 early or commercial structures established with a goal 10 to transfer risk as much as possible. These assessments 11 are all subjective, because they are untested at the 12 planning stage." 13 In that last paragraph, it is not so much assuming 14 that the project is unique, but it is assuming certain 15 qualities or capabilities of either the project 16 structure or the people who are administering the 17 project. 18 A. Correct. 19 Q. Now, that once again seems very much a subjective 20 judgement. People are judging themselves and their 21 project to say: we are better than everything in the 22 class reference. 23 A. Correct. 24 Q. That seems to be very -- to the layman to be very 25 fertile ground for distorted judgement. 44 1 A. Correct. 2 Q. That being so, why is it even in the third step 3 a possibility left open that the actor might ask whether 4 they are more -- that they are less risky than the 5 projects in the reference class? 6 A. You might come across a client or a project team that 7 has a documented track record of being better than 8 average. So they have delivered one success after 9 another, and it's actually documented that that is the 10 case, that they deliver on budget and they have done it 11 now ten times in a row. 12 If you have that kind of evidence, then it actually 13 would be a mistake to apply the average, the expected 14 value from the whole reference class because this team 15 or this client is clearly better than the project in the 16 reference class. I have to say, we haven't come across 17 this. What usually happens when we do reference class 18 forecasting, and enter these considerations with our 19 standard part of the format, is that the people respond: 20 we are no better and no worse than the reference class. 21 That has happened every time, and this is easy to 22 understand because it's basically the same organisations 23 and the same methodologies that are being used in the 24 conventional approach. 25 So it's the same people who have produced the risk 45 1 estimates and the budgets and so on for the project that 2 is being planned, as for the projects in the reference 3 class. That's the explanation that we get this answer. 4 So usually we actually do use the average, the 5 expected value as the uplift. 6 Q. But in the first paragraph after the numbered 7 paragraphs, you refer to whether or not the 8 consideration, that's the consideration of whether or 9 not it's less risky, is based on objective evidence. 10 You gave the example that an entity is able to 11 demonstrate its project team have delivered project 12 after project at an overrun less than the reference 13 class. Is that the sort of thing you are looking for 14 for objective evidence, it must be -- 15 A. Correct. 16 Q. -- concrete data? 17 A. Yes, that is what we are looking for, concrete data. 18 Q. Not a matter of the opinion of anyone involved in the 19 project that they have a structure which will make 20 things better. 21 A. Correct. 22 Q. So if someone were to say: well, we have a novel 23 procurement approach which is going to reduce risk; you 24 would not regard that as a reason to reduce the optimism 25 bias uplift? 46 1 A. Not unless there was a historical track record that that 2 procurement approach actually led to better results. 3 Q. I understand, thank you. 4 I want to now turn to ask you some questions about 5 the guidance that was available at the time the decision 6 to enter the tram project in 2007/2008 was made. 7 It has been considered in your report. But what 8 I would like to do is look at some of the actual 9 underlying guidance documents. If you could firstly 10 look at CEC02084689. 11 I take it you recognise this document of course? 12 A. I do. 13 Q. This is guidance that was provided by Mott MacDonald to 14 the Treasury in July 2002 as a review of large public 15 procurement in the United Kingdom. 16 Do you see that? 17 A. Yes. 18 Q. Now, this made certain recommendations as to the uplifts 19 that might be made for optimism bias and the methodology 20 for making those uplifts? 21 A. Correct. 22 Q. But this was made in 2002. 23 Now, dealing with how, firstly, looking at today's 24 date, some 15, 16 years on, do you consider that this 25 guidance is still valid, or has it lost some of its 47 1 validity? 2 A. It has lost some of its validity in my view. 3 Q. In which particular areas is it of less use now than it 4 once was? 5 A. I would say that the data are dated. So this is now 6 16 years ago or 15/16 years ago, and that's a long time, 7 and we know much more today than we did then. 8 This is an important first document. It was a very 9 important first step in dealing with the types of issues 10 we are discussing here, but a lot of progress has been 11 made since then, and therefore the report is now 12 outdated. 13 Q. In terms of the underlying data, and looking even at the 14 time, 2007/2008, is it the case that there was new data 15 even by that time, 2008, including data that you had 16 provided? 17 A. Yes, in fact, the Mott MacDonald report triggered 18 another report in 2004 that I was involved in, and in 19 2003 it was the basis for the Green Book from the 20 Treasury. So a lot of stuff happened immediately after 21 the Mott MacDonald report, and I would say already in 22 2003 and 2004, better data were available. 23 Q. If we nonetheless go and see what this report has to say 24 about the calculation of optimism bias and look at 25 page 40. 48 1 If we start by looking at the upper half of the 2 page. Under the heading, "Calculation of Optimism 3 Bias", it states: 4 "This section describes how to calculate the 5 optimism bias for the estimated project costs and time. 6 The calculated optimism bias will be used to replace the 7 risk element in the 6 per cent discount rate, formerly 8 recommended by HM Treasury. When calculating optimism 9 bias experienced appraisers should apply a degree of 10 best judgement." 11 I just wanted to ask you really about that last 12 sentence in that first paragraph standing out to suggest 13 that judgement is used in calculating optimism bias. 14 Is that a fair comment, or is the idea of optimism 15 bias really to try and reduce or even eliminate issues 16 of judgement? 17 A. Well, at this stage the data were very rudimentary, and 18 I believe that this is one of the reasons that they 19 formulated it softly like this, that they know that they 20 actually in this report are not providing a very 21 rigorous set of data, and therefore anybody using that 22 data will have to, you know, use a certain level of 23 judgement. 24 For me this is a cautionary sentence, and it's 25 appropriate, given the status of the data, whereas what 49 1 we talked about earlier today are much more rigorous 2 data. 3 So today and already in 2004, much more rigorous 4 data were available where you would do the kind of 5 considerations that we have talked about, where you 6 would say okay, we can actually apply an expected value 7 based on the historical data where we are not using any 8 other judgement than saying that this is the most -- this 9 is the best estimate we can get of uncertainty. 10 So that's how I would explain the difference between 11 this and later reports. 12 Q. What I want to do now, we can see there's a table on the 13 screen in front of us. Those are the figures that were 14 provided by this report, but to put them in context, if 15 we can look at what the paragraph before it says, it 16 notes that: 17 "When carrying out project appraisals, full 18 allowance should be given for any suspected optimism in 19 the costs and time figures originally proposed, giving 20 regard to the outcomes of previous projects of a similar 21 nature. By accounting for optimism more explicitly, 22 project options can be compared more accurately with 23 regard to costs and times. Table 4 ..." 24 Which is the one we can see on screen: 25 "... provides indicative figures for optimism bias. 50 1 It has been prepared by taking into consideration the 2 results of the Mott MacDonald study and reductions in 3 optimism bias levels observed over recent years to 4 provide an upper bound for optimism bias. The lower 5 bound in Table 4 allows for improvements in practice 6 that were evident over the review period and new 7 procurement practices known to have been implemented in 8 the last five years." 9 Now, if we look at the table following that 10 explanation, we can see that the categories that are 11 given, and there's just six of them, are really very 12 broad indeed. 13 We don't, for example, have light rail or heavy rail 14 or roads. We've just got standard civil engineering, 15 non-standard civil engineering. 16 Did that affect the reliability of these estimates? 17 A. When you say these estimates, do you mean the estimates 18 in this table? 19 Q. Yes. 20 A. Okay. Yes, it does. And again, Mott MacDonald didn't 21 have much data at the time they did this report, and 22 they therefore had to use very simple categories and 23 very aggregate categories like they do here, and that 24 will affect the data, and even more will affect the 25 validity and reliability in using the data on specific 51 1 types of projects like light rail. 2 Q. If that's the data, looking underneath the table to see 3 how it's suggested it be used, it says: 4 "The upper bound values recommended for use when 5 calculating optimism bias represent the optimism bias 6 level to expect for current projects without effective 7 risk management and bad scope definition, and are the 8 starting point for calculating optimism bias for 9 projects. These upper bound values reflect the average 10 historic values because the average historic values are 11 similar to the highest values for optimism bias 12 currently being reported for recently completed projects 13 that have experienced high levels of optimism in their 14 project estimates." 15 Now, this seems to be suggesting that you use the 16 upper level when there is no effective risk management 17 and bad scope definition. 18 A. Yes. 19 Q. That seems to suggest that if you have any form of risk 20 management and bad scope definition, you should 21 immediately start reducing from the figures given in the 22 table? 23 A. Correct. 24 Q. Now, that was obviously the understanding at the time. 25 Is that still the understanding now? 52 1 A. I would say today you wouldn't use this approach because 2 there is so much better data available. 3 Q. Right. If we could look at the following page, page 41, 4 it considers the various steps that are involved. 5 Reading through the middle of the page, it notes: 6 "To calculate the optimism bias for project 7 estimates during a project appraisal: 8 "1. Decide which project type is appropriate for 9 the project being appraised." 10 That is simply which one of those six lines on the 11 table it is felt most closely corresponds; is that 12 correct? 13 A. Correct. 14 Q. In the paragraph afterwards it notes: 15 "For the ease of determining a project type for 16 building and civil engineering projects, a project is 17 considered "non-standard" if it satisfies any of the 18 following conditions: (a) it is innovative and/or 19 unique; or (b) construction involves a high degree of 20 complexity and/or difficulty." 21 Do you have any comment on that as being the 22 deciding factor between what is standard and what is 23 non-standard? 24 A. No, I would agree with this as a definition of 25 non-standard. 53 1 Q. If we go over the page to the second step, it then says: 2 "Use the appropriate upper bound value for optimism 3 bias from Table 4 as the starting value ..." 4 That's obviously the very highest one we have seen. 5 The following step, number 3, is: 6 "Reduce this upper bound optimism bias according to 7 the extent to which the project risk areas are managed. 8 The project risks within each project risk area can be 9 managed. If the project risk areas for a project have 10 only been partially mitigated then the contribution to 11 optimism bias can be reduced proportionally to reflect 12 the amount that each project risk area has been 13 mitigated." 14 It goes on to explain that in more detail. 15 This seems to be calling for an exercise of 16 subjective judgement? 17 A. It does. I mean, this is a wide open door for having, 18 you know, a self-defeating result. So supposedly you 19 are trying here to take optimism bias into account, but 20 as part of the process, you have actually steps that 21 would reintroduce optimism bias. 22 Q. The table gives an upper level and a lower level, and 23 essentially invites the project people to exercise 24 a subjective judgement about where in that band they 25 fall? 54 1 A. Correct. 2 Q. Do you consider that accurately gives effect to the 3 purposes of reference class forecasting? 4 A. No. 5 Q. If we could then go to a different document of guidance. 6 It's the Green Book, Treasury Green Book. It's 7 reference CEC02084256. 8 You can see the heading here, "THE GREEN BOOK 9 Appraisal and Evaluation in Central Government". The 10 box underneath it, it's not very clear, but we can just 11 about read: 12 "Note explaining changes made to the Green Book in 13 July 2011. 14 "This is the 2003 edition of the Green Book. 15 However, pages 57 to 58 which deal with the valuation of 16 non-market goods have been updated alongside the release 17 of a Green Book discussion paper on this subject." 18 Now, I'm happy to say that we are not dealing with 19 pages 57 and 58. So is this in essence the guidance 20 that was available in 2003? 21 A. Yes. 22 Q. Could we go and look at page 33 of this document on 23 screen, and look at the lower half of the screen. 24 We have got the big heading, "ADJUSTING FOR BIAS AND 25 RISKS", and the sub-heading, "Optimism bias", under 55 1 which paragraph 5.62 says: 2 "To redress this tendency ..." 3 That's the optimism tendency: 4 "... appraisers should make explicit adjustments for 5 this bias. These will take the form of increasing 6 estimates of the costs and decreasing, and delaying the 7 receipt of, estimated benefits. Sensitivity analysis 8 should be used to test assumptions about operating costs 9 and expected benefits." 10 The following paragraph is: 11 "Adjustments should be empirically based (eg using 12 data from past projects or similar projects elsewhere), 13 and adjusted for the unique characteristics of the 14 project in hand. Cross-departmental guidance for 15 generic project categories is available, and should be 16 used in the absence of more specific evidence. But if 17 departments or agencies have a more robust evidence base 18 for cost overruns and other instances of bias, this 19 evidence should be used in preference. When such 20 information is not available, departments are encouraged 21 to collect data to inform their estimates of optimism, 22 and in the meantime use the available data that best 23 fits the case in hand." 24 I'm just trying to follow through really what has 25 been said in that last paragraph. 56 1 It has to be an adjustment made for optimism bias. 2 And the Green Book, the guidance says it must be 3 empirically based. But invites the reader to use data 4 from past projects or similar projects elsewhere, as 5 adjusted for the unique characteristics of the project 6 in hand. 7 Now, from our discussions that we have had this 8 morning, that once again seems to be an open door to 9 optimism bias coming back in when correcting optimism 10 bias. 11 A. Yes, the very last thing about adjusting for the 12 uniqueness of the project, that's the open door. 13 Q. In terms of basing -- making adjustments empirically, 14 using data from past projects, that in itself wouldn't 15 give rise to bias, provided the class of projects is 16 properly selected? 17 A. Correct. 18 Q. So if you look at something, for example, in published 19 guidance, or something that's provided in the service 20 that you're able to provide projects, that should not be 21 biased? 22 A. Correct. 23 Q. But if a project team decide to start going out and 24 choosing which projects they are going to look at, both 25 inclusion and exclusion, that gives rise to the sort of 57 1 bias again you have discussed this morning? 2 A. Correct. 3 Q. So if there is to be reference class forecasting, it's 4 important that the reference classes have a degree of 5 independence to have validity? 6 A. Correct. 7 Q. Looking at paragraph 5.64, it notes that: 8 "Adjusting for optimism should provide a better 9 estimate, earlier on, of key project parameters. 10 Enforcing these adjustments for optimism bias is 11 designed to complement and encourage, rather than 12 replace, existing good practice, in terms of calculating 13 project specific risk adjustments. They are also 14 designed to encourage more accurate costing. 15 Accordingly, adjustments for optimism may be reduced as 16 more reliable estimates of relevant costs are built up, 17 and project specific risk work is undertaken. Both cost 18 estimates and adjustments for optimism should be 19 independently reviewed before decisions are taken. 20 Annex 4 provides further detail on how to deal with 21 optimism bias." 22 Now, we will come to Annex 4 in a moment, but just 23 looking at what was in that paragraph, it's saying that 24 both cost estimates and adjustments for optimism should 25 be independently reviewed. What degree of safeguard 58 1 does that independent review provide to either estimates 2 of risk or estimates of optimism bias? 3 A. It's always good to get an independent review of things 4 like this. However, if the independent reviewers are 5 using a methodology for the review, that is open to 6 bias, then, you know, it's not going to be a great help. 7 So you actually need an independent review that is using 8 historical data in order to assess whether the 9 assessment that the project team comes up with is 10 realistic. 11 Q. Otherwise in a sense you are just going to make the same 12 mistakes inherent in a QRA all over again? 13 A. Exactly, and to be frank, we often see that with 14 independent reviews, that they don't bring much 15 additional value because they follow the same procedure 16 and they use the same data as the project team, and 17 therefore doesn't come up with different results. 18 Q. Now, accepting that problem with the risk data, even if 19 it's been the subject of external assessment or 20 verification, halfway through that paragraph, it says 21 that the adjustments for optimism may be reduced as more 22 reliable estimates of relevant costs are built up, and 23 project specific risk work is undertaken. 24 Now, that is suggesting as you build up your QRA, 25 you reduce your optimism bias. 59 1 A. Correct. 2 Q. But the QRA is going to contain optimism bias? 3 A. Correct. 4 Q. So is it a valid approach to say, as we build up, as we 5 move to finer detail on our analysis of risk, we should 6 reduce our optimism bias allowance? 7 A. I would say no. 8 Q. Why not? 9 A. Because there's an assumption that you are actually -- 10 you are reducing risk as you do this, and there's 11 an assumption that the more detail you have, the less 12 risk you have, and those are just assumptions, and they 13 need to be verified in order to actually -- for you to 14 justify reducing optimism bias uplifts, and they are not 15 empirically verified in most cases that I have seen. 16 So therefore this combination is really a trap in 17 the sense that it will lead the risk assessment team 18 back into biased -- and underestimated risks. 19 Q. There was reference there to Annex 4. If you could look 20 at that, please. It begins on page 83. 21 This is just to identify that it's Annex 4 we are 22 looking at, headed "Risk and Uncertainty". If we go 23 forward to the optimism bias part on page 89, we can see 24 under the heading "Capital costs", if we go to 25 paragraph 20, it notes that: 60 1 "Appraisers should adjust for optimism bias in the 2 estimates of capital costs in the following way: 3 "Estimate the capital costs of each option; 4 "Apply adjustments to these estimates based on the 5 best available empirical evidence relevant to the stage 6 of the appraisal; and 7 "Subsequently, reduce these adjustments according to 8 the extent of confidence in the capital costs' 9 estimates, the extent of management of generic risks, 10 and the extent of work undertaken to identify and 11 mitigate project specific risks." 12 Just dealing with each of these three in turn, 13 firstly, the first stage, estimating capital costs, 14 I take it there's no objection with that. That's always 15 a necessary first stage? 16 A. Correct. 17 Q. The second one, applying adjustments to these estimates, 18 that's the cost estimates, based on the best available 19 empirical evidence, if that best available empirical 20 evidence is the reference class data that you've 21 discussed with us, do you regard that as part of a good 22 approach also? 23 A. Yes. 24 Q. The third one, however, reducing those adjustments 25 according to confidence in capital cost estimates, is 61 1 that going to cause problems? 2 A. It's likely to cause problems, and what you would need 3 here is a sort of scepticism. Every time it's suggested 4 that you would reduce the optimism bias uplift, somebody 5 needs to play the role of the sceptic and ask for solid 6 evidence why it should be done. Otherwise this is 7 likely to result in optimism bias uplifts that are too 8 low. 9 Q. Just because there's been, to use the wording at the end 10 of that bullet point, work undertaken to identify and 11 mitigate project specific risks, presumably that will 12 happen in every project? 13 A. Yes. 14 Q. And if we start reducing optimism bias uplifts on that 15 basis, will it provide a distorted view? 16 A. In my view, yes. 17 Q. So would it be fair to say on that basis that this 18 guidance is no longer likely to be helpful for the 19 proper management of project costs? 20 A. I would say on this specific point, the guidance is 21 tripping itself up. 22 Q. Looking at the following paragraph, paragraph 21, the 23 guidance states: 24 "Departments or agencies may be able to provide the 25 best empirical evidence to support adjustments for 62 1 optimism. Alternatively, and if applicable, they may be 2 taken from the Green Book home page, which provides the 3 recommended adjustments to be made at the outline 4 Business Case stage for buildings, civil engineering, 5 equipment and development, and outsourcing projects." 6 That guidance, was that the guidance derived from 7 the Mott MacDonald report that was being provided? 8 A. I would have to look at the Green Book home page to see 9 whether that's the case, but I would assume so since 10 nothing else was available at the time. 11 Q. What is noted here is that the intention is that that 12 begins to be applied at the outline Business Case stage. 13 A. Sorry, say again? 14 Q. That uplift would be applied at the stage of the outline 15 Business Case. 16 A. Yes, that's what it says under point 21. 17 Q. Is there any significance to the fact it's specifically 18 to be applied at that stage? 19 A. Well, I guess this is to make sure that optimism bias 20 uplifts are introduced as early in the planning of the 21 project as possible. You can apply uplift and you 22 actually should apply uplifts at each of the different 23 stages of the project. 24 Q. I want to come to ask you a bit more about these stages. 25 We can perhaps do it by reference to the next piece of 63 1 guidance, which is the Department for Transport guidance 2 on procedures for dealing with optimism bias in 3 transport planning. That has reference CEC02084257. 4 We can see the title I gave there on the front page 5 of the report; is that right? 6 A. Correct. 7 Q. The date given is June 2004. 8 A. Yes. 9 Q. I take it not only do you recognise this document, you 10 were one of the authors of it? 11 A. Correct. 12 Q. Could we go first to page 11. If you could enlarge the 13 upper part of the page. 14 Reading from the third paragraph there, the report 15 says: 16 "Taking an outside view requires the following steps 17 for the individual project: 18 "Identification of a relevant reference class of 19 past projects. The key is here that the class is broad 20 enough to be statistically meaningful but narrow enough 21 to be truly comparable with the specific project." 22 Now, introducing the idea of the project being 23 narrow enough is you sought with the additional data you 24 had to narrow down the classes, so you didn't really 25 have standard civil engineering, you had particular 64 1 types of civil engineering project? 2 A. Yes. 3 Q. The second bullet point is: 4 "Establishing a probability distribution for the 5 selected reference class. This requires access to 6 credible data on cost increases (or time schedule delays 7 or benefit shortfalls if these are the key parameter) on 8 a sufficient number of projects within the reference 9 class to make statistically meaningful conclusions 10 (normally at least 10)." 11 Obviously that 10 number is rather less than the 20 12 to 30 you were talking a few minutes ago? 13 A. Correct. So today we know much more about this than we 14 did at the time. We have many more data now, which 15 means that we understand the specific distributions of 16 the data, and the number of observations that you need 17 in order to reliably establish the distribution varies 18 according to which distribution it is. 19 So now that we have the detailed information, we 20 know that because of the many outliers, that the -- we 21 know that the mean is less robust and less reliable in 22 these types of distribution, which is why you need more 23 observations. 24 So that's why we have adjusted it up over time, 25 because we now know that the distributions are such that 65 1 10 is actually not enough to get a reliable estimate. 2 Q. But also the significant thing here is you have -- this 3 guidance seeks to provide the probability distribution 4 for the first time. Because that wasn't provided by 5 Mott MacDonald. But you did seek to provide the various 6 P50, P60, P70 probabilities, so that the appropriate 7 level of certainty could be selected? 8 A. Correct. 9 Q. The third bullet point there is: 10 "Placing the specific project at an appropriate 11 point in the reference class distribution." 12 You note: 13 "This step has an element of intuitive assessment 14 and is therefore liable to optimism bias." 15 I think you go on to explain later how that must be 16 controlled to avoid undermining the whole process? 17 A. Correct. 18 Q. If you go to page 14, I should say, whenever I give page 19 references, I'm referring to the electronic version. 20 You will see the page reference on screen is different, 21 but I will always use the electronic one. The second 22 last paragraph on this page states: 23 "At the same time it is important to recognise that 24 the establishment of budgets which on average are more 25 than adequate (as would be the case if uplifts 66 1 reflecting a higher percentile in the distribution than 2 the 50 per cent percentile is applied) may have an 3 incentive effect which works against tight cost control 4 if the more than adequate budget is available (or 5 perceived as being available) to the project 6 organisation." 7 Is that the problem we discussed earlier that, if 8 the money is there, people will want to feed on it? 9 A. Correct. 10 Q. That's why you have to have the control such as we have 11 already discussed? 12 A. Correct. 13 Q. Or contract structures to incentivise cost control? 14 A. Correct. 15 Q. Turning then to page 19, this is under the heading of 16 "Benchmarking of optimism bias in Britain". If we look 17 at paragraph 3.1.1 with the heading "Sampling and data 18 collection", you say there: 19 "Cost overrun is here defined as the difference 20 between actual and estimated costs in percentage of 21 estimated costs, with all costs calculated in constant 22 prices. Actual costs are defined as real, accounted 23 costs determined at the time of completing a project 24 (outturn costs). Estimated costs are defined as 25 budgeted, or forecasted, costs at the time of approval 67 1 of/decision to build a project, which is typically 2 similar to costs as presented in the Business Case for 3 a project." 4 Now, just looking here, you're choosing the estimate 5 at the time when the decision to build or to progress is 6 made. Was there a particular reason to choose that 7 point? 8 A. Yes. So basically you want to know whether 9 decision-makers were well informed when they made their 10 decision, and you want them to have the right 11 information at that stage. 12 So that's why we're choosing the time of making the 13 decision as our baseline. You could choose other 14 baseline. You can choose any which baseline you want. 15 You can choose a much earlier one like a pre-feasibility 16 study, or you could choose another one, where you had 17 the tendering for the projects, and we had data to do 18 all these different baselines. But this is what we 19 consider, and not only us but the international 20 convention is when you measure these things, that you 21 actually use the date of the decision, the final 22 decision to do a project, because you want to know 23 whether the decision-makers were well informed or not. 24 It's really the most crucial point in a project. 25 When the project is given the green light to go ahead, 68 1 or when it's given the red light not to go ahead. So 2 that's the point we use as the baseline. 3 Q. For your answer there, it presupposed that the decision 4 to go ahead, the green light would be given, and later 5 there would be tendering. 6 A. Yes. That's the assumption. 7 Q. In the tram project the green light was given right at 8 the very last minute, after -- the final green light, it 9 might be said, was given after the tendering had taken 10 place. 11 A. It's not a problem, what I'm saying -- it is the 12 assumption, I should actually say more accurate, that 13 this is the way it usually -- it usually happens. But 14 if the final decision to build it was after the tender, 15 you still use it as the baseline, because in theory the 16 decision-makers could decide not to do the project after 17 the tender. So no final commitment has been made yet. 18 And we are looking for the baseline, where the final 19 commitment is made. 20 Q. When we come, however, to look at what the percentage 21 uplift should be at the time of decision made, do you 22 have to make some allowance for how much information 23 will be available, because if that decision is taking 24 place pre tender, there will be presumably much less 25 information than when that decision is made post tender? 69 1 A. Yes, but that is taken into account. So in the 2 reference class, that would be included in the numbers 3 that you have for the different variables. 4 If the numbers were -- are reflecting projects where 5 the tender came after, and sometimes long after, the 6 decision to build there, then that situation would 7 affect the outcome, and presumably you would have more 8 uncertainty, and therefore more variance and higher 9 delays or cost overruns, or whatever, and that therefore 10 would be built into the reference class. 11 Q. So it's not necessary to make a separate allowance for 12 it? 13 A. I wouldn't say so, but if you had the -- if you have the 14 assumption or if you think that -- if you had the 15 hypothesis that this might be something that affects the 16 result, you would simply go through your data on 17 projects and look for similar projects and see whether 18 it does actually affect the outcomes. If it does, then 19 you would have to build your reference class of that 20 kind of project. 21 So if there's a significant difference between 22 projects with that type of project cycle, sequence like 23 that, and the more conventional sequence, then you would 24 have to look for projects that are similar to the 25 project that you are estimating. 70 1 Q. Just to be sure that I'm understanding that, if your 2 decision to proceed is what we might term a late 3 decision to proceed, post tender, you would try and find 4 a reference class consisting of projects where the 5 decision has been taken late at that stage, because that 6 might generate a different risk profile, optimism bias 7 profile? 8 A. Correct, and all other things being equal, the risk 9 should be lower if you wait with making your decision. 10 Q. In the following paragraph, you note that even if the 11 project planning process varies with project type, place 12 and time, it is typically possible to locate for a given 13 project a specific point in the process that may be 14 identified as the time where formal approval was given 15 to build the project: 16 "Usually a cost estimate is available for this point 17 in time. If not, the closest available estimate was 18 used, typically a later estimate resulting in 19 a conservative bias in the measurement of cost overrun." 20 Is the point there that when you're analysing the 21 reference classes, you need to have some sort of fixed 22 date, so the choosing this one, at least you are 23 choosing a date where there normally is a cost available 24 by way of budget, so you've got your starting point for 25 your comparison? 71 1 A. Correct. 2 Q. If you could go then to page 31. Look at the lower half 3 of the page. It's just for the graphical representation 4 of what we've been talking about. You say: 5 "On the basis of the probability distribution for 6 rail shown above, required uplifts may be calculated, as 7 shown in Figure 9. With a willingness to accept 8 a 50 per cent risk for cost overrun in a rail project, 9 the required uplift would be 40 per cent. If the 10 Department for Transport were willing to accept only 11 a 10 per cent risk for cost overrun, then the required 12 uplift would be 6 per cent." 13 I think those figures are apparent if we look at the 14 graph, whereby the -- we've got the chance of overrun 15 along the X axis along the bottom, and the required 16 uplift on the Y axis to the side. So you see the nearer 17 the left-hand side you are, the lower your appetite for 18 risk, the higher the costs will be? 19 A. Correct. 20 Q. Again, that is a graph that has been produced by you as 21 part of this project from the data that you were 22 analysing? 23 A. Correct. 24 CHAIR OF THE INQUIRY: Mr Lake, will you find 25 a convenient -- 72 1 MR LAKE: This would actually be a convenient time. We will 2 be a bit longer with this guidance, so we won't get to 3 the end of it. 4 CHAIR OF THE INQUIRY: We will have a 15-minute break for 5 the benefit of the shorthand writer. 6 We will resume again at 11.25. 7 (11.06 am) 8 (A short break) 9 (11.26 am) 10 CHAIR OF THE INQUIRY: You are still under oath, Professor. 11 A. Yes. 12 CHAIR OF THE INQUIRY: Yes, Mr Lake. 13 MR LAKE: Thank you very much, my Lord. If we could go, 14 please, in the same document back to page 16. Sorry. 15 I'm using the wrong page reference. Page 18, please. 16 If could you enlarge the table we see there, we can 17 see the categorisation of the transport schemes used for 18 this report and see how much more broken down it is into 19 the various categories; rather than civil engineering, 20 standard, civil engineering, non-standard, you've been 21 able to look at the particular types of construction? 22 A. Correct. 23 Q. Is that because you had access to additional sources of 24 data as opposed to those used by Mott MacDonald in 2002? 25 A. Correct. 73 1 Q. Did that mean that it was able to predict more 2 accurately for each individual type of project, rather 3 than having to take a general approach for civil 4 engineering, standard and non-standard? 5 A. Yes. And it turned out that some of the project types 6 that had been lumped together when we ran statistical 7 tests, we could actually see they should not be lumped 8 together because they performed differently even at 9 statistically significantly critical levels. 10 So that a statistician would say that: they cannot 11 be put together, that would create noise instead of 12 information; so therefore we actually, you know, on 13 a scientific basis, we had to take them apart. 14 So for instance, road and rail are statistically 15 significantly different, and therefore should not be put 16 together, to take one example. 17 Q. So it wasn't just you had -- you did it because you 18 could do it. You did it because it yielded better 19 answers? 20 A. Yes. 21 Q. Previously we saw there was the standard and 22 non-standard distinction, and we saw that explained in 23 the Mott MacDonald guidance. Your categorisation has no 24 distinction between, for example, standard rail and 25 non-standard rail or standard road and non-standard 74 1 road. 2 Was that because you looked at it and decided that 3 wasn't statistically significant? 4 A. Yes. 5 Q. So once again the later work, the work that you have 6 done which had the greater analysis and the more 7 information to analyse, is likely to give a better more 8 accurate picture than the earlier guidance? 9 A. Correct. 10 Q. If we go to page 33. Under heading 4.2 you note that: 11 "The uplifts for optimism bias presented above refer 12 to, and should be applied to, estimated budgets at the 13 time of decision to build. The time of decision to 14 build is typically equivalent to the time of presenting 15 the business case for a project with a view to obtaining 16 the go or no-go for that project. The uplifts refer to 17 cost overrun calculated in constant prices." 18 That's what we've already discussed. You have 19 chosen that figure as an easy fixed point to identify in 20 each project; is that correct? 21 A. Not just easy. As the most relevant fixed point, the 22 most relevant baseline for each project. 23 Q. If you skip the following paragraph, but jump to the one 24 beginning "table 6" you have: 25 "Table 6 shows a simple example of how one would use 75 1 the established optimism bias uplifts in practice." 2 Perhaps if we go over the page to look at table 6, 3 if we can enlarge -- that's fine. I'll read the text 4 from the previous page, but it's useful to look at the 5 table, to have the table in mind when reading it: 6 "If, for instance, a group of planners were preparing 7 the business case for a new motorway, and if they or 8 their client had decided that the risk of cost overrun 9 must be less than 20 per cent, then they would use an 10 uplift of 32 per cent on their estimated capital 11 budget." 12 We can see that you get that easily from looking at 13 the table by looking at the roads row, and then there is 14 a number of different percentiles. They can simply 15 choose the 80 per cent and they know that the uplift 16 must be 32 per cent? 17 A. Correct. 18 Q. So this table gives you more detail both in terms of the 19 type of the project and the probability? 20 A. Yes. 21 Q. So as far as rail is concerned here, if we look at the 22 row beneath the roads, we can see that the 50 per cent 23 percentile, I should say, uplift, is 40 per cent, but if 24 you want to get to 80 per cent, it would be necessary to 25 have an uplift of 57 per cent? 76 1 A. Correct. 2 Q. That would be applied at the time of the decision to 3 build? 4 A. Correct. 5 Q. Although you might have to make some -- take into 6 account for the present project, the Edinburgh Tram 7 Project, the fact that that decision was taken later, 8 when more information was available? 9 A. Mm-hm. Yes. 10 Q. Because the reference class that's been used to provide 11 that data is of ones where the decision to -- has been 12 taken earlier? 13 A. Yes. 14 Q. If the decision was to be taken later, is there 15 generally available data as to the uplifts appropriate 16 for a later decision, or is that something if someone 17 was to come to you or other providers of your service, 18 you would be able to provide that information to them? 19 A. Yes, we would. 20 Q. You would be able to provide it? 21 A. Yes, and I assume others would also. I mean, these are 22 data that are available, and if you collect the data 23 from your projects, you will know this. 24 Q. Right. 25 Now, if we just look there in the rail category, 77 1 just to emphasise again, the 40 per cent figure is given 2 for a 50 percentile confidence level. Do you see that? 3 A. Yes. 4 Q. If we go back just briefly to look at the Mott MacDonald 5 paper, that's CEC02084689. And we go to page 40 within 6 that. 7 We can see there -- enlarge the table -- through 8 standard or non-standard civil engineering projects, 9 we've got the upper limit of respectively 66 per cent 10 and 44 per cent, in relation to capital expenditure? 11 A. Correct, yes. 12 Q. So the figures that seem to be quoted for the 13 Mott MacDonald numbers, does it appear from that that 14 they appear to be quoting around the 50 percentile/60 15 percentile level? 16 A. Well, this is unclear in the Mott MacDonald report, what 17 they do precisely in that respect. 18 Q. Mm-hm. So is it -- do you think it is valid to make any 19 general comparison about the number, the recommended 20 uplift, to compare it with the recommended uplifts in 21 your figure and reach a conclusion as to the probability 22 taken by Mott MacDonald or is that not likely to be 23 reliable? 24 A. So what I would suggest here is that the data in the 25 later report, so the data in the 2004 report, are more 78 1 reliable than the Mott MacDonald report. 2 So as soon as the data in the 2004 report were 3 available, in the areas where they could substitute for 4 the Mott MacDonald data, the data in the 2004 report 5 should be used and you should not use the data in the 6 Mott MacDonald report. 7 Q. Right. That's very clear. Thank you. 8 Go to page 35 within -- go back to the DfT advice 9 that you co-authored. Go to page 35 in that which we 10 now have on screen. Look at the second paragraph on the 11 page there. If you scroll down, it's probably useful to 12 have the chart in sight also. You note there: 13 "Similarly, where a project has not yet reached the 14 approval stage, but is at the strategic outline case or 15 outline business case stage, uplifts should be adjusted 16 to reflect this. Typically budget uncertainty is 17 reduced throughout the project cycle from inception over 18 feasibility studies and approval to construction and 19 start of operations (please refer to Chart 1 below)." 20 Just pausing there, we can see that in 21 a diagrammatic way, you indicate that as one goes 22 forward, the level of uncertainty narrows? 23 A. Correct. 24 Q. Going back to the text in the paragraph: 25 "Therefore, in order to arrive at a valid cost 79 1 estimate for a stage prior to the approval stage, the 2 uplifts listed above should generally be adjusted 3 upwards. Conversely, if a project has moved beyond the 4 approval stage to the stage of detailed design or 5 construction, uplifts should normally be adjusted 6 downwards." 7 A. Correct. 8 Q. Just there, you mention specifically detailed design. 9 Presumably the purpose of that is that once one has 10 a detailed design, if that is -- if the detailed design 11 is costed, there's less scope for bias as to the outturn 12 cost? 13 A. Correct. 14 Q. So it's not just that detailed design has been done. It 15 must be that it is what is costed and contained within 16 the contract? 17 A. Correct. 18 Q. Reading below the chart, you say: 19 "In the latter, or post approval, case, one way of 20 adjusting uplifts downwards would be to follow the 21 simple rule that uplifts are reduced at a specific point 22 in time by the same percentage ... of the total budget, 23 which has been spent up to this point, for instance as 24 follows." 25 If we look at the table, we can see that if when 80 1 you've spent 0 per cent of your budget, you give 2 100 per cent of the uplift, if we read down a few rows, 3 once you have spent, say, 30 per cent, you make 4 an adjustment of 70 per cent of the uplift. You take 5 30 per cent of the uplift away? 6 A. Correct. 7 Q. Now, this is obviously a somewhat arbitrary means of 8 reducing it. 9 A. Correct. 10 Q. The suggestion may be that if we move, as we move 11 forward, it's possible to say: well, we have developed 12 the design further; we have further costings; we can 13 just decide that that the risks inherent in the optimism 14 bias are now dealt with and we can reduce it. Is there 15 a problem with that? 16 A. Yes. Like we discussed earlier, that's opening the door 17 to optimism bias again, and our suggestion here to do 18 this more arbitrary and mechanistic approach is actually 19 to keep optimism at bay, and therefore we prefer this to 20 having subjective assessments reintroduced about how 21 much risk you have actually eliminated. 22 Q. Is this approach that you recommend or you set out here 23 one that in your experience is being applied in 24 practice? 25 A. Yes, it is. 81 1 Q. In large projects, infrastructure projects? 2 A. Correct. 3 Q. If we go over the page, to page 36, underneath the table 4 you say: 5 "Using this simple rule of interpolation has the 6 advantage of avoiding the reintroduction of optimism 7 bias for lack of empirical evidence of how much the 8 uplifts should be reduced at key points in the project 9 cycle. The latter information would ideally be 10 available; in reality, often it is not. If the total 11 budget changes over time, from approval to start of 12 operations, as is often the case, the uplifts would have 13 to be recalculated accordingly, whether or not 14 interpolation or a more empirical method is used to 15 adjust uplifts downwards over time." 16 I think you are aware from the Edinburgh Tram 17 Projects, the uplifts were actually reduced over time as 18 things moved from the interim outline Business Case all 19 the way to the Final Business Case? 20 A. Yes. 21 Q. That wasn't done by your interpolation method. It was 22 done by a judgement on the extent to which risk 23 development meant it was no longer necessary in their 24 view? 25 A. Yes. 82 1 Q. Do you consider that was an appropriate approach? 2 A. No. 3 Q. Why not? 4 A. Again, because it opened the door to optimism, and now 5 we can see with hindsight that that is actually what 6 happened, that the assessment of risk became way too 7 optimistic; and in the way I see it, this happened 8 because the project team assumed that their risk 9 mitigation measures and the identification of risks 10 would actually come through. So the mitigation measures 11 would be effective. So they basically assumed they had 12 realistic assessment of risk, and that the mitigation 13 measures for those risks would be effective. 14 But that's an assumption. That's a theory, and only 15 reality, only the delivery of the project would show 16 whether the assumption was right, and now we know today 17 that the assumption was not right. It was too 18 optimistic. And the way I see it, it was too optimistic 19 because it opened the door again for these subjective 20 assessments of the risk, and risks were adjusted 21 downwards much more than they should have been. There 22 should have been a downward adjustment of risk as the 23 project progressed like we've talked about. That is 24 normal, and that is standard practice. You would 25 actually make an error if you didn't do it, but there 83 1 was way too little scepticism and realism in assessing 2 how much the optimism bias should be reduced over time. 3 Q. Your rule of interpolation as set out in the table is 4 significant in that there will be an optimism bias 5 allowance until 100 per cent of the project costs had 6 been spent? 7 A. Yes. 8 Q. So that would continue right through the construction 9 works? 10 A. Yes. 11 Q. Again, in terms of Edinburgh tram, it was said that they 12 considered that optimism bias would be eliminated at the 13 time of the decision to proceed with the project? 14 A. Yes. 15 Q. Is that realistic? 16 A. No. As long as there's a future, there's optimism about 17 the future. 18 Q. If we look down to paragraph 4.4, you say: 19 "The strength of the uplift for optimism bias 20 established above is that they are firmly grounded in 21 empirical probability distributions of cost overrun for 22 different types of transport projects. Thus the uplifts 23 allow true reference forecasting for specific projects 24 under consideration. It is crucial that uplifts be 25 empirically based in this manner, otherwise the risk is 84 1 high of re-introducing optimism and bias in project 2 preparation and decision making." 3 You're stressing there that the uplifts must be 4 empirically based. That's using the reference classes? 5 A. Correct. 6 Q. But is it equally true that -- perhaps I should go on 7 and look at the next paragraph first. You say: 8 "It may be argued that uplifts should be adjusted 9 downward as risk assessment and management improves over 10 time and risks are thus mitigated. It is however our 11 view that planners and forecasters should carry out such 12 downward adjustments of uplifts only when warranted by 13 firm empirical evidence." 14 Again, the point is there's no point in adjusting 15 empirical uplifts by subjective downgrades? 16 A. Correct. 17 Q. You say: 18 "For 70 years, optimism bias has been high and 19 constant for the types of transport projects considered 20 above, with no indication of coming down. With 21 practices of optimism as deep-rooted as this, hard 22 evidence from post-audits would be required to 23 convincingly argue the case that optimism bias is 24 finally coming down. In general, only at such a time 25 when this evidence is available should uplifts be 85 1 reduced correspondingly." 2 Now, this indication of the importance of empirical 3 evidence, in your experience, is that being applied in 4 practice in large projects? 5 A. Yes, in some projects it is. I would say in the 6 majority of projects it isn't, because it's the -- the 7 balance is changing. So more and more projects are 8 beginning to do this around the world, and in the UK. 9 Q. This guidance has been available generally, I take it, 10 since June 2004, and is -- was published -- prepared for 11 and published by the Department for Transport? 12 A. Mm-hm. 13 Q. You have referred there to post audits being required. 14 Are you involved in any capacity for government in 15 carrying out audits of projects once they are complete? 16 A. From time to time, yes. But we are more involved in 17 preparing projects and in the delivery of projects. 18 There's actually not a great interest in post audits. 19 The national audit office here in the UK and the 20 national audit offices in other countries, for instance, 21 Denmark where I'm from originally, are doing this and we 22 work with both of these auditors in developing their 23 methods and data for how to do this, and other auditors, 24 including in Sweden and in the United States, in more 25 countries. 86 1 Q. Do they have a routine programme of auditing projects of 2 a certain size once they are concluded? 3 A. No. These auditors don't have a routine programme. 4 They are doing it more on an ad hoc basis, plus they 5 have been developing frameworks that they recommend to 6 be used in general for auditing projects. 7 The closest we come to a routine programme is the 8 IPA, the Infrastructure and Project Authority, under the 9 Cabinet Office here in the UK. They have been 10 developing a process and methodology for how to do post 11 audits and are carrying out post audits. 12 Q. And the IPA, the infrastructure projects authority, do 13 they publish the findings of their audits? 14 A. They do. 15 Q. I take it they are available to anybody then who wishes 16 to consider them? 17 A. Yes. 18 Q. Do they only look at projects of a certain size? 19 A. They look at major projects, so the larger end of 20 projects, yes. 21 Q. What is required to qualify as a major project? 22 A. Well, this is not like a scientific definition. But 23 I would say generally projects that cost more than 24 GBP100 million. 25 Q. I think it's correct to say they are not just looking at 87 1 necessarily transport projects. They would also look at 2 computing projects and things such as that; is that 3 correct? 4 A. That's correct. They look at government projects as 5 such, and I should also emphasise that the authority 6 didn't exist, and of course their data therefore were 7 not available at the time that we are talking about now, 8 around 2004. 9 CHAIR OF THE INQUIRY: What's the extent of the jurisdiction 10 of the IPA? Does it cover England and Wales and 11 Northern Ireland or -- 12 A. My understanding is that they cover the UK, but you 13 would have to -- or we would have to verify that, to be 14 sure. 15 CHAIR OF THE INQUIRY: So -- thank you. 16 MR LAKE: If we go over to page 37 within your guidance 17 document, and look at the upper part of the page, you 18 say: 19 "Having stated this general rule ..." 20 Which was the cautionary rule about not making 21 adjustments downwards to the uplift figures, you say: 22 "... it must be observed that individual projects 23 may exist where the claims to improved risk mitigation 24 are so strong that downward adjustment of uplifts is 25 warranted in order to avoid double counting. This may 88 1 be the case if advanced risk analysis (eg risk 2 identification workshop and statistical calculations of 3 volume and cost risks for individual project components) 4 has been applied and their results adequately reflected 5 in the established budget." 6 It might be said that the persons involved in the 7 Edinburgh Tram Project had in fact carried out detailed 8 risk analysis. They had also had it externally checked. 9 Is that enough to warrant a downward adjustment of the 10 recommended uplift? 11 A. It is not, and maybe a word is missing here actually, 12 that it should be realistic statistical calculation, 13 because you can have a statistical calculation of the 14 type garbage in, garbage out. 15 Q. Could I ask you to slow down a little? 16 A. Maybe a word is missing here, and that would be 17 "realistic" in front of statistical calculations, 18 because you can have statistical calculations of the 19 type garbage in, garbage out, that we talked about 20 earlier. And you would have an unrealistic assessment. 21 That would not justify a downward adjustment of the 22 optimism bias uplifts. 23 Only if you'd done a realistic assessment of that 24 type, and it turned out that the results showed you that 25 the risks were lower than what you had originally 89 1 assumed, then you would be justified in down adjusting 2 the uplifts. 3 Q. You make the point at the start of the next paragraph 4 that those cases anyway need to be bolstered by 5 empirical evidence? 6 A. Exactly. We have a whole paragraph explaining what this 7 realism would mean. 8 Q. If you look to the foot of the page, under the heading, 9 "Possible pitfalls": 10 "Another important pitfall in employing the approach 11 described above is that forecasters, when estimating the 12 future costs of a specific transport project, would 13 depart from the basic principles of reference 14 forecasting and would gradually return to the practices 15 of conventional forecasting, with forecasters focusing 16 on the details of the project at hand and attempting to 17 forecast the specific events that would influence the 18 future course of this project. The track record of 19 conventional transport cost forecasting shows that with 20 90 per cent likelihood, this would re-introduce optimism 21 bias in forecasting. This pitfall may be avoided by 22 consistently sticking to the method of reference 23 forecasting described above." 24 That's just emphasising that whenever you start to 25 diverge from the reference classes, making any 90 1 adjustments, that is when errors are going to creep in? 2 A. Yes. 3 Q. That would be equally true of making the downward 4 adjustments because they come later in the project? 5 A. Yes. 6 Q. I have finished with that guidance document for the 7 moment. 8 What I would like to do now is look at the STAG 9 guidance. Reference CEC02084489. 10 Just bear with me one moment. 11 Although we don't see a date in this document, you 12 note that this guidance was in fact issued in 13 September 2003. Is that your understanding? 14 A. Yes. 15 Q. If we go to within this guidance to page 4, and look at 16 paragraph 12.4.1, you can see it's suggested here that: 17 "Reference should be made to the Treasury Guidance 18 for the specific upper and lower bounds for the 19 contributing factors to optimism bias. These will 20 differ depending on the nature of the project. There 21 are six specific project types." 22 And they are narrated. 23 I appreciate this was obviously published the year 24 before your guidance document. And it seems to be 25 harking back to what was contained in the Mott MacDonald 91 1 work and the Green Book; is that correct? 2 A. Correct. 3 Q. So on the basis that the Green Book and the 4 Mott MacDonald are no longer the most up to date and the 5 most reliable, the same would follow this STAG guidance 6 in referring to them? 7 A. Correct. 8 Q. Could we look at 12.4.2 and 12.4.3. 12.4.2 says: 9 "The majority of transport projects will be 10 classified as either standard or non-standard civil 11 engineering projects." 12 Just pausing there, that's because they don't have 13 the benefit of your more refined classes, more accurate 14 classes produced later? 15 A. Yes. 16 Q. Then if we read forward to 12.4.3, the guidance says: 17 "Ideally, rather than use these generic factors, 18 adjustments for bias should be based on empirical 19 evidence from past and/or similar projects and adjusted 20 for the unique characteristics of the projects in hand. 21 It is anticipated that further work will be carried out 22 by the Executive and other bodies in order to refine the 23 figures for optimism bias in transport projects. Advice 24 on applying optimism bias factors should be sought from 25 the Scottish Executive at an early stage of project 92 1 development. Before reaching decisions, both cost 2 estimates and adjustments for optimism should be 3 independently reviewed." 4 Are you aware of whether or not the Scottish 5 Government has issued any further guidance after this? 6 A. No, I'm not aware of this. 7 Q. In the statement at the start of 12.4.3, that rather 8 than use generic factors, which seem to be the standard 9 and non-standard civil engineering projects, it 10 recommends that adjustments for bias should be based on 11 empirical evidence from past and/or similar projects, 12 and adjusted for the unique characteristics of the 13 projects in hand. 14 Now, that seems to be guidance to go away and try 15 and find past or similar projects yourself. 16 A. Yes, yourself or others. So basically, this is the 17 sentence that is taken from the Green Book. 18 Q. In a sense of choosing past or similar projects to make 19 up your own reference class, and then adjusting it for 20 the unique characteristics of the project in hand, just 21 as with the Green Book, is that going to create the very 22 sort of subjectivity and error that this exercise is 23 intended to avoid? 24 A. No, it wouldn't, if you followed the procedure for 25 selecting the reference class that we talked about 93 1 earlier, then it would bring more realism into the 2 assessment. 3 Q. But then if you make an adjustment for the unique 4 characteristics of the project in hand? 5 A. That's where the door is opened again for optimism and 6 other biases. 7 Q. If you could look at the following paragraph, the part 8 of it over the page: 9 "Having adjusted for optimism, the planner should be 10 in a position to provide a better estimate, earlier on, 11 of key parameters. Enforcing these adjustments is 12 designed to complement and encourage, rather than 13 replace existing good practice in terms of calculating 14 project specific risk adjustments and contingency 15 allowances." 16 I take it you wouldn't disagree with any of that, it 17 is important still to analyse risk and seek to reduce 18 it? 19 A. Yes. 20 Q. If we read on: 21 "They are also designed to encourage more accurate 22 costings. Accordingly, adjustments for optimism may be 23 reduced as more reliable estimates of relevant costs are 24 built up, risks are explicitly assessed and quantified, 25 and work to minimise project-specific risk is 94 1 undertaken." 2 Is this the same flaw as we saw in the Green Book 3 guidance, and the Mott MacDonald, in that it assumes as 4 the risk work is developed, it would be appropriate to 5 reduce optimism bias? 6 A. In my view that is the same flaw, yes. 7 Q. That's quite at odds with what was contained in your 8 guidance about the need for an objective basis for 9 reductions, either your interpolation or something 10 evidence based? 11 A. Yes. 12 Q. So once again, if you were asked to consider the 13 validity of this advice for future projects, what is 14 your view as to how valid this advice is? 15 A. I would say that this advice is problematic in the sense 16 that it -- it's counter productive in the sense that it 17 would -- it might defeat the very purpose of what the 18 exercise is about, which is rooting out bias and this is 19 reintroducing. 20 Q. If you look at the following paragraph, 12.5.1, it 21 notes: 22 "As project design and development progresses, it 23 should become possible to explicitly quantify and value 24 risk factors. Ultimately, appraisers should aim to 25 adjust costs and benefits in order to calculate risk 95 1 adjusted 'expected values'. As the previous section 2 explained, in the early stages of an appraisal, these 3 adjustments may be encompassed by a general uplift to 4 a project's net present cost, to offset and adjust for 5 undue optimism. But as the appraisal proceeds, more 6 project specific risks will have been identified ... 7 reducing the need for the application of more general 8 optimism bias factors." 9 Could you comment on that advice? 10 A. In theory it is correct that as you do more and more 11 detailed risk analysis, and as time passes, risks become 12 lower and therefore you should adjust optimism bias 13 uplifts accordingly. But like we talked about earlier, 14 this is where you have the risk of reintroducing 15 optimism bias, and therefore we suggest this more 16 mechanistic approach, where you simply looked at how 17 much of the budget has already been spent, and then you 18 adjust according to that, instead of adjusting to the 19 more subjective assessments of how much risk is taken 20 into account. 21 What we've seen is that if you do it this way, you 22 tend to end up with optimism bias uplifts that are too 23 low. 24 CHAIR OF THE INQUIRY: When you say this way, you mean the 25 way envisioned in this paragraph. 96 1 A. Correct. 2 CHAIR OF THE INQUIRY: What does your more mechanistic 3 approach involve? 4 A. That was the table we looked at earlier in the 2004 5 report, where we said if 10 per cent of the budget has 6 been spent, adjust the optimism bias uplift down by 7 10 per cent. If it 20 per cent of your budget has been 8 spent, downwardly adjust the optimism bias uplift by 9 20 per cent, et cetera. 10 CHAIR OF THE INQUIRY: Thank you. 11 MR LAKE: You've already said this was based on the 12 Mott MacDonald work. If we could go back and look at 13 that for a moment, please, it's document CEC02084689. 14 If we could go back to page 40 that we looked at before, 15 and the lower part of the page. 16 We looked at the second last paragraph which 17 commented on the upper bound values, but if we could 18 look at the very last part of the page, it says: 19 "The lower bound values identified represent the 20 optimism bias level to aim for in current projects with 21 effective risk management by the time of contract 22 award." 23 Now, looking at those lower bound values, for 24 non-standard civil engineering it's 6 per cent for cost, 25 and 3 per cent in respect of standard civil engineering. 97 1 It's suggested that these are the levels of optimism 2 bias you can aim for by reasons of effective risk 3 management. Is that a valid approach? 4 A. In my view, no. So I know enough data now. But we have 5 to be fair to both Mott MacDonald and to the Edinburgh 6 tram project management. They did not have the 7 knowledge that we have today. This is very early 8 stages. This was completely new territory at the time. 9 So the Mott MacDonald report is a very rudimentary 10 attempt to get a grasp on the issues here, and there are 11 lots of biases and errors in it we can see with the 12 knowledge that we have today, and this is one of them. 13 So these lower bound values are way too low, given what 14 we know today about projects, that at this stage that 15 they are talking about here, you actually need 16 substantially higher optimism bias uplifts than the ones 17 that are indicated by the lower bound here. But they 18 didn't know that at the time. They simply didn't have 19 good enough data to establish that. 20 Q. The impression appears to be given by this guidance that 21 the upper bound is when you have no effective risk 22 management, and the lower bound is what you must aim for 23 by effective risk management? 24 A. Yes. 25 Q. Which appear to give the impression that once you get to 98 1 a risk management place, you can just drop it right down 2 to the 6 per cent or the 3 per cent? 3 A. Yes, that's the idea that you get from this. 4 Q. And in a sense, again, you have to be fair to people 5 trying to conduct projects in 2007 and 2008, that that 6 was the guidance they were being given. 7 A. Yes. 8 Q. But it's not very good guidance, is it? 9 A. In my view, no. Also in 2007 and 2008, the 2004 data 10 were available. So there were better data available. 11 But in 2002 and 2003, that was not the case. 12 Q. Had the 2004 data been used, that's your work, the 13 result would have been a larger allowance would have to 14 have been made for optimism bias? 15 A. Yes. 16 Q. Now, that's quite unattractive for a project to have to 17 make that larger allowance, if they're trying to fit 18 within a certain budget? 19 A. That depends on the philosophy of the project management 20 and of the owner. To some project management and 21 owners, that is actually very attractive. They want to 22 be realistic. They know -- want to know what they're 23 getting into, and they also want to have the money if 24 they get into a situation where it is needed, because 25 experienced project managers know that it's a terrible 99 1 situation to get in for a project, that you don't have 2 enough funds, you don't have enough time. 3 It totally disrupts everything, and increases cost 4 and time, in its own way, the vicious circle that we 5 talked about earlier. 6 So from that point of view, it is desirable, 7 attractive to have the contingencies. But from a more 8 conventional point of view, where people are afraid of 9 contingencies, first of all, they are afraid they might 10 not get their project approved and funded if it has 11 a large contingency, because it's more expensive and 12 a more expensive project is less likely to be approved 13 and funded. That's the first thing. 14 The second thing is that some managers are also 15 afraid to have a large contingency, because they know if 16 the contingency is there, the builders, the contractors, 17 are going to go for it, and therefore, just having the 18 contingency might drive up costs. 19 So from that point of view, it might be 20 unattractive, yes, to have a large contingency. But 21 those are really two fundamentally different 22 philosophies that you find out there in reality on real 23 projects. 24 Q. We have just been looking at the Scottish Transport 25 Appraisal Guidance. I'll come to look in a moment at 100 1 the English version, but sticking for a moment with 2 Scotland, we have the Scottish guidance appears to take 3 the approach in the Mott MacDonald work and the initial 4 Green Book? 5 A. Yes. 6 Q. That's one strand of advice? 7 A. Mm-hm. 8 Q. The other strand of advice is your advice from 2004, 9 which, although up to date, has not yet been reflected 10 in the Scottish guidance over the last 15 years or so. 11 A. Yes. 12 Q. In terms of persons trying to implement projects, I take 13 it, it will be a simple thing to say that it would be 14 better if there was not conflicting guidance, there was 15 one set of authoritative guidance? 16 A. I would agree with that. The existing set of guidances 17 are confusing on some points. They are not up to date, 18 and they have these traps that will lead planners back 19 into reintroducing optimism into the planning process. 20 Q. If we could turn now to the Department for Transport, 21 Transport Appraisal Guidance, please. It has reference 22 CEC02084255. 23 We can see the title sheet there. 24 Could we look firstly at page 24. I would like to 25 look at section 3.6. 101 1 This guidance notes: 2 "As a project develops, the Department expects the 3 scheme cost estimate to be refined over time. As it 4 becomes possible to better quantify and value risks, it 5 should be possible to better capture the factors that 6 contribute to appraisal optimism within the risk 7 management process. Therefore, as risk analysis 8 improves as a scheme develops, it is expected that on 9 average the risk-adjusted scheme cost estimate will 10 increase while the applicable level of optimism bias 11 will decrease. 12 "It follows that in general, the Department expects 13 that the allowances for optimism bias should be the 14 largest at the initial stage of the life of a transport 15 project (eg Strategic Outline Business Case). This 16 allowance is expected to be smaller in a more detailed 17 business case (eg Outline Business Case) and smaller in 18 the presence of a fully detailed business case (eg Full 19 Business Case). 20 "As a scheme progresses, there are techniques for 21 reducing optimism bias uplifts through greater certainty 22 over costs and use of risk mitigation measures, and 23 independent reviews of risk and optimism bias. The 24 promoter will be expected to provide reasoning and 25 justification for any reductions in optimism bias 102 1 adjustment, from the recommended optimism bias uplifts." 2 Now, this seems to be suggesting there has to be 3 some reason for reducing it, but it's still not going to 4 your mechanistic interpolation approach, and therefore 5 do you consider that there's still room for subjectivity 6 and error? 7 A. There is. I mean, if reasoning and justification is 8 taken really seriously, then there would be less 9 subjectivity, but this -- the answer is that this is 10 also a door open for reintroducing subjectivity. 11 Q. To that extent do you consider that this guidance would 12 be more appropriately modified? 13 A. Would be more appropriately -- 14 Q. Would we better have this guide modified, so it didn't 15 leave that door open? 16 A. Yes, I believe there's a problem here that the 17 quantitative risk assessment, the QRA that we talked 18 about earlier, whatever method you're using for trying 19 to understand and reduce risk, is at odds with the 20 optimism bias uplift. So you have these two different 21 methodologies together in this exercise, and what 22 happens as you lean more and more on the QRA is that the 23 optimism bias uplifts become reduced and, as we see in 24 the Edinburgh case, completely eliminated in fact. 25 The balance between the two methods is not 103 1 appropriately addressed in the different guidances, 2 including this one. That's the problem. 3 Q. Focusing in particular on what is said in 3.6.3, the 4 last sentence, it says: 5 "The promoter will be expected to provide reasoning 6 and justification for any reductions in optimism bias 7 adjustment." 8 Now, obviously it's better to have reasoning and 9 justification than nothing, but that seems quite 10 different from the empirical evidence that you required? 11 A. Yes, exactly. 12 Q. If we go to paragraph 3.7.4, on the following page, we 13 can see that there are four stages or four steps 14 identified to carrying out the optimism bias assessment, 15 the first being to determine the nature of the project, 16 and the second one, which is a new one, is to identify 17 the stage of scheme development. Do you see that? 18 A. Yes. 19 Q. I think this guidance for the first time introduced 20 expressly different levels of uplift for each stage of 21 the project as it moved through. 22 A. Correct. 23 Q. If we could go to page 26, and 3.7.6, we can see that 24 for the purposes of this guidance: 25 "The Department has identified three main stages in 104 1 the life of a transport project for which default uplift 2 values have been provided, as illustrated in Table 8." 3 If we look at that table, if we scroll down 4 a little, we can see that for railways, for present 5 purposes, you've got three stages, grip stage 1, grip 6 stage 3, and grip stage 5, or to use the descriptive 7 terms, pre-feasibility, option selection and design 8 development. Do you see that? 9 A. Yes. 10 Q. Looking at the following page where we finally come to 11 the table which shows the recommended uplifts, we can 12 see here that there are different categories of roads, 13 rail and fixed link. So to that extent it has followed 14 the approach in your guidance of separating out the 15 different types of project in more detail. Do you see 16 that? 17 A. Yes. 18 Q. But nonetheless we now have the stage guidance, if we 19 look at rail, veers from 66 per cent in the first stage, 20 40 per cent at stage 2, and 6 per cent for stage 3. Do 21 you see that? 22 A. Yes. 23 Q. Now, the figures of 66 per cent at stage 1 and 24 6 per cent at stage 3, I think were respectively the 25 upper bound and the lower bound for non-standard civil 105 1 engineering projects at Mott MacDonald? 2 A. Mm-hm. 3 Q. Do you know, were those figures actually the ones 4 transposed into this document, or was new work done and 5 it's just coincidental that it arrived at exactly the 6 same figures? 7 A. I don't know that. 8 CHAIR OF THE INQUIRY: At the foot of the table, you see 9 beside the figures there's an asterisk, and at the foot 10 of the table it refers to both your report and 11 Mott MacDonald's, 2004 and 2002. 12 A. And that would indicate to me that additional work has 13 not been done since they are building on these two 14 sources, yes. 15 CHAIR OF THE INQUIRY: But if you then look at the note, it 16 refers to the Department for Transport currently 17 undertaking further research into optimism bias in rail 18 schemes. And table 9 will be revised in light of the 19 results of that research. But in the meantime, anyone 20 undertaking a rail appraisal should refer to the 21 forthcoming guidance on rail appraisal. 22 A. Correct. 23 CHAIR OF THE INQUIRY: So does that mean that there was 24 draft guidance or something? It speaks about 25 forthcoming guidance. How would you get access to that, 106 1 or how would a developer get access to that? 2 A. I'm not aware that any draft guidance was available, 3 and I guess the way that developers would get access to 4 this was by asking. 5 MR LAKE: I wonder if it would be possible to have two 6 documents on screen side by side. The page at the 7 moment would be one side of the screen and then on the 8 other side would be the page from the Mott MacDonald 9 report with reference CEC02084689, page 40. 10 Just bear with me, Professor Flyvbjerg. 11 On the right-hand side of the screen we have the 12 Department for Transport TAG Guidance, and on the 13 left-hand side of the screen, we have the Mott MacDonald 14 report. Do you see those? 15 A. Yes. 16 Q. We can see there the figures of 66 per cent and 17 6 per cent appear in the TAG guidance to the right-hand 18 side as the stage 1 and stage 3 respectively for rail 19 projects, whereas on the Mott MacDonald guidance, they 20 are the upper and lower bands for non-standard civil 21 engineering projects? 22 A. Yes. 23 Q. In the Mott MacDonald guidance we were told that the 24 lower bound was a target for which the parties should 25 seek to aim throughout the project? 107 1 A. Yes. 2 Q. They would do that by means of risk management? 3 A. Yes. 4 Q. You've already expressed some concern about whether or 5 not that was a truly valid approach? 6 A. Yes. 7 Q. If we look at the Department for Transport TAG Guidance, 8 it's been elevated from being a target, albeit 9 a questionable one, to being an assumption that the 10 optimism bias will be reduced to 6 per cent at that 11 stage 3? 12 A. Yes. 13 Q. Is that a valid approach? 14 A. I would say no. In my view, no. 15 Q. In your experience, would it be reasonable to assume 16 that by the stage 3, optimism bias has in fact been 17 reduced to merely 6 per cent? 18 A. It is not, but again, I have to say, I have the benefit 19 of hindsight. We know so much more today. So I know 20 that usually optimism bias cannot be reduced to this 21 level of 6 per cent at this stage in the project cycle. 22 If you want to be realistic at this stage, you need 23 a number that is higher. And I would say as a general 24 rule of thumb, whenever you see a single digit number, 25 the red lamps, the warning lamps should go on. That's 108 1 a warning sign, because very rarely are the risks single 2 digit. 3 Q. Are you aware whether there's any published guidance to 4 that effect, saying that the 6 per cent figure is not 5 a realistic one? 6 A. Not that I am aware. 7 Q. If we could change the document on the left-hand side, 8 that is the Mott MacDonald guidance, and substitute it 9 instead with the 2004 work you did, that's reference 10 CEC02084257. It's page 34. 11 We can now see that it's the work that you 12 participated in on the left, and in terms of the rail 13 figures, whereas on the right-hand side, the stage 2 is 14 given as 40 per cent, it's difficult to fit that exactly 15 into your figure, save that 40 per cent was the 16 50 per cent percentile figure for rail? 17 A. At the Final Business Case stage, yes. 18 Q. At the Final Business Case stage. 19 A. Yes. 20 Q. Perhaps the difficulty is that's not the stage that the 21 DfT guidance recommended that it be applied at, because 22 it's being applied at option selection? 23 A. Yes. So it's unclear exactly where option selection is 24 in relation to outline Business Case, and Final Business 25 Case and tendering. So those are just two different 109 1 baselines in the project cycle, and -- but apparently 2 the same number is being applied. So our number is 3 strictly empirical, and that number is then applied in 4 the TAG document for stage 2. 5 Q. Do you consider that's a valid transposition of your 6 number? 7 A. Not really, but I would have to know more about option 8 selection exactly where they see this in relation to the 9 Final Business Case, and the further it is from the 10 Final Business Case, the more inaccurate this assumption 11 would be. 12 Q. In your guidance you provide, as we can see to the 13 left-hand side of the screen, the different percentiles 14 for confidence levels, and we have no information at all 15 in the DfT guidance as to what confidence level they are 16 assuming. Does that affect the usability of this 17 information? 18 A. In my view, yes. 19 Q. Why? 20 A. Because you need to know what the probability is. It 21 relates to this thing we talked about earlier, about 22 buying insurance. If you don't know what level you're 23 insured at, the numbers are not much use, and that's 24 actually the situation with the TAG report, is that you 25 need to know what level of -- what level of security you 110 1 have with the numbers that are given. And that's not 2 immediately evidenced. 3 Q. Bear with me one moment. 4 If we go to page 28 now of the document on the 5 right-hand side, and we just look at the paragraph at 6 the top of the page. It notes: 7 "If the Department is provided with sufficient 8 evidence, it is possible to use uplifts other than those 9 recommended. Uplifts that deviate from the recommended 10 uplifts will reflect both the stage of development of 11 the option, the quality of the risk assessment provided, 12 and the extent to which optimism bias may or may not 13 have been mitigated. The Department does not expect to 14 see uplifts used that are below those given for the next 15 stage of scheme development in table 9." 16 Once again, that suggestion of using evidence to 17 depart from the uplifts, is that, do you consider, 18 a problem, or does the fact it requires evidence mean 19 that it should be problem free? 20 A. It is correct that you need to adjust the optimism bias 21 uplift down over time. So in principle this is correct, 22 and it's also right that evidence is required. The 23 question is what is evidence? And I would say empirical 24 evidence. So if it's empirical evidence it's okay. But 25 if it's evidence that is more arbitrary, stemming from 111 1 the QRA, then it's problematic in my view. 2 Q. Okay. I'm finished with those documents on screen now. 3 CHAIR OF THE INQUIRY: Can I just ask, Professor, the 4 impression I get is that the STAG guidance is out of 5 date, and if you were planning a major project, 6 something of GBP100 million plus, would it be prudent to 7 rely on the STAG guidance to establish the Business Case 8 or to go forward, or what would your advice be about 9 information that should be -- 10 A. Do you mean today or at the time? 11 CHAIR OF THE INQUIRY: Today. 12 A. Today? I would say it needs to be updated. But to be 13 fair to the STAG guidance, it reflects the time that it 14 was made, 2003, and it reflects the information that was 15 available at that time. There are some problematic 16 things, like the relationship between the QRA and the 17 optimism bias uplifts, but otherwise it really reflects 18 its time. 19 But it's not adequate for today. It needs to be 20 updated. 21 CHAIR OF THE INQUIRY: We know that an extension of the 22 tramline is being contemplated, and the likely cost 23 would put it into a major project. Really what I was 24 seeking is: what should be done to try to avoid the 25 difficulties that arose in the line to date? 112 1 A. So I would strongly advise to update the guidance in 2 light of all the information that we have today, which 3 is a lot more than we had when it was first made in 4 2003. 5 CHAIR OF THE INQUIRY: Thank you. 6 MR LAKE: Could I ask you then to look at your report again, 7 which is reference TRI00000265. Can we look at page 22, 8 please. 9 We can see here we've got the heading, "Expert 10 Evaluation", and you start with: 11 "Treatment of Optimism Bias in the Draft Interim 12 Outline Business Case". 13 Now, most of this we can read for ourselves. What 14 I would like to do is look at a passage starting at the 15 top of the following page. 16 We see there it notes that: 17 "... the Draft Interim Outline Business Case argued 18 that the headroom between the then base cost estimate 19 and the funding envelope was 54 per cent and thus 20 covered the project's cost plus 44 per cent of optimism 21 bias uplift suggested by the Mott MacDonald data." 22 First of all, that 44 per cent assumed this was 23 a standard civil engineering project. You said in your 24 view that was bogus because those are all road projects? 25 A. Correct. 113 1 Q. Then it says: 2 "It should be noted that the DfT guidance on 3 optimism bias procedures available at the time shows 4 that at P80 rail projects ought to use an uplift of 5 57 per cent." 6 You also give the figures for P90 and P95, and that 7 was the report that you co-authored? 8 A. Correct. 9 Q. You say then: 10 "At P80 the headroom between the base cost and the 11 funding envelope would have been used up." 12 Just to be clear what that's telling us, if they had 13 used the 2004 guidance that you co-authored, they would 14 have known that for a rail project at P80, if they did 15 want to take a P80 approach, would have required 16 a 57 per cent uplift in the optimism bias figures? 17 A. Correct. 18 Q. There simply wasn't the headroom between the available 19 funding and the anticipated risk adjusted cost to 20 accommodate that? 21 A. Correct. 22 Q. So had the calculation been done appropriately at the 23 time, it would have at least indicated a concern as to 24 how things would be managed? 25 A. Yes. 114 1 Q. You follow that by saying: 2 "In the view of the experts, the Draft Interim Outline 3 Business Case overstates its case with regards to cost 4 risk. The project team argued that it would deliver 5 according to the budget envelope with more than 6 95 per cent certainty while the data in DfT's guidance 7 on optimism bias, which were available to the planners, 8 suggested that a 20 per cent risk of exceeding the 9 funding envelope existed, ie a risk four times higher." 10 Now, 95 per cent we know was the figure that was 11 being advanced by the scheme promoters as being their 12 level of confidence as to their costs. Is that where 13 you get that figure from? 14 A. Yes. 15 Q. When you talk about the 20 per cent risk of exceeding 16 the funding, is that because there wasn't sufficient 17 headroom to achieve the P80 figures? 18 A. Yes. So what we basically do here is that we look at 19 what is available, and then we look at our data from the 20 2004 report, and from that we can read with that budget 21 there's actually a 20 per cent risk that the budget will 22 be higher than what is available. 23 So therefore instead of a 5 per cent risk, so the 24 5 per cent above the 95 per cent, there's a 20 per cent 25 risk, the 20 above the 80 per cent, the P80, and that's 115 1 four times higher than -- the real risk is four times 2 higher than the assumed risk. 3 Q. Or rather the risk that was being reported? 4 A. Yes, exactly. 5 Q. We then see the heading. You go on to look at the 6 treatment of the optimism bias and risk in the Final 7 Business Case. 8 Again, we can read most of this to ourselves, but if 9 I can look at a passage on page 24 first. It's the 10 third paragraph on that page, beginning "In the experts' 11 view". You say: 12 "In the experts' view, there are some doubts about 13 the quality of the quantitative risk analysis which 14 estimated a cost risk of 15 per cent at P90, which seems 15 low given the high level of confidence and the evidence 16 in the official guidance documents available to the 17 project at the time, when the project created the Final 18 Business Case." 19 Just pausing there, when you talk about the 20 documents available to the project, you would be 21 including your own guidance in that? 22 A. Yes. 23 Q. We can look there, we saw the P80 uplift you have 24 already referred to for a rail project would be 25 57 per cent? 116 1 A. Yes. 2 Q. It just goes up and up from there as you get to P90, 3 disproportionately larger? 4 A. Yes. 5 Q. So with that document available, and they do a cost risk 6 analysis which brings out a figure of 15 per cent at 7 P90, what does that tell you? 8 A. Again, that tells me that the team is being optimistic. 9 Again, this is the QRA has had this effect that the team 10 has arrived at a number that is -- that is simply 11 unrealistic. 12 Q. Again, is that because that was generally available 13 information, could that and should that have been 14 ringing alarm bells that the risk figure was being put 15 as low as 15 per cent? 16 A. In my view, it should, and that's not only hindsight. 17 That's actually with the data available at the time, 18 this should have been an indication that something is 19 wrong here, and what is wrong is that optimism has 20 re-entered the planning process. 21 Q. If we could go over to the following page, page 25, and 22 look at the last paragraph, the Inquiry has already 23 heard evidence to the effect that it was the view 24 expressed in the Final Business Case that instead -- 25 A. Sorry, where are we? 117 1 Q. I'm not reading something from here. I'm just setting 2 the scene. 3 A. Sorry. 4 Q. In the Final Business Case, rather than apply even the 5 reduced 6 per cent optimism bias with a risk probability 6 set at P mean, the decision was made to apply a risk 7 figure at a P90 level, on the assumption that that would 8 make a greater allowance than P mean plus the optimism 9 bias. 10 A. Correct. 11 Q. Are you aware of that? 12 A. Yes. 13 Q. Now, the effect of that is, because we're using a P90 14 figure, that's all out of the QRA, and therefore is 15 entirely an inside view? 16 A. Yes. 17 Q. If they had used the other guidance that had been P 18 mean, which is inside view, plus 6 per cent which is an 19 outside view, that would be correct? 20 A. What do you mean by that would be correct? 21 Q. What I mean by that would be correct? Sorry. Not 22 that -- yes. Piling questions upon questions. 23 You say in your report that that there's a switch 24 from a wholly inside view, and I can see that using P90 25 would be a wholly inside view? 118 1 A. Correct. 2 Q. If they had used P mean plus the 6 per cent optimism 3 bias uplift, would that have had any realistic outside 4 view component in it? 5 A. Yes. So that would have had the 6 per cent realistic 6 outside view, but again, like we talked about earlier, 7 that in my view would also be optimistic. 8 Q. Because the 6 per cent is too low? 9 A. Exactly. 10 Q. But I think you have done an analysis, and the claim 11 that is made in that case, that P90 is larger than P 12 mean plus 6 per cent, is in fact incorrect? 13 A. Yes, according to our analysis, it is incorrect. 14 Q. Perhaps rather than work through it, that's what's set 15 out at the foot of page 25. If I could ask us to go and 16 look on to page 26, that's what you're explaining there? 17 A. Correct. 18 Q. If I could ask you to look in page 26 at the paragraph 19 beginning "Moreover". You say: 20 "Moreover, the project's justification for this ..." 21 That is using the 90 per cent probability: 22 "... the project's justification for this seemingly 23 low estimate of the cost risk exposure was centred 24 around two points: (1) that utility diversion (MUDFA) 25 works had already commenced; and (2) that the procurement 119 1 strategy would significantly de-risk the project. The 2 project expressed the view that at contract award, 3 optimism bias is 0 per cent. In other words, the 4 project assumed that due to ongoing refinements of its 5 risk register and its risk analysis and due to its 6 commercial strategy, all risks will be fully known. 7 This indicates to the experts that the risk management 8 team did not fully understand the nature of optimism 9 bias and, because of this, the team and project would 10 have been particularly prone to such bias." 11 Your view that they didn't understand the nature of 12 optimism bias, is that because if they had understood it 13 they would assume that it would never be 0 per cent? 14 A. Exactly. Risks are never fully known in that sense that 15 you can eliminate the optimistic bias uplift. So the 16 fact that this is eliminated or set at zero is the 17 indication that they don't understand the nature of risk 18 and the nature of optimism bias. 19 Q. Turning to the question of recommendations, we see on 20 page 28 of your report -- again, we can read through 21 that ourselves. I don't propose to go through it just 22 now. But just picking out some of the original points 23 that have arisen in our discussion this morning, I think 24 one of the things in response to a question from 25 Lord Hardie is that it would be useful to have updated 120 1 work on the various reference classes and the databases 2 to produce more accurate, more refined figures -- 3 A. Yes. 4 Q. -- for the uplifts? 5 A. Yes. So we actually -- so we have the updated data. So 6 we can test for this. It's quite easy with the database 7 that we have, and like we mentioned in the report, we 8 have already tested for this and we can actually 9 document that the existing data in the guidances are 10 dated. The up-to-date data will give you different 11 results. 12 For instance, for the P80, which is the value most 13 used, the number is now higher than it is in the 14 guidances, just to take a specific example. 15 Q. That's something which you think could usefully be done 16 at intervals of no more than ten years, and perhaps five 17 years would be appropriate? 18 A. That is my view, yes. 19 Q. If that guidance is to be provided, do I take it from 20 our discussions that certainly it would be useful to be 21 broken down into different types of work, as in rail, 22 road and so on and so forth, and that there be 23 probabilities provided for those? 24 A. Yes. 25 Q. In terms of the reduction over time over the course of 121 1 a project in the optimism bias uplifts, is that 2 something where some clarity is needed as to the basis 3 on which it is done and how appropriate that 6 per cent 4 figure is? 5 A. Yes, very much so in my view. 6 Q. Could we look quickly at page 30 of your report. If we 7 look at the paragraph beginning "To provide adequate 8 challenge and control". This is under recommendations 9 to people engaged in projects, rather than government. 10 You note that: 11 "To provide adequate challenge and control, the 12 governance bodies need to receive unbiased and 13 up-to-date information about project performance. In 14 similar projects the experts found that effective 15 governance relies on multiple channels of information to 16 senior decision makers, for example, data-driven reports 17 on project performance and forecasts combined with 18 reports by the management team and independent audits. 19 In the reporting, special emphasis must be placed on 20 detecting early warning signs that cost, schedule and 21 benefit risks may be materialising, as they tend to do, 22 so damage to the project can be prevented. When early- 23 warning signs emerge, projects should revisit their 24 assumptions and re-assess risk and optimism bias 25 forecasts." 122 1 What you are saying there is if it doesn't appear to 2 be going according to plan, that's a sign your risk 3 forecasts may need to be revisited? 4 A. Yes, and it's also a sign that you need to take action 5 immediately. And we found that that is a weakness in 6 many projects, that when the early warning signs appear 7 action is not taken. 8 First of all, sometimes people don't want to report 9 the truth upward in the organisation because that's 10 often not seen as a positive thing, somebody saying that 11 this project is going off-track, we won't be able to 12 meet the schedule or the budgets. Or even if it's 13 reported, it's not clear who is supposed to take action. 14 It's another thing we find, that it's not clear which 15 part of the organisation is supposed to deal with this. 16 This actually hasn't been specified as part of the 17 delivery. 18 So, again, that means that action is delayed and not 19 taken, and that's the worst thing that can happen. The 20 more time that passes between a problem arises and it's 21 dealt with, the more likely it is that it will have 22 a large impact, a negative impact, on schedule or costs 23 or both. 24 Q. The Inquiry has heard evidence that part of the 25 procurement strategy for the tram project was based on 123 1 getting design carried out and completed at an early 2 stage, so it was intended that the detailed design would 3 be 100 per cent complete at contract award, and also 4 that the works to divert the utilities would be complete 5 or largely complete at the time the infrastructure work 6 started. 7 Again, if you can just take it from me that the 8 Inquiry has heard evidence that that wasn't happening, 9 that the design was late and the utilities works weren't 10 being completed, is that the sort of thing that should 11 have immediately raised concern that the risks had not 12 been properly understood? 13 A. Yes, that's typical. Those are two typical early 14 warning signs that not only has risk not been properly 15 understood, but also that risks are going to be 16 increasing from now on and you need to deal with it 17 quickly. 18 Q. That's what I was going to say. What would the 19 appropriate step be to take in response to that 20 situation? 21 A. Two things. First of all, deal with it quickly so that 22 the damage is limited. 23 Second, readjust the risk assessment and risk 24 management accordingly, so it takes into account the 25 change that has happened. 124 1 Q. If there are going to be external checks upon a project 2 from time to time, in your view, is one thing that would 3 be particularly useful to check whether or not there are 4 these divergences or early warning signs from the 5 proposed risk treatment strategies? 6 A. Yes. 7 MR LAKE: Thank you very much, professor. 8 A. You're welcome. 9 CHAIR OF THE INQUIRY: I don't think there's any other ... 10 Thank you very much, professor. You're free to go. 11 We appreciate your assistance. 12 A. Thank you very much. Thank you. 13 (The witness withdrew) 14 MR LAKE: My Lord, the next witness is Stuart Fair, who is 15 from the Chartered Institute of Public Finance 16 Accountants, again speaking to a report which has been 17 intimated. The evidence will be taken from him by my 18 colleague, Mr Mackenzie. 19 I don't know if my Lord would like to rise now and 20 have an early lunch and start again early, or what would 21 be most -- 22 CHAIR OF THE INQUIRY: Yes, that probably makes sense 23 actually. We will rise now and resume again at 1.45. 24 (12.38 pm) 25 (The short adjournment) 125 1 (1.44 pm) 2 CHAIR OF THE INQUIRY: Yes, Mr Mackenzie. 3 MR MACKENZIE: Thank you, my Lord. The next witness is 4 Stuart Fair. 5 MR STUART FAIR (sworn) 6 Examination by MR MACKENZIE 7 CHAIR OF THE INQUIRY: I see you've got some documents in 8 front of you. You will actually be shown any documents 9 that we need to show you because people have got to know 10 what you're looking at. 11 A. Thanks, my Lord. 12 MR MACKENZIE: Thank you, my Lord. Good afternoon. 13 A. Good afternoon. 14 Q. Can you state your full name please? 15 A. Stuart William Fair. 16 Q. And your current occupation? 17 A. Chartered Public Finance Accountant. 18 Q. Thank you. I would like to go to your CV, please. It 19 will come up on screen. CVS00000087. We can see your 20 name at the top, along with your professional 21 qualifications. I can see, I think, LLB obviously is 22 a law degree. Blow up that. Thank you. 23 At the end, JP, I think, is Justice of the Peace. 24 What do the other letters stand for? 25 A. I'm a fellow of the Chartered Institute of Public 126 1 Finance and Accountancy. That's CIPFA. I'm a certified 2 practising accountant for CPA Australia, and I'm 3 a fellow of the Association of Chartered Certified 4 accountants. That's ACCA. Also a fellow of the 5 Royal Society of Arts. 6 Q. Thank you. So in short, I think you have both legal and 7 accountancy qualifications? 8 A. Yes. 9 Q. We see then in the skills you have summarised at the 10 top, we can also see your current position is both 11 a senior consultant of CIPFA and a tutor at the 12 Strathclyde law school. 13 We can then, I think, read for ourselves in due 14 course the various bullet points setting out your 15 professional and vocational experience. 16 Over on page 2, I think we'll see a long list of 17 relevant consultancy work, and again, I won't spend time 18 on that. 19 Then finally, if we can go to page 3, please, you 20 have summarised your employment history there; is that 21 correct? 22 A. Yes, it is. 23 Q. Thank you. Now, before we come to your report, you 24 mentioned CIPFA, the Chartered Institute of Public 25 Finance and Accountancy. Can you summarise for us what 127 1 is CIPFA, what does it do? 2 A. Yes. CIPFA, the Chartered Institute of Public Finance 3 and Accountancy was formed in 1885. It is the only 4 chartered accountancy body that specialises in the 5 public sector, in fact it's the only accountancy body 6 globally that specialises in the public sector. 7 The institute itself is a registered charity. Its 8 primary aim is to serve in the public interest and to 9 regulate its members. Its members, there's about 14,000 10 in total, including students, and they are mostly in the 11 UK, but we have about 10 per cent or so overseas. 12 Q. Thank you. That's the regulatory role of CIPFA. 13 Am I right that CIPFA also has a consultancy arm that 14 undertakes financial management and governance reviews? 15 A. It does indeed. There is a commercial consulting arm 16 that undertakes a variety of consulting work for all 17 public sector bodies, UK and across the world, and that 18 work is channelled back into the charity for the 19 furtherance of public sector finance. It's one of the 20 aims of the institute. 21 Q. Thank you. You have prepared a report for the tram 22 Inquiry. We will go to that, please. It's TRI00000264. 23 We can see that is entitled "REVIEW OF FINANCIAL 24 MANAGEMENT AND GOVERNANCE REPORT", dated January 2018. 25 Was this report produced by you, Mr Fairley? 128 1 A. It was, yes. 2 Q. Before we go through some of the text and details, just 3 by way of an overview, can you start, please, by 4 explaining how you undertook this review? 5 A. We were asked by the Inquiry team what information 6 requirements that we would wish to have at our disposal, 7 and we put together a list of different documents and 8 evidence that you would need to conduct a review of the 9 financial management and governance aspects of the tram. 10 We were able to secure a sufficiency of evidence to 11 form a view. We had cause to ask for more evidence 12 which the tram Inquiry team were able to relay to the 13 appropriate parties, and were able to get some of that 14 back, which we used in our assessment and the 15 compilation of this report. 16 Q. So that's the first step, you have the relevant 17 documents, what happens next? Do you adopt a particular 18 methodology or what? 19 A. Yes. Rather than doing a deep dive into specific 20 components of the tram project itself, whether it be 21 procurement, the outline Business Case, or the actual 22 project management, we took a holistic view, to try and 23 break down against acknowledged good practice within an 24 overarching framework of financial management and 25 governance, what we would expect to be at that time and 129 1 place in terms of good practice regardless of the 2 complexity of the project. 3 So in terms of trying to determine the benchmark for 4 good practice across financial management and 5 governance, we had cause to use two particular models 6 that CIPFA had available: the CIPFA financial management 7 model, and there is a governance mark, a G mark, which 8 tests the effectiveness of the capability of governance 9 arrangements. 10 So using both models, and a particular good practice 11 standard on the role of the chief financial officer in 12 public sector organisations, we were able to bring 13 together components of good practice that we would 14 expect in the currency of the project from cradle to 15 grave so to speak. 16 And looking at the evidence that we had at our 17 disposal, trying to form a view on various components of 18 that good practice that were met or not met. 19 So it's effectively what we considered to be 20 a GAAP(?) analysis on prevailing good practice at the 21 time that this project was undertaken, and to assess the 22 positive and the not so good aspects of this as it went 23 through. 24 Q. Now, we looked in your CV at page 2 briefly, the long 25 bullet point list of consultancy work you have carried 130 1 out. 2 To what extent does the approach you have just 3 described that you followed in the review of the 4 Edinburgh Tram Project compare with the approach you 5 would take when asked to conduct a financial management 6 and governance review of other public projects or public 7 bodies? 8 A. Well, based on your last point there, public bodies, 9 traditionally we would do a financial management 10 capability assessment of an organisation, whether it be 11 City of Edinburgh Council, which we haven't done, but 12 Her Majesty's Treasury, Ministry of Defence, London 13 borough of Harrow. We can assess the overall financial 14 management capability of the organisation, and that's 15 not just the strength of the finance function. It's the 16 strength of financial management capability throughout 17 the organisation. 18 So that's looking at both the finance community and 19 the non-finance community, and making an informed 20 assessment of how close they are to good practice. 21 In terms of this particular project, we used an 22 approach we have used in some other projects, namely 23 a hospital funding project on the Island of Jersey. 24 We've looked at other proposed projects, perhaps not in 25 the same scale as this infrastructure type project, but 131 1 looking for attributes of good practice, whether it be 2 in the procurement side, the Business Case side, the 3 actual tendering, the management, the actual contract 4 management capability of it. We've been able to use 5 this kind of approach before, but not with so much 6 evidence. 7 Q. So subject to that caveat, would it be correct to say 8 that you have followed a similar approach in conducting 9 the review of the Edinburgh Tram Project as you would 10 use in conducting other reviews? 11 A. On the principles, yes. But obviously the scale of 12 evidence is something that is much larger than what 13 we've experienced before. 14 Q. Thank you. 15 Now, finally, by way of overview, what were your 16 main conclusions after undertaking this review of the 17 tram project? 18 A. The project itself was beset with many difficulties from 19 the very start, and looking at the evidence, it felt 20 like these initial difficulties set the scene, and the 21 die was cast for significant problems to arise, and they 22 emerged from these initial difficulties and weaknesses. 23 The most frustrating thing that comes across in the 24 evidence is that there's certainly a will, you could 25 see, to employ good practice. There was frameworks in 132 1 place that should have delivered a much better outcome, 2 but they didn't. And I think this is the most 3 frustrating part, looking at the evidence. 4 There was governance arrangements in place that 5 should have produced better outcomes. There was 6 financial management framework there that should have 7 produced a better outcome, but both were thwarted in the 8 currency of the project itself. 9 The procurement and the risks around the procurement 10 side of it looked pretty solid initially, but in the 11 run-up -- in the run-up to actual practice in 2002 -- 12 2008, I think it became clear that by financial close, 13 the seeds were sown for real problems ahead. There's 14 certainly evidence that there was undue haste to enter 15 into financial close, that perhaps members were not 16 given the correct information to make an informed 17 decision, and that contingency and risk was never 18 properly evaluated. And there was a lack of 19 understanding what that contract that the Council were 20 entering into following the report to the Policy 21 Committee in May 2008, and from that the Council and tie 22 were locked into a contract that was insoluble, 23 effectively, and we didn't think that adequate -- there 24 was adequate consideration to termination at that point 25 in time. 133 1 Even up to 2009, there was roughly about 2 GBP230 million spent there, and obviously half a billion 3 pounds later, we get a final result in 2014. 4 So there's opportunities to objectively look at the 5 project, where it was going, and work out whether or not 6 there was value being obtained from it all. 7 I think that's the most frustrating part of it. 8 There was frameworks in place. They should have 9 operated. They obviously didn't. And we had a project 10 where a spend that started off for a significantly 11 greater routing ended up from GBP350 million-odd to 12 three-quarters of a billion, and even then, we're not 13 wholly convinced that the GBP776 million of the final 14 project cost is the true cost associated to this 15 particular project. 16 By the time the 2011 mediation activities were put 17 in place, certainly it looked as if there was an applied 18 focus, a betterment and application, and with a new 19 Chief Executive in post, and driving this with external 20 help in terms of consultancy support, it certainly saved 21 the project. 22 But was still at that point in time, there was still 23 opportunities to consider termination. We don't think 24 that all the options were properly evaluated. 25 Q. Thank you. I'll come back to explore some of these 134 1 points a little more when we go through your report, but 2 if I could just seek to clarify three points just now 3 while I remember. 4 Firstly, you mentioned latterly there that you were 5 not wholly convinced that 776 million represented the 6 true costs of the project. 7 Now, I think, firstly, I think you have seen the 8 statement provided by Mr Connarty of the Council to the 9 Inquiry which sets out the costs of the project; is that 10 correct? 11 A. Yes. 12 Q. So why do you say then that you're not wholly convinced 13 that 776 million represents the true costs? What's 14 missing? 15 A. We looked at the balance sheet for City of Edinburgh 16 Council and we couldn't get a precise match between the 17 776. There was a letter that -- we asked for a detailed 18 breakdown, principally to get information on the asset 19 registers, which apparently in the form that we would 20 expect didn't exist. But we did get a charting of the 21 actual expenditure going through the balance sheet from 22 about 2004/2005 onwards, and the figure attributed to 23 the tram expenditure wasn't exactly the same. 24 During our review of all the evidence, there was 25 a couple of issues that came out that did not seem to 135 1 correlate with the tram expenditure figure. One was the 2 reinstatement works, and they would have been from 3 St Andrew Square down to Leith Walk, et cetera. 4 We would have expected, through proper accounting 5 practice, for those costs to be associated with the 6 project itself, rather than being classified as an 7 infrastructure capital expenditure. 8 We were also made aware that some of the legal costs 9 in terms of the contractual disputes were not 10 capitalised. Again, proper accounting practice in local 11 government would require a spend to be associated 12 specifically with its cause. 13 Because of these things, including -- evidence was 14 provided that all the overheads, in terms of overhead 15 allocation from the Council, CEC's side of it, rather 16 than tie, were not properly distributed to the project. 17 So putting these points together, we would have some 18 difficulty in accepting that, with real precision, that 19 the GBP776 million was the final cost of this project. 20 Q. Thank you. We could perhaps consider in due course if 21 we have to go back to the Council on these points. 22 Two other points you had mentioned. You started 23 off, I think, by saying that there were initial 24 difficulties or weaknesses which set the scene for what 25 was to follow. 136 1 Just for the avoidance of doubt, what do you mean by 2 these initial difficulties or weaknesses which set the 3 scene for what was to follow? 4 A. Certainly the -- at the start, there was some good 5 efforts and good endeavours to put in some good practice 6 frameworks. OGC provided risk assessment advice and 7 found that arrangements with tie and the Council were 8 robust. 9 But when it came to the mechanics of providing 10 costings and estimates on a specification, including 11 risk contingency, it feels as if, rather than building 12 it up in a granular way, the overall quantums that were 13 emerging perhaps were made to fit a particular overall 14 cost envelope, rather than having specifically costing 15 and putting in contingencies and optimism bias for areas 16 that were really unknown and unforeseen, particularly 17 utilities diversion, and not just the costings for that, 18 the timeline for all of that. 19 When tie and the Council were trying to pull this 20 together, I think there was a point in time when it was 21 clear that there was a lack of clarity over who was 22 actually designing large components of the work, and 23 when it came to what we considered to be -- what was 24 known as the final deal report in May 2008, we felt that 25 members were pushed into that, and that the benefits on 137 1 accepting that particular contract framework were 2 significantly overstated. 3 The contractor still had a significant role in the 4 design of their own brief, and in any project 5 procurement scenario, having your contractor actually 6 designing the brief is going to be suboptimal in some 7 way. 8 So by accepting a contract framework and the price 9 change at that particular time, it felt that the project 10 was setting the scene for further problems ahead, and so 11 it transpired. 12 There seems to be a lack of clarity and 13 understanding about that contract itself, whether it was 14 a fixed contract, fixed price contract, and I think 15 that's what the members were advised it would be 16 predominantly, and it ended up a contract that could be 17 disputed significantly. 18 So the difficulties arose fairly early, in the first 19 part of the project, and I think set the scene for what 20 was to come. 21 Q. Thank you. I would like now, please, to go through some 22 particular passages in your report and ask you some 23 questions about that. 24 If we could start, please, at page 3 in 25 paragraph 1.2. I will just read that out. You say: 138 1 "Whilst we express certain views in this report, on 2 the basis of the evidence available to us, the purpose 3 of this report is to avail Lord Hardie, as the ultimate 4 fact-finder and decision-maker, in coming to his own 5 views in these matters on the basis of all of the 6 evidence before him." 7 So as I understand that, I think there are two 8 things you are drawing attention to there, at the 9 beginning of your report. 10 Firstly, you've only had the opportunity of looking 11 at part of the evidence to the Inquiry, and Lord Hardie 12 of course has the benefit of seeing all of the evidence; 13 is that correct? 14 A. Yes. 15 Q. I think another point you are drawing attention to there 16 is that you recognise that it is, of course, for 17 Lord Hardie as the Chairman of the Inquiry to come to 18 his own views on all of these matters. Your report is 19 simply to provide any expert assistance that can be 20 provided in that regard; is that correct? 21 A. Absolutely. 22 Q. Could I also, as a preliminary matter, check, presumably 23 your views, Mr Fair, are based on the benefit of 24 hindsight, so that when you're commenting on what was 25 done or reported at the time, you have also had the 139 1 benefit of knowing what happened after that? 2 A. After submitting the report? 3 Q. No. By way of example, for example, when you comment on 4 the reporting to the Council in May 2008, I think you 5 had referred to the benefits of the contract being 6 significantly overstated. You have the benefit of 7 knowing what happened after the contract was entered 8 into. 9 A. Yes. I would qualify that by saying the lack of 10 information given to the decision-makers for a contract 11 of that size should have rung alarm bells. 12 CHAIR OF THE INQUIRY: So your evidence on that matter is 13 not based on your knowledge of what happened 14 subsequently. Are you applying some test as to what the 15 proper practice was in 2008? 16 A. We tried to use the prevailing standards of good 17 practice at that time, my Lord, but I do take your 18 point. It's easy with hindsight to look back and be 19 critical. So I would have to agree with Mr Mackenzie's 20 position on that. 21 MR MACKENZIE: Thank you. Could we then please go over the 22 page to page 4. I want to understand your methodology, 23 how you went about preparing this report. 24 At the top of the page we see a reference above 1.7 25 to "best practice standards", and you then in the bullet 140 1 points set out what these standards are. 2 Dealing firstly with the first bullet point: 3 "CIPFA Financial Management Model including 4 prevailing legislative requirements on prudential 5 borrowing and Best Value". 6 If we go to page 95, please, this is Appendix 3 of 7 your report. Sorry, yes, page 95. 8 That's a reference to the CIPFA Financial Management 9 Model there; is that correct? 10 A. Yes, that's an extract from one of the -- it was 39 11 statements of good practice at that time. The statement 12 L1 is the strapline statement, and the supporting 13 questions that help determine the actual strength of 14 whether or not an organisation meets the attributes that 15 are required of in that statement. 16 Q. Thank you. Now, was this guidance in force at the time 17 of the tram project, so between roughly 2002 and 2014? 18 A. Yes, it was. There was further iterations to this 19 model, the latest one in 2016. But this would have been 20 the model that would have been available. 21 Q. Thank you. Back to page 4 of your report, please. The 22 second bullet point in 1.7, we see a reference to 23 guidance on: 24 "The role of the chief financial officer in Public 25 Sector Organisations." 141 1 If we go to page 119, this is Appendix 5. Is this 2 the guidance you were referring to? 3 A. Yes. 4 Q. Was that guidance in force at the time of the tram 5 project? 6 A. This guidance -- that publication was dated 2011. 7 Before that we had an internal -- as the regulatory body 8 for our own members, we had our own internal standards 9 on the role of the CFO. This was merely to extrapolate 10 that into a publication. 11 Q. Thank you, and I think I recall Mr McGougan in his 12 evidence or witness statement referred to guidance 13 coming from CIPFA for the Chief Financial Officer. We 14 can perhaps check back to see the document he provided 15 to us. 16 Back to page 4 of your report, please. The third 17 and last bullet point there: 18 "CIPFA Governance Mark of Excellence Standard ..." 19 If we can go, please, to page 152 of your report, is 20 that the guidance you referred to there? 21 A. Yes. 22 Q. If we go, please, to page 153, we can see, if we go up 23 the top half of the page, it was published in 2004 by 24 the OPM, which was the Office for Public Management 25 Limited, and also CIPFA. Also please go to page 157. 142 1 In paragraph 1 we can see how this came about, and the 2 final sentence in paragraph 1: 3 "The role of the Commission was to develop a common 4 code and set of principles for good governance across 5 public services." 6 So that's that guidance. 7 Back to page 4, please, of your report. Above 8 paragraph 1.9 you set out the evidence you considered 9 and you say: 10 "Primary sources of evidence consisted of: 11 "Document Review." 12 If we can go to page 85 of your report. This is 13 Appendix 1. I think there you list the documents you 14 took into account in preparing your report; is that 15 correct? 16 A. Yes. 17 Q. I think you explained earlier how this set of documents 18 came about. I think in short it was a combination of 19 the Inquiry providing certain documents, the Inquiry 20 thought may be of assistance to you, and you also 21 identifying other documents yourself; is that correct? 22 A. It is. 23 Q. Is it also correct to say that you have not had regard 24 to any of the transcripts of evidence or statements or 25 further documents beyond what's set out in Appendix 1? 143 1 A. No, no, we haven't. We first lodged a draft with the 2 Inquiry, 1 August 2017. And we purposely have ignored 3 any of the oral witness testimony that's come out in the 4 Inquiry. We've had due regard to written testimony that 5 had been provided. I think there's a listing of those 6 witnesses that have given written testimony before 7 these -- before the public hearings. 8 Q. Yes. We are really back to the point we discussed at 9 the beginning, that you recognise that Lord Hardie is in 10 a better position in assessing all of the evidence, 11 because he has had a chance to consider all of the 12 evidence? 13 A. Absolutely. 14 Q. Thank you. Back to page 4 of your report, please. The 15 second bullet point under 1.9, you refer to CIPFA data. 16 What is that a reference to? 17 A. Certainly we looked at some aspects of good practice 18 that we already have, primarily a financial management 19 model, the role of the CFO, the governance mark of 20 excellence, prevailing good practice that we have, and 21 standards that we have for our members, chartered public 22 finance accountants. 23 Q. Thank you. I have been asked to clarify two other 24 matters. Is it correct to say that you have not 25 reviewed the financial management and governance 144 1 arrangements of other tram and light rail schemes in the 2 UK? 3 A. No, we haven't. But we've had due regard to high level 4 history of the way these projects turned out to give us 5 some sort of context, but other than that, we haven't. 6 Q. Thank you. I've also been asked to clarify, have you 7 had sight of the report that Professor Flyvbjerg 8 provided to the Inquiry, either a draft or final 9 version? 10 A. No. We haven't seen it. We are unaware of its 11 existence. 12 Q. Thank you. Although I think I should say you have had 13 the benefit of sitting in on at least part of the 14 professor's evidence this morning; is that correct? 15 A. It is. Indeed, we actually identified the professor for 16 his work on project lock-in previously, and we refer to 17 it in the body of our report, as we think that the 18 attributes of the behaviour of project lock-in were 19 evidenced within this project. 20 Q. Thank you. We will come back to ask you about 21 project lock-in in a little more detail later. 22 Back to page 4 of your report, please, in 23 paragraph 1.11. Just trying to go over and understand 24 your methodology. Here you say in 1.11: 25 "Applying the relevant standards of prevailing good 145 1 practice, we were able to identify 12 specific financial 2 management related activities identified from available 3 inquiry evidence which potentially did not meet good 4 practice." 5 I think these 12 areas are then listed in the table 6 that follows; is that correct? 7 A. It is. 8 Q. Then in 1.12 you say: 9 "Comments on these headings are outlined below (and 10 are discussed more fully later in the report)." 11 I think then I'm not going to go into this in 12 detail, but we're now on page 5. If we just then scroll 13 through page 5 to 10, I think we'll see you summarising 14 bullet points your conclusions under each of these 12 15 good practice areas. So that's page 6 and on to page 7, 16 we can see the sub-headings there. We can see all that 17 for ourselves. 18 Page 8. Go on to page 10. We then see halfway down 19 the page, "Overall conclusions". I would like to ask 20 you some questions about some of these, please. 21 In paragraph 1.15 you say, the third line down: 22 "The inherent assumption demonstrated by some CEC 23 officials within witness evidence suggesting that the 24 tie special purpose vehicle would be able to insulate 25 CEC from the assumption of risk was fundamentally flawed 146 1 in the context of the political and operational 2 considerations within which this ambitious 3 infrastructure project was being managed." 4 Now, what do you mean by the inherent assumption by 5 some CEC officials suggesting that tie would be able to 6 insulate the Council from the assumption of risk? 7 A. From the evidence that we had available to us, it became 8 clear that the connectivity between the Council and its 9 wholly owned subsidiary wasn't as it should be, and that 10 behaviourally there were -- there was some difficulty in 11 transferring information between them. Partly on -- 12 justified under grounds of confidentiality. 13 But witness evidence statements from 14 Donald McGougan, the then Finance Director, and 15 Tom Aitchison, the Chief Executive, clearly suggests 16 that tie was responsible for managing the risks, and we 17 would not agree with that. We do not think that the 18 Council can abrogate its risks for the management of 19 public money, and the wholly owned company, albeit 20 a different legal entity structure, in itself doesn't 21 insulate the Council from risks. 22 But the flavour of the evidence provided by these 23 two officials of the time suggested that tie provided 24 distance between the Council and the project, that tie's 25 existence was partly in play to manage the project and 147 1 to absorb the risk on behalf of the Council. 2 Q. Now, against that it may be suggested that the Council 3 having set up a team of experts, it was for that team of 4 experts to identify and manage the risks at least on 5 a day-to-day basis? 6 A. Absolutely, but the overarching responsibility lay fully 7 and squarely with the Council for the management of this 8 project, and it was public money. They were the conduit 9 for central government grant, and for all other projects 10 that the Council has to undertake, they're fully 11 responsible for it. 12 Yes, the private limited company set up to attract, 13 in terms of remuneration and the skill base, set up 14 a vehicle to provide the correct skill sets and 15 individuals for this highly complex infrastructure 16 project. 17 That aside, the chief officers of City of Edinburgh 18 Council were still responsible for the overall 19 management of the company that the Council owns. 20 Q. Does it amount to this: that while the Council could 21 delegate the day-to-day responsibilities for the 22 project, the Council couldn't delegate the ultimate 23 responsibility for the project? 24 A. No. No, but for that to -- for that to actually work, 25 there needs to be strong connectivity and 148 1 communications. There needs to be continual dialogue 2 and control over performance and accountability. 3 It has certainly been a feature for some local 4 authorities in the UK perhaps 10, 15 years ago, when 5 special purpose vehicles first came on the scene, the 6 relationship management between the Council and the 7 special purpose vehicle. But certainly the 8 collaboration, the ownership and the overarching 9 management is now better across the UK public sector on 10 these things. 11 Q. Yes. I was going to broaden the question a little bit, 12 asking -- putting the tram project completely to one 13 side, are there inherent difficulties in local 14 authorities exercising effective control over special 15 purpose vehicles, arm's length companies or whatever? 16 A. Theoretically it shouldn't be. Certainly there's an 17 added level of governance that's required for setting up 18 a special purpose vehicle. Certainly some public bodies 19 use them to -- for a specific purpose that allows them 20 to employ people in different terms and conditions and 21 able to set a parameter around a timeline to deal with 22 a specific project. 23 But the chief officers ultimately are still 24 responsible for the special purpose vehicle. 25 The finances of any special purpose vehicle, if 149 1 they're wholly owned, are still embedded within the 2 balance sheet of the Council. So within the Council's 3 finances, tie Ltd, all their activities in the balance 4 sheet, income and expenditure, it is within the 5 consolidated group accounting position. 6 So it's not -- it's not a position that any local 7 authority can change. There is a direct accountability 8 for it. 9 Q. Again, in general, and without any reference to the tram 10 project, what steps can and should be taken by a local 11 authority to exercise effective control over a special 12 purpose vehicle? 13 A. It's certainly all about the management of risks, and 14 the management of the special purpose vehicle. So it 15 doesn't -- its journey doesn't diverge from the 16 Council's main focus and objectives. It's steered and 17 properly managed from the start of its inception to when 18 it is terminated. 19 There's no -- there's nothing inherently different 20 to any approach of a public body working with another 21 entity that it creates. It's just direct management. 22 Q. Now, finally on this point, and again as a generality, 23 it may be suggested that it ought to be easier for 24 a local authority to exercise effective control over 25 special purpose vehicle that is providing services that 150 1 a local authority has experience of providing itself, 2 for example social care, roads repair, cleaning services 3 or whatever. 4 In addition to that, it's harder for a local 5 authority to exercise effective control over a special 6 purpose vehicle that's delivering something the local 7 authority has little or no experience of delivering. Do 8 you have any comments on that suggestion? 9 A. Yes, there's some mileage in that some high level 10 successful examples were really borne out of services 11 that local authorities were already delivering. 12 Glasgow Life is a good example of that, where all 13 leisure activities, art galleries, et cetera, in Glasgow 14 were supplied through a special purpose vehicle and -- 15 in preparation for the Commonwealth Games, that allowed 16 the entity to bring to it synergies that perhaps 17 wouldn't have been available for the local authority. 18 Some of the core local authority services are 19 statutory, and there's inherent risk of failure in some 20 of these statutory services. In Scotland it's been 21 difficult to see how local authorities could do that 22 under the special purpose vehicle. 23 Having said that, in other parts of the UK, local 24 authorities have basically gone down special purpose 25 vehicles to provide some of the more difficult services. 151 1 In housing, housing has got a good track record of being 2 provided through special purpose vehicles. Leisure 3 trusts, et cetera, have been good. 4 Where it gets more difficult is the areas where the 5 risk is high for failure. It's not necessarily about 6 money. It's about the risk of non-performance on 7 service delivery. 8 Q. Thank you. I would like to move on to page 11 of your 9 report and pick up on one or two points here. So again 10 we're in your overall conclusions section. 11 In paragraph 1.17, and in the final sentence here, 12 you say: 13 "However, in relation to the Tram Project, 14 compliance with prevailing good practice on governance 15 and financial management was found to be variable over 16 the currency of the project and partial at best." 17 Would it be fair to say that this in short is your 18 overall or headline conclusion? 19 A. Yes, it would, although in paragraph 1.20, that would -- 20 1.20 and 1.21 would be probably, if we tie that in, that 21 would draw the conclusions roundly. 22 Q. I understand. Then if we go, please, to paragraph 1.18, 23 we're back to the issue of project lock-in. We can see, 24 about halfway down this paragraph, you say: 25 "We are strongly of the view that project 'Lock In' was 152 1 a key inhibiting behavioural factor. Project Lock-in is 2 a behavioural dissonance where objectivity in 3 decision making is impaired due to decision makers and 4 advisers being unable (through behavioural influences) 5 to consider all available options. This tends to happen 6 when commitments or investment made are deemed to be too 7 large to warrant [significant] change or termination of 8 a project regardless of the merits of considering such 9 options." 10 Can you firstly please explain the concept of 11 project lock-in as a generality before then going on to 12 say how you apply that concept to the tram project? 13 A. Project lock-in is a behaviour dissonance. It's 14 a suboptimal behaviour, where decision-makers or 15 advisers to decision-makers cannot objectively see all 16 available options, and objectively assess the correct 17 decision, the most optimal outcome, and that might be to 18 terminate or change the original scope into something 19 that barely resembles the project in the interests of 20 economy, efficiency and effectiveness. 21 There's been -- there were several points in 22 a timeline that highlighted the potential for 23 project lock-in to be in play within the project. 24 Within the initial determination of the scope of the 25 original contract, the haste that -- we called undue 153 1 haste for members to agree the schedule for price 2 changes for the May 2008 committee did seem to indicate 3 that there was real pressure at play for a decision to 4 be made at that time, that certainly we were aware of 5 media attention for the lack of work at certain points, 6 and no doubt there would have been significant pressure 7 on decision-makers to show that they were managing this 8 project properly. 9 It was high level, high profile, and the option of 10 not delivering on this project might have been too great 11 for some to objectively see what was in front of them. 12 The other factor was the financing of this 13 particular project. With an initial amount set aside 14 and agreed by the Minister of 375 million in 2006, which 15 changed in 2007 to half a billion, so that was centrally 16 funded money. Most of that half a billion pounds would 17 have been a component of the Scottish Government's block 18 grant from Westminster. So the source of that would 19 have been tax. 20 Then following the re-assessment of the whole 21 project at various stages, we ended up with a gap 22 funding position to 776 million from approximately 23 545 million, 500 million being central government grant, 24 45 million being a sum to be met within City of 25 Edinburgh Council's capital programme from developer 154 1 contributions and some asset sales. The balance to be 2 borrowed, typically that would have been from the Public 3 Works Loans Board, which is a component of the UK Debt 4 Management Office, and there seemed to be justification, 5 reasons to justify borrowing the gap that appeared to be 6 not too painless at the time. 7 The option of termination, we believe, wasn't 8 properly considered at various junctures in this 9 project. Certainly the haste and the pressure to agree 10 to the deal in 2008, we think, was indicative of 11 project lock-in. Even when the new Chief Executive 12 arrived and there was the Mar Hall discussions in 2011. 13 The actual spend at that point in time was 14 significantly less than the final outturn. So there 15 seemed to be significant pressure on producing 16 a particular outcome. There was some reasons, 17 accounting reasons, for not considering termination 18 which we do not agree with. 19 But certainly behaviourally, from the witness 20 statements that we have seen, there seemed to be 21 indications at certain points in time that 22 decision-makers were unwilling to objectively consider 23 termination or something that looked radically 24 different. 25 Q. I'll come back to the issue of termination shortly, but 155 1 dealing with the concept of project lock-in as 2 a generality, so again putting the tram project to one 3 side, are there any steps that can be taken to avoid or 4 reduce the risk of project lock-in? 5 A. It certainly needs external scrutiny. It needs to be -- 6 it needs to be challenged on those who are involved in 7 the project management itself. Ideally it should be 8 independent. It should be independent and there should 9 be objectivity in it all. 10 You would expect that some of the dependencies of 11 those that are involved in the particular project would 12 not be such that they would be unduly influenced anyway, 13 whether it be remuneration or longevity in appointment, 14 or any other dependency or link. There needs to be an 15 element of robust objectivity in the management of 16 a project to the extent that those involved don't feel 17 as if, in terms of just a quote, they're in too deep and 18 they can't get back out for all sorts of reasons, 19 political, media or whatever. 20 It just needs independence and challenge within the 21 process. 22 Q. Thank you. That's very clear. 23 Next paragraph, please, page -- page 11, 24 paragraph 1.19. That starts by saying: 25 "We do not believe that the option for terminating 156 1 the project in 2010/2011 was given adequate 2 consideration ..." 3 I'll pause there. Why do you say that? 4 A. First of all, we didn't really see any evidence of 5 costing the potential legal liabilities of terminating 6 the project. There were certainly costings on 7 differentials on claims, on work carried out to date. 8 But we really didn't see any assessment of how much it 9 would cost to terminate the arrangement with the 10 consortium, which is certainly indicative of not really 11 having it on the radar. 12 Now, it may well be there. It may be we just don't 13 have it. Certainly we would have expected it to be 14 discussed. We didn't actually see it in the minutes of 15 the Board meetings, that it was there. It may well be 16 the case that the overall focus was on delivering the 17 project. We did get the feel that delivering, 18 physically delivering the project was prime. The 19 finance was a secondary consideration. Partly because 20 of the way it was funded perhaps, and maybe the direct 21 or indirect link to taxpayers bearing the cost of the 22 project itself. 23 Q. Could I also clarify perhaps some points of detail on 24 the termination option. 25 When you refer to terminating the project here, do 157 1 you mean terminating the Infraco contract and abandoning 2 the tram project entirely, or do you mean terminating 3 the contract and instructing a different contractor to 4 complete the works, or do you mean both options? 5 A. Potentially both. I mean, in terms of option appraisal, 6 one has to consider every option. 7 Certainly the way it was looking, and because of the 8 complexity and the timeline, it might have been the 9 first option, Mr Mackenzie, that might have been the one 10 that would have come to mind there. 11 But secondly we would have expected all alternatives 12 to be considered. 13 Q. When you say termination ought to have been considered, 14 do you mean only if there were good legal grounds for 15 doing so, or do you mean looked at purely from 16 a financial perspective that ought to have been looked 17 at, regardless of whether there were good legal grounds 18 for doing so, if that would have been the more 19 cost-effective or cheaper option? 20 A. Probably the latter. I mean, it was only going to work 21 if you get legal resolution anyway in terms of 22 negotiating a termination to the contract, and even then 23 one would need to assess the overall liability for that, 24 which may well be significant. 25 Q. Yes. So one can't ignore whether there are good legal 158 1 grounds for doing so, because that then is relevant to 2 the question of legal liabilities, which one then has to 3 factor into any financial assessment? 4 A. Absolutely. 5 Q. Thank you. 6 Going back, please, to paragraph 1.19 and the second 7 part of that first sentence, you say -- I'll start from 8 the beginning: 9 "We do not believe that the option ... nor was the 10 financial implications and funding requirements arising 11 from meeting the project forecast spend of approximately 12 GBP776.5 million first highlighted in the CEC accounts 13 of 2010/2011." 14 So a similar question, firstly, why do you say that 15 you don't believe that there was adequate consideration 16 given to the financial implications and funding 17 requirements arising from meeting the project forecast 18 additional spend? 19 A. First of all, as an observation, the project had 20 significant difficulties in establishing forecasting in 21 the early components of the timeline there: to actually 22 forecast in 2010/2011, exactly forecast almost the 23 outturn for the project itself is a bit difficult to 24 accept. But what I'm actually saying there is that the 25 financial implication of funding requirements, there 159 1 seemed to be a justification in borrowing the gap to 2 776 million, and effectively the Finance Director at the 3 time, just before he retired, had suggested it was 4 prudent to borrow this amount of money because of 5 headroom within the Council's capital programme, the 6 fact that interest rates were historically low, and that 7 effectively now was a good time to borrow, if we needed 8 to borrow. 9 It was almost as if the affordability aspect of 10 borrowing was totally subservient to the need to deliver 11 this extra spend, which was well over the original 12 concept. 13 When members were asked to agree to the contract in 14 2008, they were -- we were only talking about half 15 a billion. We weren't talking about 50 per cent over 16 that. 17 So in terms of the ability to finance that, my 18 comment there was we do not think the financial 19 implications and the funding for that were appropriately 20 considered. 21 From the evidence we were able to assess, the 22 financing of it seemed to be a moot point. It seemed to 23 be a fairly minor consideration. 24 CHAIR OF THE INQUIRY: Sorry, could I just clarify what you 25 mean by the fact that you were surprised about the 160 1 figure of 760.5 million being highlighted in the 2 2010/2011 accounts. Where would that appear? Would 3 that be in the balance sheet? 4 A. No, that was -- my Lord, that was a note to the accounts 5 in terms of where the project was going, and that to 6 effectively fully implement the project in its fullness, 7 it would take another three, four years, and the 8 estimated outturn cost for the overall project cost 9 would be 776.5 million. 10 That was narrated in the accounts at that time, 11 which we've found a bit coincidental, that it ended up 12 there, given the fact that the project struggled so 13 badly to get its forecasting right in the early years. 14 CHAIR OF THE INQUIRY: Thank you. 15 MR MACKENZIE: Thank you. 16 The second part of paragraph 1.19 reads: 17 "Indeed, the approach adopted in securing the 18 borrowing of gap funding of GBP247.2 million beyond the 19 agreed [CEC] exposure of 45 million and the original 20 Scottish Government Grant funding of GBP500 million does 21 not appear to recognise the opportunity costs to CEC of 22 committing such additional revenue financing costs of 23 repayment and interest." 24 Can I ask, what do you mean by the reference to the 25 opportunity costs? 161 1 A. That's basically the options foregone by making 2 a decision to complete the project. The Council could 3 have used that equivalent sum to do something else with 4 it. The fact that it didn't obviously precludes 5 a decision in what it could have used the money for. 6 Q. What is the basis for you reaching the conclusion set 7 out there that the approach adopted in respect of the 8 funding gap did not appear to recognise the opportunity 9 cost to the Council? 10 A. From the evidence we looked at, it felt as if the 11 borrowing to finance as capital expenditure was not 12 going to be a problem. 13 In essence it works out at -- the equivalent of 14 a Band D council tax payer is about just over GBP75 per 15 head per year, the borrowing being delivered over 16 30 years. 17 That almost a quarter of a billion pounds worth of 18 financing could have been used for other service 19 activities of the Council. It didn't feel as if the 20 overall funding capability for the Council for 21 particular projects were ranked in a way to look at 22 other options. Given the fact this project had been 23 well under way, it was almost as if, as in 24 project lock-in, that there was no other alternative 25 but, just like any other financing, whether it's 162 1 a personal financing issue or a corporate financing 2 issue, when you borrow a sum of money, it precludes you 3 necessarily for borrowing the same amount for another 4 purpose. Effectively you're further into debt, and if 5 you want to borrow on top of that, you're further, 6 further into debt. 7 So it basically -- your indebtedness increases 8 because of your decision. 9 CHAIR OF THE INQUIRY: Do I understand that this comment 10 here is related to the loss of opportunity to spend on 11 revenue projects, or revenue expenditure, rather, so 12 that if you take GBP75 per head for community charge 13 payer, that's the cost of the borrowing of whatever it 14 is, the borrowing. Whereas if they didn't borrow, then 15 that GBP75 per head for community charge payer would 16 then be available as income to provide other services 17 that the Council are statutorily obliged to provide. 18 A. Yes, my Lord. It's equivalent to GBP75 of Council Tax 19 income. 20 The overall financing of it is 429 million, 21 including -- it is -- that original borrowing of 247 22 plus interest over 30 years comes out at 429 million. 23 There's financing of about GBP15.5 million to 24 GBP16 million a year for principal and interest costs. 25 The Council could have used that to finance a school or 163 1 whatever alternative. 2 The issue really is that GBP16 million is a revenue 3 spend for principal and interest. So that hits the 4 general revenue account for the Council. So that never 5 goes away over the 30 years, and that's -- in terms of 6 revenue spend, money that could have paid for other 7 things. 8 CHAIR OF THE INQUIRY: Yes. 9 MR MACKENZIE: So does it come to this: that in deciding 10 whether X should be spent on a particular project or 11 item, a body should also consider whether the money 12 would be better spent on other projects or items or 13 services? 14 A. Absolutely. It's a prioritisation exercise. Public 15 bodies up and down the land have to do it, in budget 16 setting every year. 17 The point is really made here that this is not 18 insubstantial. It's a quarter of a billion that turns 19 into 429 million over 30 years. It feels -- when you're 20 putting it between GBP15 million and GBP16 million 21 a year, in context of the Council's overall revenue 22 expenditure, quite modest. But it's that whole -- it's 23 the whole attitude to long-term borrowing that impressed 24 upon us to put that comment in there. 25 Q. So in short, is it your position that it's a matter of 164 1 good practice that in considering whether to spend money 2 on X, one should also consider if it would be better 3 spent on Y or Z? 4 A. Absolutely. If there's an option not to spend, and yet 5 you've got to consider doing nothing as well. What is 6 the optimal use of resources for the Council? And it 7 felt as if, given the context to this project, that 8 there was -- it wasn't really an option not to go down 9 this route. 10 Q. Thank you. Back to your report, please, in 11 paragraph 1.20. I think we have touched upon this 12 paragraph previously. I think you drew our attention to 13 that as another one of your headline conclusions; is 14 that correct? 15 A. Yes. 16 Q. Then at paragraph 1.21, please. I'll read this out. 17 You say: 18 "In summary, whilst some aspects of good governance 19 and financial management were in place such frameworks 20 were significantly undermined prior to 2011 by an 21 absence of a robust CEC led strategic planning and 22 contract management capability, including operational 23 and financial performance management." 24 Just for the avoidance of doubt, can you explain 25 what you mean by that sentence, please? 165 1 A. Yes. 2 Q. It might be helpful for you to see the first part of it 3 as well if it's possible to bring up both pages on the 4 screen at the same time, please? 5 A. A frustrating aspect of the project was that some of the 6 framework for governance was in place, but in operation 7 they were ineffective. There seemed to be behaviourally 8 difficulties in communication between officials from tie 9 and CEC. In terms of financial management, tie seemed 10 to be left to get on with it, with a very light touch 11 from the Council's finance team. 12 That changed over time, which was good, and 13 certainly contributed to a betterment in the overall 14 rigour of management. 15 But for a crucial period in the early part, early 16 and mid part of the life of the project, there was 17 a lack of robustness in financial management control. 18 Looking at the evidence that we had at our disposal, 19 it was difficult to see if there was any financial 20 performance management over tie over the early and mid 21 years. You know, costs that were being incurred by tie 22 appeared not to have been budgeted for, particularly the 23 pension deficit. 24 That sort of -- that sort of position should never 25 exist where -- the Council overseeing the company with 166 1 significant admin expenditure, I think ended up with 2 95 million or so, that should have been tightly 3 controlled, albeit that the framework for that 4 expenditure would not mirror the Council's own 5 structures and frameworks. 6 Within the early days, there was a framework 7 agreement between tie and the Council that would have 8 indicated that good practice should have prevailed. 9 There was certainly an undertaking from tie to use best 10 endeavours to apply good management and value for money, 11 and deliver value for money. There was no real 12 mechanism to enforce that. We didn't really see CEC 13 having rigour in taking tie to task for its ordinary 14 performance management of itself, which we would have 15 expected. Yes, within the governance arrangements like 16 the Tram Project Board, CEC were represented by 17 Mr McGougan and some other officials from time to time, 18 and they did have a continuous role there. But there 19 was no sense of real input, and real control, and whilst 20 we would accept that some of the individuals that were 21 appointed to tie were very good in their own field, we 22 didn't feel that there was the oversight from the 23 Council owning the special purpose vehicle and the 24 rigour that was needed. 25 Q. On that point, what controls would one ordinarily expect 167 1 to see in place for a local authority exercising 2 effective control over a special purpose vehicle? 3 A. Certainly an input to the SPV's corporate management 4 team, senior management team. That should be a standing 5 meeting where there's the Council's senior 6 representatives at it. There should be a continuous 7 dialogue there, a two-way flow of information. 8 Some of the witness statement evidence was really 9 quite difficult to read in some parts of this indicating 10 real push back and distrust in terms of the relationship 11 side of -- between the two, the lack of information 12 coming back, tie making decisions to seek legal advice, 13 and basically not fully tracking it with the Council. 14 It's where that lack of combined focus and direction and 15 the sort of parting and dissonance there, it doesn't 16 contribute well to the project. 17 In terms of your question, we would expect councils 18 to be very close to the SPVs that they operate, and most 19 do this very, very well. For certain purposes they 20 bring external help in, which is sometimes needed to 21 substantiate some of the SPVs, particularly those that 22 are set up as trusts. But direct control should 23 nevertheless be manifest itself at least at officer 24 level. 25 Q. You also, I think, later in your report said you could 168 1 find no objective evidence of formal performance 2 appraisal. 3 Now, is that part of how a local authority can 4 exercise effective control over a special purpose 5 vehicle, formal performance appraisal? 6 A. Well, certainly being the owner of a special purpose 7 vehicle, you would expect a local authority to have some 8 control over performance management and the ability to 9 influence remuneration arrangements at the SPV. 10 There's all sorts of good practice required of that, 11 notwithstanding the fact that local authorities are high 12 profile in the areas they operate in terms of media. 13 There needs to be real accountability of a performance, 14 and most authorities are able to apply sufficient 15 influence without having to secure control by overt 16 means. 17 MR MACKENZIE: My Lord, I'm conscious that I may be another 18 20 or 30 minutes. It may be an appropriate time to 19 break. 20 CHAIR OF THE INQUIRY: Yes. We are going to have a break 21 for the sake of the shorthand writers. We will resume 22 again at 3.15. You will get a cup of tea or coffee if 23 you want. 24 (3.00 pm) 25 (A short break) 169 1 (3.15 pm) 2 CHAIR OF THE INQUIRY: You are still under oath, Mr Fair. 3 MR MACKENZIE: Thank you, my Lord. 4 Could we go next please to page 22. Just some 5 various bits and pieces in your report I would like to 6 ask you about, please, Mr Fair. 7 At page 22, in paragraph 3.15, we can read the first 8 part for ourselves in relation to: 9 "Financial close on the Infraco contract was 10 achieved on 14 May 2008." 11 A reference to the statement from the CEC Director 12 of Finance. You then go on to say: 13 "The introduction of the pricing provisions in 14 Schedule 4 and subsequent negotiations around it led to 15 the whole dynamic of the nature of the original contract 16 being changed. Whilst there were late changes to the 17 bid submission the CEC Director of Finance '... did not 18 see, or seek, the version of the Infraco Pricing 19 Schedule (Schedule 4) in existence at that stage'." 20 That was a quote, I think, from Mr McGougan's 21 written statement from "did not see" to the end of the 22 sentence. 23 You go on to say: 24 "Given the significance of this late change, we 25 would have fully expected the CEC Director of Finance 170 1 (who was at the core of the financial oversight) to have 2 made himself fully aware of the potential impact of 3 pricing schedule changes." 4 How? 5 A. Well, certainly you would expect, if the Finance 6 Director was not able to dive into the detail, get one 7 of his staff to avail him of the likely impact of the 8 change. There seemed to be a feeling that there was 9 a lack of understanding of what it actually meant in 10 terms of the potential exposure and additional costs. 11 So certainly the Finance Director, if I could refer 12 you to one of our good practice standards, the CFO 13 standard, the Finance Director is expected to help the 14 authority manage its risk. It's not a silent 15 expectation. It's a requirement. 16 This is so significant. We would have expected the 17 Finance Director to at least consider the assessment of 18 the quantum of risk here, and what this really meant in 19 terms of the best scenario and the worst-case scenario 20 for exposure by the overall project. 21 It didn't feel as if that was the case and we make 22 the point that there seemed to be undue haste to accept 23 the terms of this late change, and indeed, members were 24 given the recommendation on the basis that this was 25 preferred terms they would be accepting, when in fact it 171 1 seemed to be setting the scene for immediate problems as 2 soon as it was agreed, given a level of the 3 specification still to be worked out and designed by the 4 contractor. 5 Q. Just to explore that little, if one has a very busy 6 Director of Finance dealing with any number of things, 7 on the one hand, and in a very large contract one has 8 a particular schedule which perhaps contains particular 9 provisions which create risk, how is a Director in that 10 position supposed to become aware of that detail in the 11 contract that creates a risk? 12 A. The role of the Chief Financial Officer set down by 13 CIPFA requires the arrangements within the organisation 14 to be such that the Chief Financial Officer is able to 15 be fully conversant with all matters. 16 This necessarily means having a structure there of 17 specialists to support him or her. Because of the 18 complexity, it doesn't abrogate from the responsibility 19 here, no matter how difficult and complex this is. 20 Mr McGougan, I think, was close to this over the 21 early life of the contract up to he retired. In the 22 minutes of the Tram Board, he certainly was 23 a significant feature throughout. 24 Given the significance of this pricing change and as 25 a way to, I think, get round some sort of impasse with 172 1 the consortium at that time, it seemed as if the Finance 2 Director was willing to accept the consequences, 3 whatever that would have been at the time. But it 4 doesn't feel, and the evidence that we have seen to date 5 doesn't show, that there was a full understanding of 6 what was being agreed to; and the fact that Mr McGougan 7 accepted that he hadn't actually seen it seems to me 8 that there wasn't a full understanding of what was being 9 agreed, and yet Mr McGougan was a co-author of the 10 report to the Policy Committee in 2008, strongly 11 recommending that the members agree to the latest 12 position. 13 CHAIR OF THE INQUIRY: If Mr McGougan were one of three 14 senior executives, including the Council Solicitor, to 15 whom the Chief Executive had delegated the 16 responsibility for being satisfied, would that reinforce 17 what you're saying? 18 A. Well, the Chief Financial Officer in Scottish local 19 authorities is a statutory role under Section 95 of the 20 Local Government (Scotland) Act 1973 as the proper 21 officer responsible for ensuring that appropriate 22 financial management arrangements are put in place. 23 So the CFO has a statutory role to discharge, just 24 as the Monitoring Officer has in another context. But 25 we would have expected, given our standard on the CFO, 173 1 one of the three principles of the CFO is that the CFO 2 must be actively involved and able to bring influence to 3 bear on all material business decisions, to ensure that 4 immediate and longer term implications, opportunities 5 and risks are fully considered, and alignment with 6 organisation's financial strategy. 7 At that point in time, I think it would be difficult 8 to argue that that was achieved. 9 MR MACKENZIE: As a generality for a Director of Finance as 10 the Chief Financial Officer to understand the risks 11 arising in a contract being entered into by a special 12 purpose vehicle, presumably a director in that position 13 could look to his own staff to advise him of the risks, 14 also the staff of the special purpose vehicle, but also 15 perhaps consider obtaining independent external advice 16 as well as a check? 17 A. Absolutely. Because there doesn't feel to be the full 18 understanding and alignment here, it kind of further 19 emphasises what we're also saying in the report. This 20 gap between tie and CEC at the time, that appeared to 21 prevail, and because of that lack of continuity, it may 22 well have been difficult for Mr McGougan to be fully 23 aware of this. 24 But it certainly doesn't take away from the fact he 25 should have, and he was also the co-author of that 174 1 report recommending acceptance. 2 So if he was recommending acceptance for something 3 that he didn't fully understand, then I do think there's 4 a departure from a required standard. 5 Q. Although from Mr McGougan's perspective, it may be that 6 he thought he understood the risks at the time, but 7 subsequent events changed that view, but we can go back 8 over and check Mr McGougan's evidence in that regard. 9 Can we move on, please, to another point. The next 10 page of your report at page 23, in paragraph 3.19, 11 raises a different matter, the question of Scottish 12 Government grant, and you start by saying: 13 "We further concluded from the evidence available to 14 us that there may have been concern that if Scottish 15 Government Grant allocation had not been used, it may 16 have been recoverable and Council funded. Although 17 conditions of grant may well have created the potential 18 for grant to be recoverable, we could find no evidence 19 that this was likely to occur. We are also not aware of 20 central government grant being recovered from 21 a delivering public entity as a result of non-compliance 22 with grant conditions in the context of an 23 infrastructure project of this size." 24 Now, to pause there, while you can find no evidence 25 of that happening, presumably it must at least have 175 1 remained a risk that if the project wasn't completed in 2 accordance with the grant conditions, the grant may have 3 to be repaid? 4 A. Absolutely. We have not seen any of these grant letters 5 that do not have express conditions in them that don't 6 allow the funder, the grantor to recover the money, for 7 non-compliance, it is definitely always there. One 8 would expect that. 9 In fact, it certainly was in evidence when we looked 10 at the grant agreement between the Scottish Government 11 and CEC. 12 That said, there was no indication that the Scottish 13 Government were wanting to recover any grant that had 14 been claimed. Local authorities will always receive 15 grant retrospectively, subject to the submission of 16 a properly vouched grant claim. So there's always 17 a time lag between receiving, applying for the grant and 18 receiving it after the activity has been discharged and 19 expenditure has been made for vouching purposes and for 20 control purposes. 21 In that period, what normally the Council would do 22 in their accounts is to accrue it as grant income 23 received, if there's some certainty that there is no 24 difficulty. 25 But at various points, even up to about 2009, we 176 1 were talking about a sum of about 229 million being 2 spent, perhaps not being fully claimed at that point. 3 221,581,000 had been spent up to 31 March 2009. That 4 would have been within the envelope for this particular 5 grant. So there was significant commitment still to be 6 discharged on that. 7 Certainly it would have only, I think, crystallised 8 as a potential problem if there was a significant 9 divergence of opinion between the Scottish Government 10 and the Council further on in the -- around the 11 expenditure side, perhaps nearer 2011, rather than -- 12 certainly not at financial close in 2008. So I don't 13 think there was any prospectivity of an issue of a grant 14 being recoverable on that basis. 15 Q. Can I pick up also please, in the last line on the page, 16 the sentence: 17 "Given the way local authorities are able to 18 flexibly manage their cash flows on capital programme 19 work through highly developed Treasury Management 20 practices we do not see that the timing of grant claim 21 would have or should have had any material impact on 22 decision-making on the tram project." 23 I understand that. But could I raise a separate 24 point, please. There may have been concern around the 25 time of financial close that if financial close was not 177 1 achieved sooner rather than later, then the portion from 2 the grant that was available on that financial year may 3 not be carried over to the next financial year. 4 Do you have any comments on that suggestion? 5 A. No, certainly there's no -- if there's a commitment by 6 the Scottish Government to fund up to a particular 7 quantum, the fact that the grant hasn't been used within 8 a particular year, or it is lower than the forecast that 9 was given to the Scottish Government, it certainly 10 doesn't suggest that there's any loss to the Council, 11 certainly in the accounts of the Scottish Government. 12 There would be a commitment for the overall sum, but 13 obviously only the actual spend on grant made. 14 I don't think there was any problem or prospectivity 15 of that being an issue for Edinburgh city. The fact 16 that the spend was behind wouldn't have made any 17 difference, I don't think, to the commitment that had 18 been made by the Scottish Government. 19 Q. The other point perhaps to bear in mind in relation to 20 the tram project, I think, is that while at one time it 21 had been intended to achieve financial close in 22 January 2008, by May 2008, when the contract was entered 23 into, the project was already in the next financial 24 year. So presumably on the face of it, it's hard to see 25 why the grant wouldn't have been available at any time 178 1 in that financial year? 2 A. It would have made no difference. 3 Q. Thank you. Separate point, please, at -- 4 CHAIR OF THE INQUIRY: Sorry, ultimately the decision to -- 5 whether or not to ask to reclaim the grant, ultimately 6 would that be a political decision? 7 A. It certainly would be -- my Lord, it would be a decision 8 for the Scottish Government. 9 Where the grant conditions weren't being met, 10 there's obviously the potential for it to be recalled, 11 but I think they would have to have due regard to the 12 financial position of the City of Edinburgh as well 13 before trying to recover grant. 14 At the point in time in 2008, we were nowhere near 15 the actual spend that we were two years later. So 16 I think it would have still been within the realms of 17 being manageable. 18 CHAIR OF THE INQUIRY: Thank you. 19 MR MACKENZIE: Thank you. 20 Page 40, please. We have touched upon this point 21 earlier and I apologise if I'm going over old ground. 22 In paragraph 3.61 you say: 23 "We understand that the consequential impact of 24 borrowing GBP247.162 million (GBP231.66 million over run 25 spend plus GBP15.5 million on shortfall on developer 179 1 contributions) over 30 years equated to some 2 GBP14.3 million per annum - a total of some 3 GBP429 million in principal and interest payments - 4 interest payments being approximately GBP6.578 million 5 per annum." 6 A few lines down you say: 7 "We understand that the revenue impacts equate to 8 approximately some GBP75.43 per annum of Council Tax in 9 relation to Band D equivalent tax banding." 10 Now, we have touched upon that earlier, but just so 11 I understand what is meant by that, could we perhaps go 12 to Appendix 4 at page 114, please. Here I think you set 13 out the calculation that arrives at the figure of 14 GBP75.43. I can look then at the next page just to see 15 an overview of that calculation. 16 Then again the next page as well. 17 We can see, in the middle of the page, in bold text: 18 "Total cost per Band D equivalent ... for 30 years." 19 GBP75.43. So that's the detailed calculation, but 20 if we go on to the next page, at page 117, we see a much 21 simpler calculation or sense check. Just so I can 22 understand that, we can see the figure of GBP15 million, 23 and that I think represents the cost of the overspend on 24 the tram project; is that correct? 25 A. No, the GBP15 million was just an equivalent -- it was 180 1 just a sense check on the principal and interest which 2 would have hit the revenue account every year. It is 3 15.5. We move on to a wee bit more precision on it, so 4 when we did the sums in the correct way, we got a rather 5 more precise figure, GBP75.43, which is further up the 6 appendix. 7 Q. Yes. At a very simple or general level, the 8 GBP15 million per annum over 30 years represents the 9 additional borrowing and interest in relation to the 10 tram project; is that right? 11 A. Yes. 12 Q. Then underneath that, the line, number of Band D 13 equivalents, the 196,435, is that the number of Band D 14 Council Tax payers in Edinburgh? 15 A. Correct. 16 Q. When one divides the 15 million by the 196,435, one 17 arrives at the GBP76.36; is that correct? 18 A. Yes. Yes, that is what the calculation maps out at. 19 Q. So in your report, when you refer to the revenue impacts 20 equating to approximately GBP75.43 per annum of council 21 tax in relation to Band D, that's based on the 22 hypothesis that the additional expenditure is met only 23 by Band D Council Tax payers. So it's simply an 24 illustration. In reality, of course, the other band 25 taxpayers will be making a contribution as well? 181 1 A. It's the median point, yes. Absolutely. 2 Q. So essentially the greatest number of Council Tax payers 3 are in Band D? 4 A. And they would equate to GBP76 roughly. 5 Q. I understand, thank you. 6 Back to the main body of the report, please, at 7 page 45 and a separate point. 8 In paragraph 3.80, please, now, about halfway down, 9 towards the right on line 4, you say: 10 "Given the emerging challenges with financial 11 modelling and staffing changes arising from tie's own 12 structural changes we believe that CEC should have 13 acquired more assurance that the figures reported were 14 accurate and that assumptions around forecasting were 15 appropriately stress tested." 16 Now, to pause there, can you explain why you say 17 that and what you consider ought to have been done? 18 A. Yes. The Finance Director at CEC placed significant 19 reliance and in retrospect, undue reliance on the 20 finance director at tie at the time. Certainly there 21 was information produced that we had sight of, but it 22 seemed to be at various points accepted rather than 23 tested. 24 I think it was only in the latter stages of the 25 project that CEC finance staff were on it and 182 1 effectively more assigned to policing the -- and testing 2 the figures coming out of tie, by which point a lot of 3 the difficulties had arisen. There had been some 4 turbulence and turnover in tie, and as a result of that, 5 there seemed to be a lack of connectivity between the 6 two finance functions. As a result of that, we would 7 have expected, particularly when there was absolute 8 knowledge of people leaving and changing, for the 9 finance team at CEC to be even more concerned about the 10 rigour of financial monitoring. 11 Q. Thank you. Then the paragraph underneath that, please, 12 3.81. In the second sentence you say: 13 "There is significant evidence of operational and 14 financial monitoring reporting being based on what 15 appears to be 'stand-alone' spreadsheet analysis. Undue 16 reliance on manually manipulated data worked within 17 spreadsheet analysis significantly increases the 18 potential for human error to manifest itself and can 19 disproportionately consume staff resources. This can 20 arise as manually adjusted figures are added to system 21 sourced data. The higher the number of adjustments, the 22 greater the potential for error to occur." 23 Can you just explain, please, what you mean by that 24 passage and what you say ought to have been done. 25 A. We looked at some of the information from tie at the 183 1 time on monitoring, and it became very apparent that 2 a lot of the monitoring presentation had been derived 3 from Excel spreadsheet working, and whilst there is no 4 problem with that, because of the extent of modelling on 5 Excel worksheets and work books, we didn't get the feel 6 that a lot of the information came from the base 7 financial systems. 8 When you move further away from the core data on the 9 base financial system, you get the potential for figures 10 to be presented to decision-makers which are distorted, 11 changed, manipulated. 12 Mostly with the best intentions and professionalism, 13 of course, but the more spreadsheet working is used, 14 there is the potential for error. There is the 15 potential for figures to be changed in such a way that 16 it fits the particular mood of the moment, rather than 17 coming from the core systems to show the actuality. 18 The quote here was just an observation on the extent 19 of the workbook Excel reporting that came from the tie 20 finance team. 21 Q. So what ought to be done if too many spreadsheets are 22 considered a bad thing, and I have no difficulty with 23 that view at all, what ought to be done? 24 A. Well, certainly there should be assurance on the 25 validity of the core numbers. Where the numbers are 184 1 coming from, where is the data coming from, and good 2 financial management systems basically kind of hardwired 3 to capture all data, so that the only manipulation that 4 gets applied to them is the way that they are reported, 5 rather than manual intervention to change figures for 6 profiling or for information that becomes known. 7 Certainly influencing factors, good systems are 8 flexible enough to incorporate influencing factors and 9 changes, but what you do get with these systems is an 10 absolute trail, so that you can actually see the 11 changes. You can see the reason, the rationale for the 12 changes, and that there's full tracking and 13 transparency. 14 With Excel spreadsheets, there isn't that. You have 15 the capacity for a member of staff to change a couple of 16 figures within that, that can have a real significant 17 change in the outcome of information that's presented to 18 decision-makers. 19 If you get that wrong, then there's obviously -- the 20 scope for human error increases when you have to make 21 more manual changes. Then can be very, very difficult, 22 and we have seen -- we have seen figures presented that 23 are manifestly wrong and have made a significant impact 24 on decisions that have been made arising from then. 25 Q. Thank you. 185 1 Mr Fair, I appreciate there's a lot more detail in 2 your report we could go into, but for my part I'm 3 content to take the rest of your report as read, and 4 I have no further questions. 5 A. Thank you. 6 Questions by CHAIR OF THE INQUIRY 7 CHAIR OF THE INQUIRY: Could I ask about the 2010/2011 8 accounts. 9 You said there's a note to the effect that the 10 outturn cost of the project was expected to be 11 GBP770 million plus. 12 First of all, are the accounts published and 13 available to the public at large? 14 A. Yes, the Inquiry team should have those notes. 15 CHAIR OF THE INQUIRY: Well, yes, but I mean, as a matter of 16 course, the local authority has to publish its accounts. 17 A. Yes. 18 CHAIR OF THE INQUIRY: So any member of the public can have 19 access to the account and would they see that note? 20 A. They would see it if they had cause to look for it. 21 It's certainly there. 22 CHAIR OF THE INQUIRY: Yes. 23 The other thing is: is there a timescale for local 24 authorities to provide information in their reports 25 about projects? For instance, the tram project final 186 1 accounts, when would you expect these to be available 2 generally? 3 A. Certainly in terms of the Code of Local Government 4 Accounting, which is a prescriptive code that had been 5 structured by CIPFA, in the notes to the accounts, there 6 is a requirement that group accounting and information 7 on the latest position on the financial status of 8 a project should be reported proximate with the final 9 accounts of the authority. So where -- the authority's 10 financial year in Scotland is 31 March, we would expect 11 within the annual report and accounts of the authority 12 for -- a full tracking and exposure of a project of the 13 size of the tram project every year. 14 As I'd indicated earlier, the financial position of 15 tie is incorporated within the consolidated balance 16 sheet of the Council within the group accounts. 17 There is regulation around the way that's 18 constructed to make it meaningful, and we would expect 19 significant disclosure by the local authority on its 20 wholly owned companies and the way that the accounts are 21 incorporated and consolidated together, to give a full 22 group account situation. 23 Certainly it in some way tries to mirror group 24 accounting in a commercial world, where you have 25 subsidiaries within a group and a holding company. It 187 1 tries, as far as possible, to mirror that. 2 CHAIR OF THE INQUIRY: So at the end of each financial year, 3 for the local authority, it would -- it should -- these 4 accounts should include the total expenditure in that 5 financial year on the tram project, be able to find that 6 by looking at the payments to tie, or would you look at 7 the tie accounts or would you look at both? 8 A. No, the tie activities should be within the City of 9 Edinburgh Council's financial accounts. 10 The position on the actual assets that were created 11 and managed by tie are actually reflected in City of 12 Edinburgh Council's balance sheet itself. So the trams, 13 the actual tram units, the tram infrastructure itself, 14 is in the balance sheet, and the balance sheet notes 15 should show the additional capital expenditure for that 16 particular activity every year as it builds up. 17 CHAIR OF THE INQUIRY: So when you get to the end of the 18 project, it's completed. 19 A. Mm-hm. 20 CHAIR OF THE INQUIRY: So would you then expect, in the 21 accounts ending with that financial year, the March 22 after completion, these accounts should show what the 23 total for expenditure was -- 24 A. Yes. 25 CHAIR OF THE INQUIRY: -- during that year on the tram 188 1 project? 2 A. Both the accumulation, because we're creating an asset, 3 my Lord, and effectively we should be looking for the 4 balance sheet showing GBP776 million worth of asset 5 creation in the balance sheet, that written down by 6 impairment, depreciation, disposal, or even addition to 7 that. Where investment needs to happen to make an 8 asset, its payback, be useful for more than a year, you 9 would capitalise that. 10 So for specific projects like the tram, it should be 11 easily identifiable within the balance sheet of the 12 local authority that these are tram assets. And we 13 would ask for the breakdown. It wasn't overly apparent 14 what it was, but we were able to get enough of 15 a breakdown to see that. 16 There's also notes in the accounts as a requirement, 17 because of the Code of Practice, that the activities of 18 the subsidiaries, the high level narrative is put in. 19 So there will be the activity for tie Ltd as well as the 20 creation of assets in the CEC accounts. 21 So it's all there in the accounts of the local 22 authority. Sometimes quite difficult to know where to 23 look, but it should be there. And it's audited by 24 Audit Scotland. So Audit Scotland would expect 25 compliance with the Code, and if there was any 189 1 departures from it, they will leave a note if they don't 2 qualify the accounts, which they haven't. 3 CHAIR OF THE INQUIRY: Did I understand you earlier to say 4 that there was no asset register? 5 A. Yes. We asked for details of asset registers for the 6 tram assets, and we got a letter back, 31 March, from 7 the Council to say that they didn't have -- but they 8 were in contemplation. They were taking steps to create 9 them. 10 An asset register, in terms of good practice, is 11 a prerequisite for decision-making. It records the 12 detail of each asset created, the value of it, 13 expenditure, its valuation, its expected life, its 14 economic payback in terms of the economic benefit for 15 having it, and an assessment should be within that asset 16 register of its state, so that decisions could be made 17 on investment. If it was housing stock, we'd expect the 18 authority to have a stock maintenance inventory on it 19 and assess what the deterioration in that stock should 20 be and what the investment requirement would be. 21 So the asset register is not just about the local 22 authority accounting requirements. So that you have the 23 figure in the balance sheet, and you calculate 24 depreciation, impairment, et cetera. It's about 25 actually managing the asset properly, and you can only 190 1 effectively have the right figures in the accounts if 2 you know how to manage the asset, what its life is 3 likely to be, what its investment requirements -- or 4 indeed its total replacement. 5 So traditionally local authorities have been weak in 6 this area, because they've used information and assets 7 primarily for financial reporting, when in fact the good 8 practice requires that assets are managed through 9 a detailed asset register. 10 You can only have the correct figures in the 11 accounts if you know the status of your assets, and when 12 we asked the question of City of Edinburgh Council, they 13 confirmed that they didn't have one. 14 So primarily the figures that they have are for 15 financial reporting purposes in the balance sheet, 16 because we asked the question: there was 27 tram units 17 procured and we understood that there were only 17 18 needed; where is the additional ten; what condition are 19 all these tram units in? And we couldn't get anything 20 back on that. So they did concede that they were 21 working on creating robust asset registers. 22 CHAIR OF THE INQUIRY: The reason I was asking the questions 23 about the accounts for the tram project, Mr Connarty 24 gave evidence, and in the course of his evidence I asked 25 him if there was a final account for the project, and he 191 1 said that he gave an undertaking that he would -- that 2 it would be produced. It hadn't been finalised. Would 3 that surprise you? 4 A. It certainly would, because when you create an asset on 5 a balance sheet, you have a defined figure. If you've 6 properly recorded the creation of that asset from cradle 7 to grave, you will know exactly in your balance sheet 8 how much has been expended on the creation of that 9 asset. 10 I have no idea why that should be a problem because 11 it should already exist. Other than that we had -- 12 I explained earlier, my Lord, that we had reservations 13 over the final figure because of perhaps some items that 14 had been incorporated. 15 CHAIR OF THE INQUIRY: You had problems with the final 16 figure because things like reinstatement of pavements on 17 Leith Walk were not included. They were included under 18 the -- would be apparently paid under some other budget 19 for roads or whatever. 20 A. It may well be the case, my Lord, that that might have 21 been capitalised under infrastructure, other 22 infrastructure of assets. 23 The worst-case scenario in that is that that 24 capitalisation, there would have been a borrowing 25 requirement for that to finance it, which gets us back 192 1 to over and above our GBP75, our Council Tax D Band. 2 There may be an increased exposure. We don't 3 precisely know that. We certainly had issues over legal 4 costs and overheads. The Local Government Code of 5 Accounting Practice requires the capturing of all costs 6 specific to the creation of that asset. 7 So we would expect, if there was full compliance, 8 that the balance sheet should have exactly the 9 expenditure on tram assets, and that would include 10 overheads, it would include project management. 11 The definition in the code is really about anything 12 that gets an asset into service. That could be anything 13 that gets a specific asset to its location and in 14 service that has an economic payback to the user. 15 CHAIR OF THE INQUIRY: Another thing I wanted to ask about 16 was that you said at one point that there were 17 accounting reasons given for not considering 18 termination, but you didn't agree with them. 19 A. No. The Finance Director indicated two particular 20 reasons for -- he did say borrowing was the lesser of 21 two evils, and he'd indicated that if a decision was 22 made to terminate, all expenditure that would have been 23 in the balance sheet that would have been capitalised 24 would have been required to be unwound and charged in 25 the one year to the revenue account for City of 193 1 Edinburgh Council, and the bit I do agree with is the 2 fact they would have struggled to contain that from 3 reserves. 4 We don't agree with that because of effectively the 5 way the financial -- the Code of Practice Local 6 Authority Accounting Terms of Finance and International 7 Accounting Standards, which it's part of, basically 8 determined that an asset's capitalised when you mean it 9 to be capitalised. If it's not capital, it becomes 10 revenue. Anything up to the previous financial year 11 that's capitalised doesn't become revenue. It 12 effectively is impaired and through statutory 13 mitigation, we could actually deal with that impairment 14 over a number of years, rather than it being written off 15 in one year. 16 The fact that a goodly part of that was Scottish 17 Government grant was also another issue that we 18 contended with in that the likelihood of unwinding that 19 right of way in a one-year hit to the Council would have 20 been remote. 21 The fact that an accounting representation issue 22 would feature as a main component of decision-making, we 23 didn't agree with, my Lord, because it shouldn't have 24 made any -- it shouldn't have had any influence in the 25 decision-making of whether to proceed with the project 194 1 or terminate it, because we don't believe there would 2 have been an immediate requirement to hit the revenue 3 account with all the capitalised expenditure made to 4 date. 5 CHAIR OF THE INQUIRY: So if the suggestion was that the 6 termination of the project would result in the capital 7 expenditure becoming revenue, and it having to be repaid 8 in one year with a resulting consequence of about 9 80 per cent increase in community charge, are you just 10 saying that that's not correct? 11 A. That's not correct, my Lord. 12 CHAIR OF THE INQUIRY: But if that information were given to 13 councillors, that there might be an 80 per cent increase 14 in community charge if they decide to terminate the 15 project, do you see that that might well influence the 16 decision of the Council? 17 A. Certainly if they were given minimal information in 18 other aspects of the project, if you add it to that, it 19 may well have influenced them not to consider 20 termination. Certainly we don't agree that if an asset 21 is created early in the financial year and there's 22 a decision to terminate it and it doesn't become an 23 asset, then yes, it's a charge to revenue in that 24 financial year. 25 I suppose, in the currency of a financial year, the 195 1 later on in a financial year you make that decision, the 2 potentially more problematic it becomes. 3 But certainly it's our position that all 4 capitalisation up to the financial year that has just 5 gone would not be unwound immediately and charged to the 6 revenue account of CEC. It would be there would be 7 statutory mitigation, and yes, the asset may well be 8 impaired, but other factors regarding the state of the 9 asset, any additional secondary use or any scrap value, 10 would need to be considered too. It's not just a case 11 of simply charging it to any revenue account. 12 CHAIR OF THE INQUIRY: Thank you very much. Mr Martin, 13 I think you had some questions. 14 MR MARTIN: Yes, thank you, my Lord. I have no questions. 15 CHAIR OF THE INQUIRY: Thank you very much, Mr Fair. That 16 concludes your evidence. As I say to each witness, 17 I think, you are still under citation, and it would be 18 technically possible for you to be recalled, but I think 19 that's unlikely. 20 I would just like to thank you for the assistance 21 that you've given. 22 A. Thank you, my Lord. 23 CHAIR OF THE INQUIRY: If you care to go back to the office, 24 I will just deal with other matters. 25 (The witness withdrew) 196 1 CHAIR OF THE INQUIRY: That concludes the oral evidence. 2 MR MACKENZIE: It does, my Lord, yes. 3 Closing remarks by CHAIR OF THE INQUIRY 4 CHAIR OF THE INQUIRY: Well, we've now reached the stage 5 where the evidence sessions have concluded, and we will 6 move on to closing submissions. 7 I have published a note about closing submissions 8 from which you will see that they are to be made on 23 9 and 24 May. There are two matters that I will mention, 10 but I'll put a further note out amending the note. 11 We just heard from the solicitors acting for Parsons 12 Brinckerhoff that they intend to provide the Inquiry 13 with a written submission in accordance with the note 14 and they may update it also in accordance with the note, 15 but they will not be giving any oral submissions. 16 The other matter that I should mention is that 17 Carillion, the liquidator of Carillion has been in touch 18 with the Inquiry to say that they do not intend to 19 participate in the Inquiry. 20 I had left them out. I assumed that would be the 21 case. I left them out of the timetable, and there was 22 a problem with the Scottish Ministers, because of the 23 availability of Mr Barne being in the division for the 24 second day. 25 So I have rearranged that timing so that Mr Barne 197 1 will be giving any oral submission he wishes on the 2 24th, and the core participants, represented by 3 Beltrami, will give their submission on the 23rd. 4 I think that concludes everything I need to say at 5 this stage. So we'll adjourn to 23 May at 9.30. 6 (4.01 pm) 7 (The hearing adjourned 8 until Wednesday, 23 May 2018 at 9.30 am) 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 198 1 INDEX 2 PAGE 3 PROFESSOR BENT FLYVBJERG (sworn) .....................1 4 5 Examination by MR LAKE ........................1 6 7 MR STUART FAIR (sworn) .............................126 8 9 Examination by MR MACKENZIE .................126 10 11 Questions by CHAIR OF THE INQUIRY ...........186 12 13 Closing remarks by CHAIR OF THE ....................197 14 INQUIRY 15 16 17 18 19 20 21 22 23 24 25 199