At Operis we are often asked what exactly it is we do when performing a model audit. The question arises in part given the general unfamiliarity about what comprises an audit, but also because not every audit is the same. Yes, we always ‘check for errors’, but a real understanding of our work requires a more thorough explanation.
Though every model audit has common features, the exact task at hand depends on the client’s motive and the desired outcome of the model audit. In this article, we explain why model audits are performed, briefly outline the process and scope of a model audit and highlight the benefits of using a specialist model auditor.
The rationale behind model audits
An often-cited study by Raymond Panko underlines the frequency of spreadsheet errors, including a finding that 24% of spreadsheets created by spreadsheet-savvy MBA students carried an error of some sort. Given the complexity of project finance models the chance of errors in modelling spreadsheets – whether insignificant or catastrophic – is high.
Clearly, financial models need to be verified but depending on the situation at hand there is a difference in the intensity of validation required, the degree of assurance that the final model is correct, and consequently the effort expended by Operis. We classify model audits into two broad categories, depending on the outcome the client requires:
- High Level Review: performed to give clients extra confidence in a model
- Formal Model Audit: executed to provide unequivocal assurance to the client and stakeholders
It is worth highlighting that the purpose of the audit is not to say whether a deal is good or bad, but to confirm the spreadsheet reflecting the transaction is consistent with the agreements governing it. This confirmation provides the stakeholders with confidence to rely on the model’s results, and possibly the green light a funder requires to sign-off a lending decision.
The High Level Review: a stamp of approval
Many clients come to us for an independent opinion on a model. They do so to seek comfort that a financial model does what it sets out to do, and to prevent errors from derailing a project, with embarrassing consequences. However, they do not require scrutiny of every single detail in the model. Therefore, we perform not so much an audit as a high level review.
Internal teams can and should review the models they build, but they are unlikely to have the broad experience or objectivity of a specialist model review and auditing firm. At Operis we know where errors commonly creep in, and under the high level review we actively hunt for these errors and at the same time perform a general overview of the logic of the model. Our consultants are highly likely to catch the most common errors.
Importantly, under a high level review we do not check every single detail in the model as the complex nature of models makes this a time-consuming process. Essentially, we allocate a carefully judged amount of time to rule out most errors, in line with the Pareto or 80/20 rule. The result is a cost-effective review leading to a letter of support, affirming that after close review (but not intense scrutiny) Operis deems that the model is fit for purpose.
The Formal Model Audit: providing assurance
Under some scenarios, a letter stating that “the model is good enough” is simply not sufficient. Instead, the entire model needs to be checked. As a typical example, in the weeks leading to bid submission or financial close, a lender will require that a model is audited in the most comprehensive and formal way possible. A formal model audit is the only way forward when stakeholders require assurance that a financial model is free of material error and as fully representative of the real world.
When performing a formal model audit we combine our team’s breadth and depth of expertise with an incredibly rigorous process that looks in close detail at the entire model, along with a number of project and financing documents related to the transaction which impact the model. So, although we examine the same points as we do under a high level review, we apply far closer scrutiny. In a formal model audit, we issue a report stating what is wrong with the model, leaving it for the client to correct, and repeat the process several times until the model is free of material error.
The final, audited outcome confirms that the finished model is free of material error, so much so that we can back our findings with a liability cap of up to £20 million GBP.
Introducing the Operis model audit process
Considering the two forms of review described, the actual process is broadly similar irrespective of the motive behind the model audit. Here we outline the process we follow when performing an audit, with the next section covering in more depth the individual factors we scrutinise.
A top-down analysis of every model
The Operis model audit process is proprietary and has been developed over a period of thirty years. It revolves around a central spreadsheet called the Audit Working Paper (AWP). We construct an AWP for each audit, clearly identifying the principal outputs of the client model and constructing various reproductions of the client’s financial statements in a manner which provides the maximum clarity, using steps that are known to test results and highlight errors along the way.
Essentially the AWP is constructed to demonstrate that Operis can match the outputs of the client model by intentionally following a different method. It is a process of triangulation: finding a destination from two different source points. This is in contrast to a more commonplace model auditing method: that of checking formula cell by cell, a method which is both tedious and ignores the broader story the model is describing about the transaction.
Our big-picture approach eliminates errors
The top-down process we follow allows our analysts to take account of the big picture. A bottom-up, cell-by-cell process can easily miss certain errors, though it will catch some. A simple example of an error missed by cell-by-cell analysis is a duplicate cost deduction: accounting for cost in one cell, with the same deduction repeated in a different cell, on a different sheet.
Merely examining the model cell by cell will not necessarily find an error in the overall result, and the likelihood that the duplication will be noticed can be particularly slim if the model is large, and if different individuals are tasked with evaluating different worksheets. Our process also reduces the reliance on perfect diligence by a single consultant as our time-proven method focuses on subjecting the most common modelling faults to two or three ways of testing, ensuring that errors are flagged at some stage.
The Scope of Work
When we engage with a client we discuss exactly what auditing work will be completed, but the agreed scope of work will broadly match one of the two audit categories defined earlier. The ground covered differs depending on the level of the review.
Our full scope of work is outlined here, but we can summarise it as follows:
High Level Review
During a high level review, we will:
- Review the logic and the integrity of the model in selected areas and consider whether the calculations made are reasonable and hold consistent with the assumptions in the model, including operational, functional and economic assumptions.
- Examine the consistency of P&L, cash flow and balance sheet projections across the life of the asset
- Review the use of key financial indicators and broadly review accounting and tax treatments
- Produce a report which summarises and discusses issues found, including a severity grade for each issue
- Review one revision of the model and check that the changes made resolve the issues we identified
- Provide a Letter of Comfort, including any notes pertinent to the client model revision
Formal Model Audit
In performing a formal model audit we will, in addition to the points covered under the high level review, also:
- Check the integrity and logic of the entire model, leaving no element unscrutinised
- Match every model assumption against the supporting information contained in the extracts of project and financing documents
- Fully check all model assumptions against local GAAP or IFRS accounting standards and relevant tax legislation while also ensuring tax treatments are consistent with accounting treatments
- Conduct a deep analysis of key financial indicators, including a review of cover ratios and other loan covenants
- Review a range of sensitivity runs to make sure that lender sensitivities are correctly accounted for and that changes in outputs correctly reflect changes in inputs, such as foreign exchange and inflation rate fluctuations
- Review several iterations of the model, providing issue reports each time, with the intention to arrive to a final, fully audited model that is 100% accurate
- Provide a Formal Opinion Letter and attend financial close if required.
Importantly, it is only the more thorough scope of work tied to a formal model audit that leads to an agreed cap of liability. Operis typically offer amounts in the range of £5 million to £10 million as liability in connection to the Formal Opinion Letter that is delivered once the audit is complete.
Why use a specialist model auditor?
Project finance model auditing is available from a range of accountancy firms and even individual consultants. However, Operis is the only firm of size that truly specialises in model auditing – we perform over a hundred model audits for clients from around the world each year. Clients recognise the breadth of experience at Operis which is why we are able to regularly fill training courses where our consultants teach industry participants how to build and how to audit models.
Our concentrated focus on model auditing allows us to deploy a deeper understanding of the modelling process and we are confident that our audits uncover more issues than consultants or firms who, either due to a very broad service line or a lack of experience, simply cannot offer the same level of scrutiny. The ability to offer some of the highest liability levels in the market underlines the reputational weight of a model audit signed off by Operis.
Now that you have a better idea of what a model audit by Operis involves, and how the different requirements of our clients determine the task at hand, we invite you to contact us with any further questions. Our consultants are always available to you for advice on building models, auditing a finished model, and to advise on accounting and tax issues surrounding project finance.
