How to Use AI in FP&A: 5-Step Framework for Oversight of Forecasting with ML
It seems like we’re all trying to figure out how to use AI in FP&A (properly). Can AI make your job easier? Probably. Will it require some work? Definitely.
Whether using ChatGPT, Copilot, or anything else, there’s a notion that artificial intelligence (AI) and machine learning (ML) are “black boxes” - meaning that it’s not possible to see their inner workings.
In finance, this lack of transparency poses a challenge for CFOs, Finance Controllers, and Financial Planning and Analysis (FP&A) teams, who need to provide accurate forecasts but, more importantly, need to understand and explain the underlying drivers to business stakeholders.
As we know, Finance can’t give an automation-built forecast to their business and, when asked about the main drivers in their model, just say "The AI tools said so".
To address this issue and foster trust in AI-driven processes for decision-making, it’s essential to establish mechanisms for overseeing AI in order to ensure that the work done is reliable and accountable.
But then comes the hard part: how do you create frameworks to oversee AI processes and ensure the work is done right?
In this article, I’ll explore a specific AI process - Machine Learning (ML) Forecasting.
PS: I’ve already gone over how to determine which FP&A tasks take priority when automating processes if you’re stuck on that.
5-Step Framework For Overseeing ML Forecasting
If you want to know how to use AI in FP&A, you'll need a clear plan to do it correctly. When it comes to ML forecasting, several key areas require oversight to ensure the work is done right. Here are the 5 steps I take:
1. Check your Assumptions
The first step is to validate the assumptions made during the forecasting process. This involves scrutinizing the underlying hypotheses and ensuring they align with the business context. You should evaluate whether the assumptions are reasonable, consistent, and based on relevant historical data.
For example, when I started forecasting Labor Spend during COVID-19, the model was trained on historical data but didn’t account for other things like increasing demand. For a human, this demand increase would be a “reasonable” new assumption but a machine wouldn’t “think” to include it. If the ML algorithm was a black box, then you could not see the impact of demand on the end result.
2. Check Your Data Accuracy
Accurate and high-quality datasets are paramount for reliable forecasts. As we say in Finance and Accounting, if your inputs are garbage, your outputs are garbage.
It’s crucial to verify that the data used for training the ML model is comprehensive, up-to-date, and representative of the target population. Data cleansing and preprocessing techniques should be implemented to address outliers, missing values, and other data quality issues.
In my experience, not having real-time (or as close to it as you can get), reliable data is the most common reason ML algorithms don’t work for companies. You need to have your data cleansed, standardized, and structured in a way that the ML model can work with.
3. Understand the Algorithm Rationale
If you want to know how to use AI in FP&A, you need to get inside the "mind" of the machine. Understanding the rationale behind the chosen algorithm is essential - you need to determine the algorithm's suitability for the forecasting task at hand.
You should assess whether the selected ML algorithm aligns with the data characteristics and the specific objectives of the forecast; as AI models are typically built upon predictive analytics, you need to check that the predictions actually make sense. This evaluation helps ensure that the algorithm's underlying assumptions align with the financial domain.
One way to do this is to break the algorithm into smaller parts and try to understand the math behind each of the components. Then, understand the relationship between them and the cases in which the algorithm will show certain results. If you don’t feel equipped to handle this part of the data analysis on your own, find an AI expert (engineer, data scientist, or person in a role like mine) to do the dissection for you. Alternatively, you could use a statistical analysis tool that's meant to simplify the process instead.
This rationale evaluation also needs to be combined with the next element: understanding the drivers.
4. Understand the Drivers (& be able to Explain Them)
Transparency in forecasting requires the ability to explain the drivers behind the predicted outcomes: whether you’re part of an FP&A team or the CFO, you can’t present data that you don’t understand.
You need to understand the key variables and features that contribute to the forecasted results. This explanation helps build trust and facilitates effective communication with stakeholders who rely on the forecast.
In my own process, before I start coding the algorithm, I spend 1 or 2 days with the part of the business I’m building it for. Go to a particular factory if the algorithm is intended to predict its utility consumption or go on a sales field trip if the algorithm will forecast sales at a particular store.
Once you have a feel for the department, team, or facility in question, write down all the drivers that might affect the forecast and make sure the algorithm does not spit a “final result” alone but rather, also explains which features or drivers contributed to arrive at that result.
5. Validate the Results
ML algorithms should be thoroughly validated to ensure their outputs make sense. This involves examining the model's performance metrics, such as accuracy, precision, recall, and mean absolute error. If the model's performance falls below acceptable thresholds, further refinement, retraining, or exploration of alternative algorithms may be necessary.
In the last few weeks, I have been talking to many finance professionals and they describe this last step in different ways - some people call it “gut feeling”; others, the “smell” of the results of the forecast.
Simply put, you need to use your empirical knowledge and subject matter expertise to assess if the forecast is making sense. You can use mathematical calculations as described above to do so but you also need to have the mindset of a true business partner and always build your business knowledge in order to properly assess an AI model result.
The Verdict
Give a person an ML automation: they automate for a day. Teach a person how to use AI in FP&A: they automate for life.
The use cases are many, as AI capabilities continue to grow and be discovered. But business decisions aren’t being left to the robots just yet.
The benefits of AI can make life easier for CFOs and FP&A professionals but, at present, ChatGPT still can’t do the job on its own.
If you want to stay ahead of the curve with AI implementation in finance, subscribe to The CFO Club's newsletter and receive the best guides right in your inbox.
