Skip to content

On - FC Validation Methodology - 2026-01-22

Metadata

  • Date: 2026-01-22
  • Company: On
  • External Participants: Federico Morando (Treasury Intern)
  • Palm Participants: Emma, Giannis
  • Type: Customer Call
  • Domain Areas: Cash Forecasting, Forecast Performance, Variance Analysis

Summary

Context

Working session with Federico (ON Treasury Intern) to understand his forecast validation methodology before his internship ends. He's building a use case to present to Christoph in Zurich end of February to demonstrate forecast trustworthiness. Amanda was sick and couldn't join.

Key Discussion Points

  • WMAPE methodology: Federico using Weighted Mean Absolute Percentage Error to track forecast accuracy - weights by entity's contribution to global cash
  • Forecast validation process: Weekly downloads from both Kyriba and Palm, comparing forecast vs actuals over 13-week horizon
  • Forecast vs Forecast comparison: Comparing different forecast versions for the same week to see improvement over time
  • Heat map visualization: Shows WMAPE evolution week-by-week; decreasing = improving accuracy
  • Minimum cash line: Concept for liquidity planning - would be set annually, monitored quarterly, by entity
  • Key entities to track: On Holding, On AG, On US, plus Asia (China, Korea, Japan) - about 7-8 main entities
  • Operational vs investments split: Want to separate operational cash forecasts from investments for cleaner analysis

Pain Points

  • Google Sheets methodology hard to maintain over time - needs to transition to Palm
  • Kyriba sometimes includes investments in reports causing comparison errors
  • Need entity-level breakdown, not just global view

Feature Requests & Needs

  • Forecast vs Forecast comparison view in Palm (Giannis showed a beta version)
  • Operational cash + investments chart with minimum cash line
  • Entity-level forecast accuracy drill-down
  • Export to CSV for Excel workflows

Jobs & Desired Outcomes

Job: Build trust in the forecast to present to leadership

Desired Outcomes: - Minimize the time required to validate forecast accuracy - Increase transparency into which forecasts can be trusted - Reduce reliance on Excel for forecast validation workflows

Job: Leave a sustainable methodology before internship ends

Desired Outcomes: - Minimize manual maintenance of validation process - Increase adoption by transitioning methodology into Palm platform - Reduce knowledge loss when intern leaves

Domain Insights

  • Kyriba AI forecasting: ON hasn't activated Kyriba's ML forecasting - "still in beta" and Federico didn't get much outcome exploring it in December
  • Validation timeframe: 6 weeks considered sufficient for initial validation (vs full 13-week horizon)
  • Minimum cash concept: Would be set with FP&A, stored in Anaplan, varies by entity based on cash pool dynamics
  • Trust = core goal: Federico explicit that the goal isn't to match Kyriba methodology exactly - it's to build trust in Palm forecasts

Action Items

  • [ ] Giannis to polish Forecast vs Forecast comparison feature
  • [ ] Schedule session with Simon & Art (forecasting experts) in 2 weeks
  • [ ] Federico to share chart ideas for operational cash + investments view
  • [ ] Weekly syncs to prepare for Christoph presentation

Notable Quotes

"Trusting the forecast is such a fundamental bit of our product. So this is very inspiring." - Emma

"We validated over the span of six weeks. Even if it's not the 13 weeks, it's fine because it's already a good span to get some insights." - Federico

"More about building trust, I would say. Because methodology is more of a draft that we can improve over time." - Federico (on whether Palm needs to match Kyriba exactly)

Full Transcript

Them: Hey.
Me: Hello. The tld. We should start soon. Pending host permission. But. You didn't receive any, like, admit TLDV notification. Now.
Them: I think. I think Jen is the host.
Me: Maybe. Hey, Federico.
Them: I can't hear you. Give me a second. Sorry. Let me try to get out and get in again.
Me: All right.
Them: Okay, sorry.
Me: Hello.
Them: Hello. Sorry. I was entering the call and the computer decided to restart just with the perfect timing, so I couldn't even delay it.
Me: Amazing.
Them: Sorry about that.
Me: Federico. Good. Thank you. Do you by any chance get admit TLDV into the meeting notification on your screen?
Them: No.
Me: I'll try and restart it. Let me know.
Them: Okay? No, actually.
Me: I have to leave and come back. One second.
Them: Yeah. Basically Monday and Tuesday because I was back in Italy. And then yesterday I had the flight at 1, so that's why I wasn't there. For our weekly send. Up, but now I'm back in Berlin.
Me: Nice.
Them: Okay? We're from Italy. Are you from. I'm from Alessandria, which is a small city in the north, but I lived in Milan for the past four years, so I was in Milan. Yeah, okay. Nice. To get a small peek in my old life and then come back, actually, but all good. Yeah. It was last holiday. Last summer holidays in Milan, near the mountain. It's so nice. Amazing. Dolomites are amazing. Beautiful. Place.
Me: Hey, my TLDV doesn't seem to start. I think it's pending host permission, but you don't seem to get any notification. Right, Federico? So let's just.
Them: Yeah. Sorry. About that.
Me: All right.
Them: All right. So imagine you got the chance to see a bit the fight that we shared last week. If yesterday to discuss it a bit or if you wanted to walk you through today? Yeah, very briefly. We discussed it with Amanda, by the way. Is she coming today or not? No, because she's sick. Okay, perfect. So then we can start, I guess. Basically what I discuss with Amanda. I was trying basically to understand the context of the Google sheets that you shared on what you're going to track in order for us to be able to replicate it on our end. Let me share the screen. Actually, I think that's easiest. Course. Okay? Because I think you discussed already a bit. But to give you a bit of a context, my aim was to actually build the methodology on validation terms. That is then going to be also applied to the liquidity plan, but as of now, it's for the validation. The thing is that it's a methodology that even if the Google sheet is actually good to overview and get insights, it's not super easy to maintain over time because then you probably you saw it and I mean it takes a bit of time to check everything and it needs to be still improved with some charts. That are bit more understandable, I would say. But the thing is, basically my internship ends. At the end of February. So my aim would have been to beat the methodology. Leave it. And before leaving, transition it to palm so that for the future, everybody who's going to use it or investigate. Its own overview in Palm. What are we going to do at the end of February is to actually present it, both the methodology and palm to Crystal. Okay, perfect. We have a call already booked. I'm going to fly to Zurich last week of February.
Me: Okay?
Them: And then we're going to present the whole project and the use case to then validate it. Yeah, seems that you seem that you put what the work here. Yeah. As you've seen it as of now, it's good to monitor week by week, the forecast, but it's not like, super still steady. I would say. Okay?
Me: Sorry. Just out of curiosity, Federico. For the final presentation in February with Crystal. What are the other key areas that you're looking to present?
Them: Sorry. The other.
Me: So for the presentation in February. What are the other key areas you're looking to present? Is there anything else? That needs to.
Them: So the idea is to present the use case because obviously share a bit more about palm functionalities, but the aim is to validate the forecast, especially.
Me: Okay?
Them: So when I say the quality of. We took this use case, we validated over the span of six weeks. Even if it's not the 13 weeks, it's fine because it's already a good span to get some insights. And then the presentation must fall, like, must be okay. We got the chance to validate the forecast for us, it's valuable. And then next steps. But it would be the point.
Me: Perfect. Thank you.
Them: No worries. So sorry, Yanis, I interrupt you? But since I wasn't there yesterday, I wanted to. Yeah, actually, we didn't discuss that through with Amanda. So if you could walk us through basically the implementation. Let's spend, let's say, 10 minutes on understanding the implementation, and then we have prepared something on our end and that we want to review and. Iterate based on that, okay? Perfect. So basically, what happens if you go to the process sheet? It's a bit more clean now, but basically every week. So we started the week 51 of 2025, so the week of the 17th, I think, of December. And basically we started downloading the forecast from both Cariba and Palm. And every week on a Tuesday since then, we had the actual we need to wait for the actual if there aren't any errors or any download problems, we usually do it on a Tuesday. And basically we download the reports. One from Palm, which is the one from the Invest, and another report that we customized and built in. So basically we get the forecast in the next week, we get the actuals. And as a KPI to monitor the difference, we established the wmaep. So now just to mention, but I'll show you better later, but yeah, in general, this is the process. So we are then uploading the data set sheet. That's just here. All the data, basically. So the goal is to monitor how the forecast is performing. So taking, for example, from week one, the forecast and then every week when we get the actuals, we monitor how is it going. And then from the second week, we start from. The second week. So basically weeks ahead is one week where if I go. 13 weeks as of week. These are basically different forecast versions for this particular week, right? No, this is like the forecast. As of that week. And then if you filter for the second week, you get the forecast as of the second week. So the goal is to also, through the categorization, see if the forecast improved or to see if, like, it's actually matching the expectations and if it's better or not. So basically, then, in the pivot, in the. In the first sheet.
Me: Very. Could I ask you still a question? So, Cariba, is it? You're working with categorization in Cariba as well?
Them: No, no, no, no.
Me: Now it's mainly palm one.
Them: Yeah.
Me: And decree before cost. Are you using any machine learning there yet? Or is that still mostly like booked actuals like Apar.
Them: Of course it works. It, it has its own machine learning system in terms of categorization. But we didn't like, we're not like working as like with you, of course with the, for the, for the forecast we were just. Because the goal is to then establish palm and in terms of forecasting going through that.
Me: I'm super curious because it looked like we were outperforming Cariba. I was just curious if the cariba for calcium you're using also have the machine learning enabled in them or if it's more based on. The data you have in other systems like AP ar.
Them: Mostly it works more on making. The categorization, like the flow from the. All the erp, from dynamics or from the other systems. Now we got the chance also to. To link it with the big carry. So it's a bit. Well, like it's more linked as of now, but at the same time. It's, like, still not super easy to, to get insights and like, and pick what you want to get from the, from the platform.
Me: M. But do you know if the actual cash forecasts. Did you activate machine learning in Cariba? Like their AI forecasting yet? Or is it? No.
Them: No. Specifically that? Not yet, because there are some AI functionalities, but they are still in beta.
Me: Okay?
Them: And we don't.
Me: All right. Okay. Sorry. That's very valuable for us. Feed their marketing. And it's just super interesting to understand if they're out yet or because we haven't. We haven't heard anyone actively using it yet, so it's just interesting.
Them: Know, of course. Of course. I got the chance to. To explore it a bit in December, but I didn't get too much outcome. But, yeah, so, yeah, I would say that.
Me: Super.
Them: And.
Me: Sorry for interrupting. It was just I was.
Them: Of course. Of course. Basically, then in the first, like, so we got the actual snapshots, the forecast, and what are we doing is just keeping the actuals in the first week, and then we link the formulas for the other weeks to that. And basically what we do is calculate the map. So instead of doing just the the normal variance. So actually minor forecast on the actual. Like we calculate the map, then we calculate the weight as of that week. So like the actual on the global of the entity on the global. And then we just waited and get the like on the volume to get like a more truthful. To understand the distribution of that particular entity to the global cast on how it's going to affect the global cash position. Right? Or, yeah. And my question is, how do you use. The other forecast versions. So we produce multiple forecast versions for the same for a particular week. Do you compare those different forecast versions or is it just mainly. Mainly for monitoring them? No, like in terms of forecast versions as the same week. You know, as of now. We're, like, getting the. Like for versions. You. You mean like including investments or just operational cash? Or let's say mainly operational cost for now. Okay. Because also, like a thing that we encountered while monitoring is the fact that, for example, from Griba, we were, there were some, like, sometimes you get some errors because some, some investments are included. So, for example, they're not, like, super truthful in terms of comparison. So what we did also with Amanda is if you go on the sheet. FC 13 weeks cash split view. Exactly. It's really beta, but it's a way to. And it would be really nice to have it in palm. Of course, the. The goal is to have it just with the operational, like, with operating cash, because it's more. I think. Yeah. The actual meaning out of the forecast. But in the end, also, we were working a lot with the investments in the past weeks. So it's fine. It was more of an idea to have also in part because then it's easy to. Since with the liquidity planning, we want to. We wanted to establish a minimum cash also for some entities, because sometimes it's kind of hard. With some entities. And how. So how do you pick the line for the minimum cost? Is it something that you decided? So basically it's not super implemented yet. But the idea was to work with FPNA because they were the ones who like established the assumptions and they will get it was the idea to include it in anaplan and then once it's decided maybe like different rates as like, I don't know. Based on how the cash pool works with some entities with others. Maybe doesn't like work like super well and establish like with rule of thumb an idea to publish it. So I have more questions. Yeah, of course. I will try to answer my best. Of course, then. Yeah, of course. And is this minimum CAS line fixed? Let's say is it like a company policy, or is it? Or does it also fluctuate over time? And how often does it fluctuate? So I think it can be, like, my idea, since it's not, like, really implemented yet. It would be to establish it at the beginning of the year. But to monitor it quarterly would be the best idea in my opinion. So it's not like that. The minimum cost changes every week. For example, according to the forecasts. No, no, no, no. Be fixed on the service.
Me: And it would be by. What's the lowest level? Is it by entity?
Them: Exactly. Oh, sorry. Does the minimum gas line change by entity, or is it like a global? This is a global, but it's really like.
Me: I think. By any, I guess.
Them: Ah, okay. Okay. Yeah, yeah, yeah. Because what should. What we should do now is actually because now the forecast is validating only the global. So like we can see from the pivot each entity. But the idea would be to put a data validation like module to investigate by the main entities. We add a selection of the main entities that we wanted to monitor. It can also give you the list should be basically the main one. So on holding on a gay and then on us, of course. And in Asia, since sometimes it's a bit tricky with China, Korea and Japan. Those should be the main ones, but it would be a pool of, like, seven, eight entities, basically. That's not a problem. That's not a problem. Hey. I'm actually curious, because this chart, actually, we can generate right now without even the new forecast version, okay? So that's very interesting. I was also interested to know about the different forecast versions because I think last time we also touched the forecast versus Forecast that you wanted to compare the different forecast versions for a particular week. Is it something that you're working on on a different, let's say, Dashboard or in another Excel? No. Basically, if you go to the main sheet, so the. And you go down. You can. Then you can see like. Yeah, a bit up. Yeah. So like there is still zero because we. We need to. How? I uploaded the actual of this weekend. So like it's modified in the other. In the original sheet. But basically what we do is calculate the average map, like weighted map. For each forecast. Okay? And this is what you use to import here? No, we if you go to WMAEP heat map. Yeah, exactly. Week by week, you see the average map for each forecast. And then gives you if it reduce. Exactly. It's. It means that it's improving. If. And then like obviously here we need to build a bit more like data to also to validate. But now it's zero because it goes down. Of course, because of that zero. But it's a way also to. See how the error is performing over time. And then in the KD Web versus part, you see the the global evolution. Of.
Me: So how would you use this practically in companionship with your forecast? I'm assuming you want to know this information to understand in some way. In what way you can trust the forecast. Is that a correct assumption or.
Them: So, yeah, basically it would be like, now that we, like, have a bit of data to work on because it's been. Basically five weeks. So we have a bit more of data to to be formulazon and investigate. It would be yes, to compare like the mapes and see how it goes. And then the best thing would be to have this view by entity. And investigate by entity how. How it's performing on those terms. I don't know if I answered your question properly.
Me: Just like, what's the reason why you're doing this exercise? Like, is it to make better use of the forecasts and feel more confident using them, knowing the limits, or is it.
Them: Exactly. As we mentioned, for example, I remember, I think the last call before Christmas with Lucia we discussed is. That would be the point as of now. To be trustful. On the forecast. And then what I'm like, what my plan would be is to actually get some insights to validate that thing. So, like, we trust it. And obviously, we use, like, way more. So we know that we. We work a lot also with you, so we know that we can trust it. But still, since we have to present it to to Christoph. We have built a use case that say okay.
Me: Yeah.
Them: We validated on those terms. We. So. Yeah, and that. That's why I'm also started working on a presentation and I don't know if. Emma, you're available. What? Giannis, if you're available to do a weekly on this, we can prepare together. Yeah, very happy.
Me: That would be lovely.
Them: Too. Like, of course, Emma, YouTube. I don't know if you maybe have time also with other stuff, but.
Me: I'll join as much as I can.
Them: Okay?
Me: Would love to. This is, to me, super interesting because. Trusting the forecast is such a fundamental bit. Of our product. So this is very inspiring. And super interesting to take part of.
Them: Y. Eah. Yeah. Federico, you may not realize it that much, but this conversation is, for us, instrumental. This work you were trying to replicate in Excel, we are trying to. Let's say, go find all ways that we can replicate it as much as we can and as best as we can in. In the platform, so.
Me: But especially draw inspiration from to see how are our users currently looking to find our trust themselves. And it's so different. Like the methodologies that different people use. And of course, of course, as much as we can make it adopt to what you need to get that trust.
Them: Yeah. And also imagine that this is, like, still, like, something that we're using, but it's still, like a draft, I would say.
Me: Yes, exactly. So it will still always evolve. Right? So that's the key. Exactly. I agree.
Them: Yeah.
Me: Well, let's not get too, like, stiff into one implementation.
Them: Of course, because I also like in basically time flight since I jump into and in a few weeks. Like my journey here is going to end and I want to leave.
Me: Of course.
Them: With meaningful that also it can be trusted by the other members of the teams and also externals over the year. So that would be trying my best on those terms. But if we. I think that if we work closely and, like, also get insights from one another, it's the best thing then to get the best outcome.
Me: Definitely. Definitely. I wonder if there's Giannis. What do you think? Is there any point in pulling Art or Simon in one call and share some machine learning?
Them: Yeah, we already have a scheduled call, so we're going to reuse the weekly meetings. Actually, because we couldn't do the presentation for the self serve Dashboard this week. It was supposed to. Simon and Art, our forecasting experts, were about to join next week, but we postponed it for the week after. So in two weeks, You will have the chance to talk to them. But if there is any need, we can schedule an ad hoc call with them as well. So if you want to schedule another call, apart from the set date that we have on Wednesdays, we can do that, and then I can ask them if they can join as well. Okay. Okay. Would be nice. Would be nice. Yeah. If there's a need, like, it would be probably, as you said, we can focus more on our. Our weekly stand up and then if there's a need also for maybe a shorter one just to check together, just to update each other. Yeah, of course. No worries at all. We'll find the time, we'll make time, and we'll figure everything out. Nice. Nice. For instance, probably this week I'll be way more and more in touch with you because I can see the dates of your final presentation is closing in, so don't hesitate to send anything our way. Perfect. Perfect. I'll try to get the best out of these last weeks. Yeah, absolutely. So, seeing the time, I'm not sure if you have a couple of more minutes, because I want to also present, of course. Basically what we did here is a forecast with forecast comparison. So what you see here basically in this table is basically the WMAP seat from the Excel that you shared. Ok. You can see basically the entities and then you can view the bank account codes per entity. This we can adjust, no worries about that. But you have latest forecast. On the left side you see the balance. On the right side you see basically. The forecast value here you see $, it's percentage. So I need to change that. But basically, it's late this week. Minus two weeks, forecast, minus three weeks, forecast, minus four weeks. So basically, these are. Different versions of a particular date of the forecast, so I set it to today for now. Just for the demonstration. And then you can pick, basically, and you can export to CSV. You can aggregate all the different forecasts. And get the variance and you can fill it out to the Excel that you used. Okay? Perfect. Now that. That looks really nice. Thanks for taking the time. And because also, what we were really looking forward is, was to actually get, like, if you want to investigate a snapshot of X Week, Now you can actually do that, like on forecast terms, and it's actually really useful. And I think that with the self serve dashboard, It can be also, like, you can also work on the data here, and if you need any specific, like, needs, you can. Yeah. Also on that part. If you can experiment with the self serve dashboard. If you need more capabilities, let us know and we'll check how we can put them in the self serve report as well. Of course, of course. But it looks really nice. Okay, so in this format, I assume that this can help you basically build the accelerite? Yes, of course. We'll polish it a bit more because it's not perfect yet. It's a beta version. And I will ping you to know when it's ready. Perfect. Perfect. No worries. No worries. And I was very curious about. Again. Investment. And because if you had shared this earlier on, we could have replicated much, much sooner. Because the data we already have, we know how much investments you have, we know how much operational cost you have. So the only thing is just drafting the chart. So to present. This next week, probably. Okay. Okay, okay. But if we come up with chart investigation modules that I'm probably going to include, since I have more data, I will share it so that we can work. Parallel here. Basically, I also have the different forecast versions per currency, so we have many ideas that we can work on. If you want, we can find the time next week to have a dedicated session on figuring out, let's say, which chart is best to use. Maybe you can come with ideas like or check with your team what metrics do you want to track? Is it gospel level? Is it, let's say regional level? Is it currency level? How much, let's say money you have or how much forecast you have on specific currency and how that involves over time. Think of these like in a plain text. In plain text and we will figure out together which art is best for you to present. Perfect. Perfect. Let's schedule something and. Yeah. Yeah, just being on slack and. Yeah, we'll make it work. Perfect. Sounds good. Nice. Any questions? For. For now. It's fine. I think we are on a good. Checkpoint. And I think next week it will come up with a really insightful session, too, because I'm going to try because I was off, off of this week, now I'm I have tomorrow. Now, today and tomorrow. To work a lot and try to build something also in the acceleroom data that I have, I will share it instantly and next week. We can get more on that, I think. Sounds good. Nice. Emma. Anything from your side?
Me: Not really. But just one question. Hope no one freaks out, but if we had some way to inform, make it easier to understand the performance of our forecast. By end of February. How important is it for you that that matches exactly your current method so that you can directly compare it to Cariba? Or is it more about building trust for the forecasts in Palm.
Them: More about building trust, I would say. Because methodology, as I told you, is more of a draft, that we can make it more, like, improve it over time. But still like being able over this month to transition like part of the methodology. At least in Palm it would be perfect. And then, obviously, working together, we can improve it.
Me: Very, very cool. Just want to make sure we're not restricted by what Cariba might be. So. So that comparison is not the core. It's more about, like, how. To be more transparent and build trust in the palm forecast. Cool.
Them: Exactly.
Me: Super amazing. Really inspiring. Thank you so much for sharing this with us.
Them: Thank you so much for actually taking the time. And we schedule something for next week. So perfect. Awesome. Nice. All right, Thanks a lot for the rest of the day and the weekends. And let's catch up next week. Nice. You too.
Me: Thank you.
Them: Thank you, guys. Bye. Bye. See you. Ciao.