The problem
A large Connected Planning platform used AWS Forecast to generate the forecasts its customers relied on for planning decisions. Forecasting was a core part of the product, but it sat inside a managed service the team could not fully inspect or tune.
That created three practical problems:
- Limited control. The team could not adjust how models behaved for the specific patterns in their customers' data.
- Limited visibility. When a forecast looked wrong, there was little the team could do to explain or correct it.
- Dependency. A capability central to the product was tied to a third-party service and its roadmap.
The goal was to bring forecasting in-house: a system the team owned, could reason about, and could improve over time.
What we built
The work was led by Sachin Singh, Fork Labs' founder, and replaced AWS Forecast with an in-house time-series forecasting system.
The approach followed three steps:
- Start with the data. Before choosing models, we studied the platform's historical time-series data to understand where a general-purpose managed service was falling short for this use case.
- Build a pipeline the team owns. We designed and built the forecasting system in Python so that every stage, from data preparation to model output, was visible and tunable by the platform's own engineers.
- Prove it before switching. The new system was evaluated against real historical outcomes, so the decision to move off AWS Forecast rested on measured accuracy rather than assumptions.
Owning the system also meant the team could keep extending it as new customers, data sources and forecasting needs appeared, without waiting on a vendor.
Results
The in-house system reached 86% average forecast accuracy across the platform's forecasting workloads, and took over from AWS Forecast as the engine behind the product's forecasts.
Beyond the headline number, the platform gained:
- Full ownership of a core product capability, with no dependency on a black-box managed service
- The ability to tune and explain forecasts when customers ask why a number looks the way it does
- A foundation to build on, so future improvements are an engineering decision rather than a vendor request
If a critical part of your product runs on a managed ML service you cannot tune or explain, it can usually be brought in-house without giving up accuracy.