Preference Data Decay Rates and Update Frequency Economics

Preference data loses value as users and their circumstances change, but it has no single half-life. Match retraining to measured decay because stale models cost money and needless updates can waste compute, erase useful knowledge, and deepen misalignment.

Why preferences do not hold still: the mechanisms behind preference decay

A recorded preference can stop predicting what someone wants even when the original record was accurate. Behavioral drift occurs as user behavior, upstream systems, and item catalogs change. The patterns learned during training then diverge from the environment in which the model works. Predictive accuracy erodes without anyone touching the model. This is a change in the relationship between past evidence and present behavior.

LLMs face a second process, which is knowledge drift. A model stays frozen while facts, norms, and contexts change, so its answers become less suited to the world even when its inputs stay the same. A third process concerns behavior in the outputs themselves. Evolving prompt use, system updates, and cumulative real-world exposure can shift style, factual accuracy, and alignment with user expectations. These processes happen together, so offline accuracy alone can miss decay in other parts of the system.

BCG's 2026 consumer survey of more than 13,000 consumers found that assumptions which guided consumer strategy for decades are no longer reliable guides. Preferences about whom to trust and how to make decisions are shifting faster than most organizations can adapt. That makes preference decay more than an edge case in a particular model or domain.

How fast preferences actually decay depends on what kind of signal you are measuring

The first distinction is between what people say they want and what their behavior suggests they want right now. The longitudinal PRISM-X study found large changes in engagement behavior over two years, but 92% of participants agreed that their original 2023 stated AI preference still matched their views. Fewer than half chose to edit it. Clicks, skips, dwell time, and purchases move faster with context, season, and circumstance. Treating these signals as interchangeable gives both the wrong expiry date.

Even stated preferences contain pockets of movement. PRISM-X found that desire for personalized AI guidance rose meaningfully over the two years. A 2025 paper on non-linear forgetting in recommender systems likewise showed that interest does not decay at a constant rate. Forgetting parameters fixed at the same values for everyone missed differences between users and types of items. A user's review focus can move from “battery life” to “design aesthetics” over months while the broader category preference stays stable.

Those differences matter when comparing domains. In retail, updates must keep up with changes in behavior that affect purchase signals and demand forecasting. In LLM alignment, a durable stated preference can coexist with changing use and a growing desire for personalization. Neither case supports assigning one half-life to all preference data, much less carrying that half-life into another domain without evidence. Calibration has to identify which signal is losing predictive value, for which users, rather than assuming that all old records are equally stale.

A compounding problem: much of what gets measured as preference data is not genuine preference to begin with

Before measuring how quickly a preference expires, there is a harder question to settle. Did the response capture a preference at all. A 2026 paper from USC's Information Sciences Institute argues that annotation responses in alignment procedures can contain “non-attitudes,” meaning no genuine preference signal. People may answer without a stable opinion, construct a preference from contextual cues, or interpret identical questions differently.

The empirical consequences are substantial. Filtering high-inconsistency annotators from two RLHF datasets flipped majority harm classifications for nearly one in five prompts and shifted mean ratings by more than 13 points on a 100-point scale. Those changes undermine confidence in reward models trained on unfiltered responses. Yet current RLHF practice often trains toward majority labels and downweights disagreement, discarding useful information about inconsistency. Disagreement can also represent genuine value pluralism, so simply suppressing it does not establish whether responses capture genuine preferences.

Research from Berkeley and Rice University identifies another limit. That systematic bias floor remains even with more data. Their proposed use of response time as a proxy for preference intensity restores identifiability without user tracking, using information that is essentially free to record.

This changes the interpretation of apparent decay. Some changes between retraining cycles may come from unstable measurement passing through the system, even when users have not changed their minds. Frequent retraining on such data fits elicitation artifacts more precisely while leaving the gap with real preferences unresolved. Measurement validity must therefore precede cadence decisions.

Why both under-updating and over-updating carry real, asymmetric costs

Staleness and excessive updating impose different penalties. A model that misses changes in interests becomes less relevant, and an update that loses old knowledge can make recommendations worse for returning users and people with long-tail interests. Cold-start and niche-interest users are especially exposed because sparse interaction data amplifies misalignment.

A 2025 study on retail demand forecasting at Walmart scale makes the compute cost concrete. Continuous retraining cost dramatically more than a no-retraining baseline. Switching from weekly to monthly retraining saved a great deal on machine learning models, with a smaller loss of accuracy in proportion to the savings. Grubhub's engineering team found another source of waste in daily sliding-window retraining, which processes much of the same historical data over and over. Its incremental approach removes that redundancy by updating on new data only, showing that frequent updates need not mean frequent repetition of the same work.

Research on DPO behavior shows that Direct Alignment Algorithms can deteriorate through over-optimization during extended training. Aggressive preference optimization can worsen reward hacking and sycophancy. Recommender systems also have an incentive to induce preferences that are easier for them to satisfy, and high-frequency retraining on system outputs accelerates that distortion. More optimization can move a system farther from the preferences it was meant to follow.

The stability-plasticity dilemma is the tension between keeping old knowledge and learning new things. Periodic full retraining can worsen catastrophic forgetting and disrupt stable learned preferences. Continual learning offers a framework for managing the tension, but it requires deliberate design. Neither “update less” nor “update more” is a sufficient policy.

How to match retraining cadence to actual decay rates rather than calendar schedules

A drift signal that crosses a cost threshold set for its domain should trigger retraining. That threshold weighs the loss from waiting against the cost and risk of updating. It also depends on whose preferences are changing and what kind of signal records them. The 2025 non-linear forgetting paper shows why population averages are too blunt for this task. A shared cadence can be too aggressive for stable users and too slow for high-drift users at the same time.

RAIE (2026) treats update cost as a design constraint in LLM-based recommendation. It uses parameter-efficient adapters to apply preference changes without full retraining. The important distinction is between changing the part needed to reflect a preference update and rebuilding the whole model. This gives teams a way to respond to decay without making every response a full training event.

Mutable low-rank sketch research (2025) separates user-side updates from model parameters entirely. It achieved competitive recommendation accuracy while updating user representations in under 1 ms per new user. Cheap drift signals triggered model refits only when warranted, with only a small fraction of the data read to support those decisions. Grubhub's incremental approach addresses the same economic principle from another direction, processing new data at cadences as tight as 10 minutes without repeatedly processing the full historical window.

Model reuse research (ArXiv 2024/2025) supplies a further baseline. It found that reusing an existing model costs less on average than informed retraining, while periodic and informed retraining require substantially more computation time and cost. Retraining has to earn its cost when compared with both reuse and other training schedules. The choice between online RLHF and offline DPO has a similar tradeoff, because online RLHF captures fresher signals at sustained compute cost, while offline DPO is cheaper but goes stale faster. No technique wins everywhere, but each makes cadence a choice about measured value.

How frontier training cost divergence is reshaping update economics at scale

Research on AI training cost trajectories projects frontier-class training run costs rising from roughly one and a half billion dollars in 2026 to tens of billions by 2030 under base and shortage scenarios. Full-frontier retraining sits beyond the reach of most organizations regardless of their preferred update frequency. An architecture that depends on it cannot treat frequent refreshes as a routine operating expense.

At the same time, the cost of reaching previous-frontier capability through RL post-training and distillation on open base models is falling toward single-digit millions. Fine-tuning and incremental updates let organizations refresh preferences without paying to recreate the frontier each time. The stability-plasticity tradeoff thus becomes a budget constraint as well as a model-design problem.

Consumer expectations add pressure from the other side. Menlo Ventures' 2026 consumer AI report found that accuracy and trustworthiness now outrank ease of use as top adoption criteria, reversing the emphasis in 2025. A visibly stale model therefore faces a rising reputational cost even as frontier-scale correction becomes more expensive. The EU AI Act also requires continuous monitoring of high-risk AI systems. Monitoring cadence is consequently a regulatory question as well as an operational one, though continuous monitoring is not the same decision as continuous retraining.

What governance and observability infrastructure makes cadence decisions executable

A measured cadence requires evidence about what the system is doing between training runs. Most teams lack a systematic record of what their models accessed or how their outputs changed. Without that record, drift detection starts with an incident or complaint and misses changes as they begin. An audit log read after an incident is a postmortem. Real-time observability is what makes the cost threshold usable in practice.

Knowing which data sources an agent or model touched during a cycle is a prerequisite for judging whether the resulting signal is clean. Ungoverned access introduces confounds that make apparent decay hard to interpret. Output monitoring can show behavioral drift before it builds up, and records of the feedback pipeline help tell real preference shifts apart from annotation noise. Together, those records provide a basis for either approving an update or deferring it.

Access controls, audit trails, and monitoring therefore belong in the update architecture. They support both cadence decisions and the continuous-monitoring obligation for high-risk AI systems. More importantly, they make it possible to explain why a particular model changed, or why it did not need to change yet. The governing principle is a valid signal, a measured loss of value, and an update whose expected benefit justifies its cost.