How Video Frame Rate Data Violates Linear Regression Assumptions A Case Study

I was recently wrestling with a set of video processing metrics, specifically looking at how compression artifacts correlated with perceived quality across different capture frame rates. My initial instinct, as it often is when you have a nice, clean dataset, was to reach for a simple linear regression model. It seemed straightforward: input frame rate (FPS) as the predictor, and some calculated visual distortion score as the outcome. After all, isn't the relationship generally assumed to be monotonic, perhaps even linear, as you move from the choppy 15fps captures to the buttery smooth 60fps sequences? I ran the initial diagnostics, expecting the usual satisfaction of checking off assumptions, but something immediately felt off in the residuals plot—a pattern was staring back at me that screamed violation.

This wasn't just a slight wobble; it was a structural problem suggesting that the simple additive effects I was modeling were completely missing the mark when dealing with discrete changes in temporal sampling. The core issue, I realized, wasn't about the *value* of the frame rate itself, but the *nature* of the data generation process tied to those specific capture rates. Let's pause right there and think about what linear regression actually requires. It demands that the relationship between the independent variable ($X$) and the dependent variable ($Y$) be linear in the parameters, and critically, that the errors (residuals) be homoscedastic—meaning the variance of the errors is constant across all levels of $X$.

When I plotted those residuals against the frame rate predictor, the variance wasn't constant at all; it was dramatically wider in the regions corresponding to the lower frame rates, say between 24fps and 30fps, compared to the tight clustering I saw around 59.94fps. This heteroscedasticity is a direct consequence of how motion estimation algorithms behave. At lower frame rates, a single frame represents a much larger temporal slice of the original action; small, high-frequency movements that are easily smoothed or interpolated out at 60fps might manifest as severe blockiness or judder artifacts when encoded at 24fps because the encoder has less temporal redundancy information to work with across those wider gaps. Consequently, the error in predicting perceived quality from frame rate isn't uniformly distributed; the uncertainty explodes precisely where the sampling density drops significantly, violating that fundamental assumption of constant error variance.

Furthermore, the linearity assumption itself was being bent out of shape by the non-uniform perceptual steps between frame rates. Moving from 15fps to 24fps introduces a massive perceptual leap in smoothness, often because 24fps aligns more closely with established cinematic standards and human flicker fusion thresholds in certain lighting conditions. However, the jump from 48fps to 60fps, while numerically larger, often yields a much smaller, perhaps even negligible, gain in *perceived* quality improvement for standard viewing scenarios, especially after accounting for display refresh rates. A linear model treats the distance between 24 and 48 as equivalent to the distance between 48 and 60 in terms of predictive power, which simply isn't true when the underlying physical process—how the visual system integrates temporally separated samples—is distinctly non-linear and dependent on specific sampling thresholds.

This forced me to abandon the simple straight line. What I needed was a model that could account for these structural breaks and the changing error variance, perhaps something involving generalized linear models or even a segmented regression approach that treats the low-rate regime and the high-rate regime differently, acknowledging the qualitative shift in encoding behavior around standard broadcast rates. It’s a good reminder that when dealing with data derived from physical or perceptual systems, blindly applying textbook statistical tools without understanding the generative process behind the numbers often leads you down a rabbit hole of statistically significant but practically meaningless results. The data structure itself dictates the appropriate mathematical tool, not the convenience of the initial hypothesis.