My first move is to check instrumentation, not the product. A 15% overnight drop is more likely a broken event pipeline, a misconfigured tracking tag, or a timezone shift in the reporting query than a real behavior change. I pull raw event counts from source tables and compare them to the dashboard number. If they match, the data is real, and I move to segmentation: break the drop by platform, geography, acquisition channel, and user cohort to find a clean fault line. A single segment driving the whole drop points to a product issue, a release, or an external event. If the drop is uniform across all segments, I suspect infrastructure: a login service outage, a broken push notification, or a payment failure. I only form a root-cause hypothesis after I have localized the drop.
Insider read
Really testing: Whether you start with data infrastructure rather than jumping to product hypotheses, and whether you use segmentation to localize the problem before theorizing about causes.
The tell: Juniors say they would check if a new feature broke something. Seniors check instrumentation first, then segment by platform, channel, and cohort to isolate where the drop is concentrated before forming any hypothesis.
Follow-up: "Your segmentation shows the drop is entirely on iOS. What do you check next?"
Say this"Check the pipes before you blame the product. A 15% drop that appears overnight is more likely a broken event tag than a real behavior change."