← All questions

Metricsstandard

Checkout conversion on an e-commerce site dropped after a release. Walk me through your diagnosis.

How to approach it

The timing points to the release, but confirm it. Compare users on the new version with users still on the old one.

How a strong answer opens

“The timing points at the release, but I'd like to confirm it. Can we compare users on the new version with users still on the old one?”

Outline of a strong answer

Try answering out loud for a few minutes before you open this.

Confirm the drop lines up with the release
Compare users on the new and old versions if possible
Break checkout into steps and find where drop-off grew
Segment by device, browser, payment method
Check errors and load times on the affected step
Decide whether to roll back while you fix

Where this question usually goes wrong

Assuming the release caused it without checking, or debating the root cause for days instead of considering a rollback while you investigate.

Learn the method

Diagnosing a metric drop in Module 2, Metrics and execution.

The full kit has worked answers in full, start to finish, for every round type. $129 once.

More metrics questions