Attribute paid orders to their originating request with durable evidence, but estimate incrementality through a controlled treatment-versus-holdout design. Report net generated sales after refunds and cancellations, disclose attribution windows, and warn when samples are too small.
- Attributed revenue and incremental revenue are not synonyms.
- Paid-order evidence is stronger than checkout or order-created events.
- Refunds and cancellations create append-only negative corrections.
- Holdout assignment and analysis rules should be set before exposure.
Build the attribution chain first
Record the request, merchant decision, offer, checkout evidence, and paid Shopify order under stable correlation identifiers. Define the attribution window and handling for fallback matches. Ambiguous fallback attribution should require review before billing or performance claims.
Reconcile net outcomes
A created checkout is not revenue and an order-created webhook is not always proof of payment. Count conversion on paid evidence. When refunds or cancellations arrive, append a correction to attributed value and any usage-fee basis instead of erasing the original event.
Use a predetermined holdout
Assign eligible requests deterministically to treatment and holdout before sending the intervention. Keep eligibility, primary metric, sample threshold, attribution window, and stopping rule stable. Compare like cohorts and show confidence intervals or uncertainty warnings.
Present metrics merchants can trust
Separate potential demand, qualified demand, offers, attributed paid orders, net generated sales, and modeled lift. Label demo values. Never turn a modeled forecast or every attributed sale into an unqualified incrementality claim.
Frequently asked questions
Is every attributed order incremental?
No. Attribution shows a traceable relationship to the workflow. Incrementality asks what would have happened without the intervention and requires a credible comparison such as a holdout.
Should refunded sales remain in reporting?
Keep the original conversion evidence and append refund corrections so gross and net outcomes remain auditable.