Experimentation allows us to safely and confidently roll out new features and UI changes by running test on percentage of your user base
Both studies must begin with qualitative or quantitative evidence. For eg, if two or more filters are added to the search results, a user will convert twice.
This data point allows one to develop a theory that takes the form of “We think Y will occur if we do X. If we see a spike in Z metric, we will know it will be valid. For this example, we may say “we are convinced that it is not easier for us to find the related items by highlighting the filters. If we see an increase in conversion and filter use, we will know that this is so.
We can now plan and develop a test. Either the A/B test or MVT methods are common, which enables us to test page variations. We have to define what measures and our degree of importance we want to influence.
If the theory has been proved correct, awesome! Why not, if not? To warn and revisit the theory using new learning, then iterate. If we have sufficient evidence to say that an idea is of importance, we may see to apply the data. We continue to measure new information until applied to inform new theories.