Whatify

FAQ

general questions //

None at all! Our process is automatic and doesn’t require any work on your end while the test is running.

We test photos, they move around, you don’t touch them, but you can make any other changes to your shop. We process the results and make recommendations. Voila. If you’d like to learn more about the statistical side of our process, check out our blog post on A/B testing.

Four weeks. This is how long it takes to gather data on your listings and then process that data into meaningful results. We will always send you an email notifying you when your results are in!

In order to gather enough results to make reliable recommendations, we have to test your photos for a while. Once we have gathered data on your listings, we process it to account for disturbances like day, time, and pure luck. Then we translate those results into actionable recommendations that will increase your sales!

We run our tests by rotating your primary product photo—in order to do this, we need access to your listing titles and photos. We do not have access to your private information, and the service is completely safe and secure. If you have any concerns, please contact us.

Even if you think you’re using the most successful photos, statistically you probably aren’t! We’ve tested thousands of photos and learned that it’s really really hard to predict which photos will perform the best. Even if you use a rule or a system, it won’t be correct all the time. If your shop has 20+ listings, there is just a 4% chance — seriously FOUR percent — that every primary photo you’ve selected is the best one. And that 4% consists of shops where alternative photos did not even depict the same item as the primary photos! If you use Whatify, you will benefit.

In short, the results from your own test are not likely to be very reliable. We have developed a sophisticated process for producing effective results; and without a systematic method, your experiment can be skewed by a variety of disturbances.

One such issue is noise: if you test two photos and photo A gets 26 views while photo B gets 31, does that really mean photo B is better, or were some of those views due to chance? Another issue is that of confounding variables: let’s say you test photo A on Mondays and photo B on Thursdays, and you find that photo A is better. Is that really because the photo is better or does your shop just get more traffic on Mondays than on Thursdays?

Our test accounts for all of these disturbances! (If you were to try to set up something like this yourself, it would be incredibly labor intensive! Imagine switching your images every several hours for a month.) If you’d like to learn more about the statistical side of our process, check out our blog post on A/B testing.

If you are seeing the images on your listings change, don’t worry! That’s the test doing what it’s supposed to do! In order to gather data on how your images perform, we actually alternate which photo is in the primary position while the test is running. If you’re uneasy about this, there are a few (totally optional) steps you can take to make sure that your shop stays looking exactly how you’d like while the test is running:

1. Check that the photos you would like to test are set as the first two photos for your listings (primary photo and the photo immediately following it).

2. Consider whether the photos you are testing are ones that you are satisfied with as a primary photo while we run the test. Since we will be switching the photos, each one will be the primary photo at various times. (For example, you may choose not to test a detail shot of your product at an obscure angle.)

3. Set a designated thumbnail for each of the photos you’re testing (you may be fine with the default thumbnail cropping). You can do this by setting the first photo as the primary photo and adjusting the thumbnail, then setting the second photo as the primary photo and adjusting its thumbnail as well. Etsy will remember how you cropped it!

Please note that if you stop your test to make these changes, the process will have to restart, and it will be another four weeks before you receive results.

technical stuff //

We perform our A/B test by randomly varying which photo is in the primary position on your listing and recording how the listing performs. We then take the raw data from the test (the number of clicks the listing receives while each photo is the primary photo) and process it to account for statistical disturbances such as time of day and pure luck. Through this process, we arrive at a realistic and reliable number for how much better one photo will perform than the other and make a recommendation accordingly. If you’d like to learn more about the statistical side of our process, check out our blog post on A/B testing.

Yes! Feel free to make changes to your listings—even the ones that are being tested! We just ask that you do not remove, edit, rearrange, or otherwise change the photos on the listings that are being tested.

Upon completion of your test, you will have the option to either apply our recommendations or revert your photos to their original positions.

Actual Views: The number displayed under “actual views” is the true number of clicks the winning photo received while it was active as the primary photo compared to the alternative photo. This is a raw piece of data that has not been processed to account for statistical disturbances such as noise and luck.

Projected Views: The number displayed for “projected views” is our statistical calculation of how much better the winning photo is likely to perform in the future compared to the alternative photo. We take the “actual views” and adjust to account for variables such as time of day and pure chance. We also consider a statistical phenomenon called “regression towards the mean,” which means that if a photo performs incredibly well during one test, it probably won’t perform quite as well on the next test. This adjustment produces the “true” difference in performance between the winning photo and the alternative. Because of this somewhat-conservative adjustment process, we can be confident that our results are not an overestimation!

Performance: The number displayed for “performance” tells you how much better you can expect your listing to perform by using the recommended photo as your primary photo compared to the alternative. It is calculated using the number of “projected views” for the winning photo compared to the total number of views the listing received overall.

We take the projected increase in views for a listing and combine it with information on the number of transactions per view as well as the revenue generated per listing per transaction to determine the increase in value that our recommendations will bring in. We do this for every listing where we recommended that you change your primary photo, and add them together to determine the total amount of revenue that our results will bring to your shop.

We go to great lengths to avoid overestimating on our projections, and are actually quite conservative in our estimations. Unlike other photo analysis tools, we have a very sophisticated statistical method in place for processing raw data into reliable results: we take the actual number of clicks your photo receives and use it to calculate our projections, accounting for variables like time of day and random chance. We also consider a statistical phenomenon called “regression towards the mean,” which means that if a photo performs incredibly well during one test, it probably won’t perform quite as well on the next test. And finally, we double-check our results using a process called cross-validation, where we compare our recommendations to untouched test data to be sure that the winning photo really performs the best. Because of this thorough testing and adjustment process, we are confident that we’re not overestimating our projections on how your images will perform! If you’d like to learn more about the statistical side of our process, check out our blog post on A/B testing.

Yes! If you’re interested in the raw data from your test, just reach out to us; we’re happy to provide you with this information.

more questions? //

We're happy to help!

Submit any questions in the contact form below.