cyle answered:
when we A/B test something, it means that one group of people has the new thing, and one group of people has the old thing. this youtube video is a very quick way of explaining that process, i suggest watching it first here.
in this case, with the new navigation layout, our hypothesis is that the new layout will be more easily understood by more people, meaning we’ll see more engagement with the navigation items themselves. like literally, more clicks to the things we now have labels for, when before they were just an icon. and the fact they’re in a place that’s become more standard across the internet: a left-aligned sidebar.
ideally, that will lead to better retention of new users, and even an “aha” moment for people who have been on tumblr for a long time but never really looked at those different destinations. i’ve been on tumblr since 2009 and even i’ve found myself clicking on the inbox and activity more than ever, with the new change.
for us to see the change as a “mistake”, we’d need to see statistical evidence to contradict that hypothesis, which we’re already tracking automatically for everyone in the test. we’d also need to get an overwhelming amount of feedback to contradict any positive gains we do see. sending in negative feedback daily won’t do that.
so if you want the whole thing to go away: we need to see a negative behavior change in potentially millions of peoples’ behavior. that has happened before! we’ve rolled changes back that caused that negative reaction. that’s the whole point of these experiments.
and if you want to send us feedback about how the experience is negatively affecting you: please, please do!!! despite the statistical evidence, we want to know if there are accessibility concerns we missed, usability concerns, design considerations, etc etc, because that’s why we’re running this as an experiment and not just launching it to everyone. that feedback is really important to us, as long as it’s constructive.
