Validating a Better Comment Experience Through AB & Usability TestingComparing two comment flows through AB and usability testing to find what actually drives engagement.
AB TestingUsability TestingMobile UX

Validating a Better Comment Experience Through AB & Usability Testing

Comparing two comment flows through AB and usability testing to find what actually drives engagement.

2018
1 week
1 Product Designer, 1 Frontend, 1 Backend
Product Designer, Researcher, and Product Manager

Overview

kumparan's mobile app had a comment feature I believed could drive stronger engagement, but the existing flow required users to move through more steps than necessary to actually leave a comment. Rather than redesign on instinct, I ran a structured AB and usability test comparing the current flow against a simplified alternative, to get real evidence before recommending a direction.

The Problem

The existing comment flow buried the comment box behind extra steps, and there was no data confirming whether a simplified, more direct flow would actually perform better with real users. I needed evidence, not assumption, before recommending kumparan invest engineering time in a redesign.

Current Feature Flow Diagram

Current Feature Screenshot

Solution

I designed a proper test methodology before touching anything: defining what to test, building a working prototype for the improved flow, writing a clear task scenario, specifying the target participant profile, and setting up recording and success-metric tracking in advance.

  • Participants: 3–8 digital users, men and women, aged 18–40
  • Scenario given to participants: "You are a person who wants to comment on a story in the kumparan app."
  • Prototypes tested: both the existing comment flow and a new "direct-to-comment" flow (built to route users to the comment box with minimal intermediate steps), each built as an interactive InVision prototype
  • Success metrics tracked across both flows: opening the article → landing on story detail → locating the comment form → completing a comment

Sessions were recorded (smartphone + camera) and run in a controlled room to keep conditions consistent across participants.

Improved Feature Flow Diagram

Result & Conclusion:

Across the test, participants had no difficulty completing the task on the new, simplified flow, and consistently preferred it over the existing design. Based on this evidence, the improved "direct-to-comment" flow was greenlit as the direction to move forward with, validated by data rather than opinion, with usage continuing to be monitored by the feature's development team post-launch.

Results & Impact

Ran a structured AB + usability test comparing 2 comment-flow designs before committing to a redesign

Tested with a targeted sample of digital users (18–40) using a clear task scenario and success-metric funnel

Found the simplified "direct-to-comment" flow was preferred with no completion difficulty

Recommendation adopted and shipped, with continued post-launch monitoring

Like what you see?

I'm always excited to collaborate with teams and individuals who value thoughtful, user-centered design. If you're hiring, or have a project in mind, let's talk.

Get in Touch