👋 Have a nice time!
© 2024 Robin Medina
Icon content moderation was manual and thorough, resulting in a large queue. Introducing a self-serve photo upload experience risked similar issues. User feedback indicated that long queue times were frustrating for creators. Moderators spent most of their time correcting and creating tags, especially for new creators.
We interviewed and reviewed existing feedback from photographers, photo moderators, icon creators, and icon moderators to understand user needs from both content types and workflows of the UGC experience. The main pain points were:
The primary bottleneck was our first-come, first-served system with no submission limits. Aligning teams to create fairer rules without losing creator trust was challenging. We expected some resistance to change, but most creators would benefit from more reasonable wait times for moderation.
We collaborated with engineers to brainstorm ways to improve moderation times. We discussed using LLMs for quality review, metadata, and visual categorization. User interviews revealed that creators typically submitted under 20 icons at a time and were open to a submission cap higher than their usual number.
Implementing a tiered system for moderation queueing emerged as the most impactful decision. This system allows us to expedite moderation for creators with a history of high-quality submissions. I designed informative UI details to help creators understand the new tiered system, ensuring transparency before we launched tiers.
During feature scoping meetings, we evaluated feasible and impactful features within our project scope, including research spikes to test LLM performance and quality. We used existing categorization dictionaries and focused on accurately identifying human and cultural identities.
We prioritized high-quality tagging over additional LLM features like color palettes. This approach addressed our commitment to accurate representation and reduced the time moderators spent on tagging.
The tiered system prevented long queues and provided consistent, predictable turnaround times. LLMs for suggested and related image tags streamlined the submission and moderation process.
The self-serve interface allowed photographers to upload, create collections, bulk edit, and recommend accurate tags following Noun Project guidelines.
The project showcased successful cross-functional collaboration and user-centered design. We made room for discussions around all the changes and solutions to prepare our teams to inform and respond to our creators with complete transparency.
In hindsight, earlier interviews with creators and moderation teams could have identified priorities sooner, allowing for iterative feature integrations like tiers.
Tools I use to guide conversations and help visualize the strategy to set up, refactor, or onboard a system.
Tools I use for discovery and definition during rebranding and refresh projects.