Agenti Insights

Agent Insights is an internal experiment by Agent Studio to help AI agent creators understand real user interactions at scale, then turn those insights into product and service decisions with greater confidence, prioritization clarity, and faster iteration cycles.

Problem and Context

As more teams deploy AI agents, they quickly run into the same issue: they can see raw conversations, but they struggle to extract reliable patterns, blockers, and opportunities fast enough to improve the product.

Agent Studio launched Agent Insights as an internal experiment to validate whether conversation intelligence could become a self-serve product. To pressure-test the idea, we ran productized-service pilots with StubHub International and Voiceflow.

In these pilots, Agent Studio delivered tailored, handcrafted PDF insight reports produced through the n8n-powered analysis workflow.

The core challenge became clear: while insight quality was strong, setup and operational tuning still required too much manual work for a plug-and-play SaaS experience.

Goals and Objectives

• Validate product value: confirm that structured conversation analytics helps teams make better roadmap and UX decisions.
• Reduce manual analysis: replace ad hoc transcript reviews with a repeatable insights workflow.
• Test delivery model fit: assess if the solution could scale as self-serve SaaS versus managed services.
• Build decision-ready outputs: surface summaries, themes, blockers, sentiment, and opportunity signals for teams.

Demo

Here’s a demo showing how conversation data can be transformed into actionable reports via the intended self-serve SAAS application.

Approach and Solution

Agent Studio built Agent Insights as a hybrid architecture:

• Custom front-end experience to make conversation review and insight consumption simple for product and operations teams.
• n8n-powered back-end orchestration to run LLM analysis workflows and generate handcrafted PDF outputs over conversation datasets.
• Structured insight outputs including executive summaries, conversation volume/engagement, core themes, friction points, suggestions, CSAT signals, and sentiment distributions.
• Pilot-led validation loop with StubHub International and Voiceflow using tailored PDF deliverables to test insight usefulness, decision impact, and operational effort.

This approach allowed fast experimentation while preserving flexibility to iterate on prompts, categorization logic, and reporting formats.

Results and Outcomes

The experiment validated the product idea and clarified the right business model:

• Strong signal on customer value: teams could identify recurrent user pain points and opportunity themes faster than manual transcript review, even through report-based delivery.
• Better decision support: structured summaries made it easier to prioritize roadmap conversations with product and leadership stakeholders.
• Self-serve friction identified early: onboarding and ongoing configuration required significant manual setup, limiting SaaS readiness at this stage.
• Clear strategic pivot: Agent Insights was intentionally deprioritized as a standalone self-serve SaaS product.
• Service-led monetization path: the capability now lives inside Agent Studio’s agency services, where hands-on setup and customization create stronger client outcomes.
• Foundation for Mosaic at FCTG: key learnings from Agent Insights were critical in shaping the Mosaic project for Flight Centre Travel Group (FCTG), turning experiment patterns into a production-facing client solution (related case study).

Reflection and Next Steps

• What worked well:
  • The combination of custom UI + n8n + LLM analysis delivered useful insights quickly.
  • Pilot feedback from StubHub International and Voiceflow provided real-world validation, not just internal assumptions.
  • The experiment reduced uncertainty around where Agent Studio can create the most value today.

👉 Interested in integrating AI conversation intelligence into your own agent operations? Let’s talk.