{"id":23374,"date":"2025-05-26T23:11:08","date_gmt":"2025-05-27T06:11:08","guid":{"rendered":"https:\/\/arize.com\/?post_type=midsize_content&#038;p=23374"},"modified":"2025-06-09T16:42:53","modified_gmt":"2025-06-09T23:42:53","slug":"ai-agent-useful-case-study","status":"publish","type":"content","link":"https:\/\/arize.com\/ai-agent-useful-case-study\/","title":{"rendered":"AI Agent: Useful Case Study"},"content":{"rendered":"<p>According to a recent survey by PwC, 78% of enterprises are readying or have already shipped AI agents or multiagent systems. Here are several examples of real-world deployments of AI agents and how they are useful to their parent organizations.<\/p>\n<h2>Geotab Builds a Generative AI Agent to Simplify Fleet Data Analysis<\/h2>\n<h3>Why It&#8217;s Useful<\/h3>\n<p>Geotab, a fleet telematics company, recently built a generative AI agent to allow fleet managers and other users to query vast and complex vehicle data systems using natural language, reducing the technical barrier to gaining actionable insights.<\/p>\n<p><iframe loading=\"lazy\" title=\"YouTube video player\" src=\"https:\/\/www.youtube.com\/embed\/0WYO7Sx2pfw?si=TW2jgbirTCMXnNXj\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h3>What They Built<\/h3>\n<p>The agentic retrieval-augmented generation (RAG) system translates natural language questions into SQL queries to extract relevant data insights. The tool simplifies data access across millions of daily vehicle trips and extensive schemas, helping users like fleet managers make timely, informed decisions without needing deep SQL or data schema knowledge.<\/p>\n<h3>Lessons Learned<\/h3>\n<p>&#8220;High quality examples are critical. We thought we\u2019d need thousands of examples, but we\u2019ve only got a few hundred\u2014and that\u2019s enough,\u201d notes Kyle Weston, Lead Data Scientist at Geotab. Simplifying query complexity, improving schema documentation, leveraging custom SQL evaluation and being thoughtful about <a href=\"https:\/\/arize.com\/blog\/prompt-optimization-few-shot-prompting\/\">prompting optimization techniques<\/a> also played a role in the company\u2019s success.<\/p>\n<h2>Priceline Launches Real-Time Voice AI for Seamless Travel Booking<\/h2>\n<h3>Why It&#8217;s Useful<\/h3>\n<p>Priceline built a real-time voice-enabled generative AI agent, Penny, to create a faster, more intuitive travel booking experience. Designed to make interacting with Priceline easier, especially in hands-free scenarios, Penny removes friction from the customer journey and aligns with the company&#8217;s mission of making travel accessible to everyone.<\/p>\n<p><iframe loading=\"lazy\" title=\"YouTube video player\" src=\"https:\/\/www.youtube.com\/embed\/bT5DmBVw2Gk?si=nObPcPmGCWHA0MFq\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<p>Real-time voice significantly improved user experience with faster, more natural interactions, which in turn enhanced conversions in Priceline\u2019s e-commerce funnel. Penny now supports a growing set of features like hotel booking and trip management, all while operating with contextual awareness and continuity.<\/p>\n<h3>What They Built<\/h3>\n<p>Penny is an AI-powered travel assistant integrated across Priceline&#8217;s platforms, now enhanced with real-time voice capabilities using OpenAI\u2019s Whisper and text-to-speech APIs. The team transitioned from a push-to-talk model to fully streaming audio conversations that enable natural, interruptible interactions. This architecture uses WebSockets to facilitate seamless bi-directional communication and includes features like business function calling, context awareness, and moderation of real-time content.<\/p>\n<h3>Lessons Learned<\/h3>\n<p>Launching Penny\u2019s voice mode surfaced key insights: event-driven architecture over WebSockets required a steep learning curve, audio processing had to adapt across devices, and content moderation worked best on the output side. Instrumentation and trace observability\u2014especially with Arize\u2014proved critical for debugging and iteration.<\/p>\n<h2>Booking.com Builds Modular AI Trip Planner To Power Personalized Travel Experiences<\/h2>\n<h3>Why It&#8217;s Useful<\/h3>\n<p>Booking.com developed the AI Trip Planner, a modular generative AI system that helps users seamlessly explore destinations, receive personalized itineraries, and book accommodations\u2014all through a conversational interface. The system integrates internal recommendation models with LLMs, enabling Booking to serve highly relevant, explainable suggestions across the entire customer journey. Results showed marked improvements in recommendation accuracy, booking conversion rates, and response latency\u2014thanks to a hybrid architecture combining in-house models and structured API orchestration.<\/p>\n<p><iframe loading=\"lazy\" title=\"YouTube video player\" src=\"https:\/\/www.youtube.com\/embed\/QWE-RJ0Zvwg?si=a7JZP8ZLdFegpZ6O\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n<h3>What They Built<\/h3>\n<p>The AI Trip Planner is a layered agent system composed of three main services: an NLP layer for language tasks like moderation and intent detection, a recommendation platform for dynamic, personalized results, and a Gen Orchestrator to coordinate internal services and LLM-driven interactions. The architecture supports modular plug-ins, allowing Booking to swap in proprietary models for cost savings and performance gains. For example, replacing GPT with a fine-tuned in-house model for intent detection led to a 133% accuracy boost and a 5x reduction in latency.<\/p>\n<h3>Lessons Learned<\/h3>\n<p>A key lesson was the importance of building and owning every component\u2014from orchestration to LLM evaluation\u2014rather than relying on off-the-shelf solutions. As one team member put it: &#8220;Many open-source orchestrators break quickly in production. We had to build our own tools to truly meet real-world needs.&#8221; Additionally, defining simple, task-specific evaluation metrics (like factual accuracy, context relevance, and answer relevance) proved more effective than trying to collapse everything into one. By using LLMs as evaluators and layering both offline and online validation across system levels, the team achieved scalable, trustworthy quality control.<\/p>\n","protected":false},"excerpt":{"rendered":"According to a recent survey by PwC, 78% of enterprises are readying or have already shipped AI agents or multiagent systems. Here are several examples of real-world deployments of AI&#8230;","protected":false},"author":2,"featured_media":0,"menu_order":0,"template":"","meta":{"_acf_changed":false,"content-type":"","inline_featured_image":false,"_links_to":"","_links_to_target":""},"coauthors":[606],"class_list":["post-23374","content","type-content","status-publish","hentry"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.7 (Yoast SEO v27.7) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>AI Agent Useful Case Study \u2013 Three Real World Examples<\/title>\n<meta name=\"description\" content=\"Examples from of AI agents being useful in the wild in everything from fleet data analysis (Geotab) to travel bookings (Priceline).\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/arize.com\/ai-agent-useful-case-study\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"AI Agent: Useful Case Study\" \/>\n<meta property=\"og:description\" content=\"Examples from of AI agents being useful in the wild in everything from fleet data analysis (Geotab) to travel bookings (Priceline).\" \/>\n<meta property=\"og:url\" content=\"https:\/\/arize.com\/ai-agent-useful-case-study\/\" \/>\n<meta property=\"og:site_name\" content=\"Arize AI\" \/>\n<meta property=\"article:modified_time\" content=\"2025-06-09T23:42:53+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/arize.com\/wp-content\/uploads\/2025\/04\/Arize-AI-featured-image.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"625\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@arizeai\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"4 minutes\" \/>\n\t<meta name=\"twitter:label2\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data2\" content=\"Aparna Dhinakaran\" \/>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"AI Agent Useful Case Study \u2013 Three Real World Examples","description":"Examples from of AI agents being useful in the wild in everything from fleet data analysis (Geotab) to travel bookings (Priceline).","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/arize.com\/ai-agent-useful-case-study\/","og_locale":"en_US","og_type":"article","og_title":"AI Agent: Useful Case Study","og_description":"Examples from of AI agents being useful in the wild in everything from fleet data analysis (Geotab) to travel bookings (Priceline).","og_url":"https:\/\/arize.com\/ai-agent-useful-case-study\/","og_site_name":"Arize AI","article_modified_time":"2025-06-09T23:42:53+00:00","og_image":[{"width":1200,"height":625,"url":"https:\/\/arize.com\/wp-content\/uploads\/2025\/04\/Arize-AI-featured-image.jpg","type":"image\/jpeg"}],"twitter_card":"summary_large_image","twitter_site":"@arizeai","twitter_misc":{"Est. reading time":"4 minutes","Written by":"Aparna Dhinakaran"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/arize.com\/ai-agent-useful-case-study\/","url":"https:\/\/arize.com\/ai-agent-useful-case-study\/","name":"AI Agent Useful Case Study \u2013 Three Real World Examples","isPartOf":{"@id":"https:\/\/arize.com\/#website"},"datePublished":"2025-05-27T06:11:08+00:00","dateModified":"2025-06-09T23:42:53+00:00","description":"Examples from of AI agents being useful in the wild in everything from fleet data analysis (Geotab) to travel bookings (Priceline).","breadcrumb":{"@id":"https:\/\/arize.com\/ai-agent-useful-case-study\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/arize.com\/ai-agent-useful-case-study\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/arize.com\/ai-agent-useful-case-study\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/arize.com\/"},{"@type":"ListItem","position":2,"name":"AI Agent: Useful Case Study"}]},{"@type":"WebSite","@id":"https:\/\/arize.com\/#website","url":"https:\/\/arize.com\/","name":"Arize AI","description":"AI Observability &amp; LLM Evaluation Platform","publisher":{"@id":"https:\/\/arize.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/arize.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/arize.com\/#organization","name":"Arize AI","url":"https:\/\/arize.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/arize.com\/#\/schema\/logo\/image\/","url":"https:\/\/arize.com\/wp-content\/uploads\/2021\/07\/Arize-AI.png","contentUrl":"https:\/\/arize.com\/wp-content\/uploads\/2021\/07\/Arize-AI.png","width":3333,"height":924,"caption":"Arize AI"},"image":{"@id":"https:\/\/arize.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/arizeai","https:\/\/www.linkedin.com\/company\/arizeai","https:\/\/www.youtube.com\/channel\/UCrVHzD-psX5IMCGoEWHXmGw"]}]}},"_links":{"self":[{"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/content\/23374","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/content"}],"about":[{"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/types\/content"}],"author":[{"embeddable":true,"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/users\/2"}],"wp:attachment":[{"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/media?parent=23374"}],"wp:term":[{"taxonomy":"author","embeddable":true,"href":"https:\/\/arize.com\/wp-json\/wp\/v2\/coauthors?post=23374"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}