Abstract Large Language Models have revolutionized AI applications, but transitioning from research prototypes to production-ready systems presents significant challenges in reliability, accuracy, and deployment readiness. This talk presents battle-tested strategies for building trustworthy LLM applications, with a special focus on RAG pipelines, chatbots, and AI agents. Drawing from real-world implementations across different industries, we'll explore practical techniques for minimizing hallucinations, optimizing performance, and ensuring ethical deployment.
Overview We'll begin by examining the reliability challenges facing LLM applications in production environments, from hallucinations to consistency issues. The talk will then dive into architectural patterns for building trusted RAG pipelines and implementing effective hallucination prevention systems, illustrated through a live example of building reliable agents. We'll explore the emerging field of reliable AI agents, covering essential safety mechanisms and validation frameworks.
Target Audience This talk is designed for software engineers, ML practitioners, and technical leads who are working to deploy LLM applications in production environments. Attendees should have basic familiarity with LLMs, but deep expertise is not required.
Takeaways Attendees will leave with practical implementation patterns, validation frameworks for chatbots and agents, and strategies for building reliable LLM applications that users can trust.
Rush Shahani is the CTO & Co-Founder of Persana AI (YC W23), where he focuses on building reliable and production-ready LLM applications and agents for GTM teams. His background includes developing AI and backend solutions at LinkedIn, Element AI and Shopify. He is the author of LLM Reliability and brings deep expertise in scaling AI systems from research to production. His work focuses on bridging the gap between AI research and practical, production-ready applications that deliver real business value.
