Large Language Models (LLMs) power modern AI applications, but their unpredictable behavior and complex workflows make it difficult to diagnose issues, optimize performance, and understand how they process data. Without visibility into each step of an LLM chain, troubleshooting and improving efficiency can be challenging.
Datadog’s LLM Observability provides end-to-end tracing of LLM-powered applications, capturing input and output text, latency metrics, token usage, and errors. By tracing each step in the LLM chain—including embedding, retrieval, and generation—teams can identify the root causes of unexpected outputs, latency, and errors, helping them troubleshoot performance issues and control costs.
In this hands-on workshop, you’ll build a chatbot application with a Retrieval-Augmented Generation (RAG) workflow using the OpenAI Python SDK to make calls to local models. You’ll then instrument the application for Datadog’s LLM Observability, using auto-instrumentation and manual, in-code setup to collect traces. Finally, you’ll analyze these traces to connect application behavior with steps in the LLM chain, identifying areas for improvement, applying changes, and observing results.
By the end of this workshop, you’ll have practical experience using Datadog’s observability tools to understand LLM application behavior and improve performance.