Lesson 4.2
Managing cost and latency
Cost is prompt design, model choice, and how often you call. Measure per-request tokens before you try to optimise anything.
13mIntermediate12,750 students
Sign in to watch this lesson
Browsing is free. Create an account to play videos and track your progress.
Overview
Cost is prompt design, model choice, and how often you call. Measure per-request tokens before you try to optimise anything.
What this lesson covers
- Attribute spend to specific calls and prompts
- Trim prompts and route by difficulty
- Set timeouts and fall back gracefully
This lesson sits in Shipping to Production, part of Building AI Apps with LLMs. It assumes what came before it and leads directly into the next lesson in the module.
In this lesson you will:
- Attribute spend to specific calls and prompts
- Trim prompts and route by difficulty
- Set timeouts and fall back gracefully