Routing, Memory, and Limits

Objectives

Explain why tasks are routed to different models
Separate context from persistent memory

Lesson

Models differ in cost, latency, privacy, compute and accuracy, so a system may route each task to a different one. Local models keep data on your hardware; larger hosted models may reason better.

Context lasts for one conversation; persistent memory is stored and reloaded. Training data has a date, and the world keeps moving after it.

Terms

Model routing, Local model, Context window, Token, MCP

Sources

Check

- Local models are always better because the data never leaves the device
- Tasks are assigned randomly to balance load across providers
- Models differ in cost, latency, privacy, and accuracy; the task decides
- Cloud models are always better because they are larger
- Context is per exchange; memory is stored and reloaded
- Memory is the model’s weights, fixed at training time
- Context is permanent; memory lasts only for one exchange
- There is no difference
- The full set of data the model was trained on
- Permanent memory the model keeps between sessions
- A browser tab
- How much text a model can consider at once