Skip to main content

Cerebras launches disaggregated inference >>

Nov 18 2026

What is disaggregated inference?

Disaggregated inference is an approach to serving large language models that runs prompt processing, called prefill, and token generation, called decode, in separate resource pools. Each pool can be configured for its phase, with request state passed between them. Cerebras explains this separation in its disaggregated-inference overview.

Cerebras is helping lead the move toward heterogeneous disaggregated inference through announced collaborations with AMD and AWS. These designs pair partner prefill platforms with Cerebras wafer-scale decode. CS-4 also has native support for this architecture.

Fast answer

Disaggregated inference separates prompt processing (prefill) and token generation (decode) into different resource pools. Homogeneous disaggregation uses the same accelerator type in both pools; heterogeneous disaggregation uses different accelerator types. This separation enables phase-specific tuning and can reduce interference. Cerebras is helping lead the heterogeneous approach through its announced AMD and AWS collaborations. Performance depends on workload, hardware and configuration.How does disaggregated inference work?

Prefill processes the prompt and builds the key-value cache, or KV cache, which stores state used during generation. That state passes to a decode engine, which generates output tokens. Cerebras’ AWS architecture explanation describes this handoff: Trainium computes the cache, then sends it through Elastic Fabric Adapter (EFA) to a Cerebras Wafer-Scale Engine for decode.

Prompt → Prefill pool → KV-cache handoff → Decode pool → Response tokens

The handoff transfers request-specific state rather than copying all model weights for every request. The two pools must use compatible model versions and state formats. They can reside in the same data center; disaggregation does not require sending a request between distant regions.

Watch the Cerebras explainer

Watch Cerebras Explains | What is Disaggregated Inference? for a short introduction to prefill, decode and the separation between them.

Cerebras Explains | What is Disaggregated Inference? — Cerebras, June 11, 2026. Watch on YouTube.

How is it different from aggregated inference?

The distinction is which resources execute each phase. Using more than one accelerator does not, by itself, make a deployment disaggregated.

Cerebras focuses on the heterogeneous approach in its March 26, 2026 overview: complementary systems handle prefill while wafer-scale systems handle decode.

Why separate prefill and decode?

The phases place different demands on a serving system. Prefill often benefits from processing many prompt tokens together. Interactive decode repeatedly produces the next token for each active request and is often sensitive to memory bandwidth. The balance changes with the model, batch size, context and serving configuration.

Separating prefill and decode lets each phase use hardware suited to its work. Cerebras’ CS-4 architecture overview describes a complementary GPU or ASIC prefill engine preparing model state and transferring it to CS-4 for decode. This allows complementary platforms to handle the different demands of prompt processing and token generation.

It also introduces a network handoff and another scheduling boundary. A useful deployment must balance both pools and transfer state efficiently. Separation alone does not establish higher throughput or lower end-to-end latency.

Where does Cerebras fit?

Cerebras brings wafer-scale token generation to architectures that combine complementary prefill and decode platforms. The following announcements describe distinct configurations; their hardware and performance claims should be read in their original context.

With Cerebras’ WSE having the highest memory bandwidth of any chip in the market, it represents a premier decode accelerator. This puts Cerebras at the center of complementary inference architectures, combining different accelerator designs within one service.

What is the current rollout status?

On November 12, 2026, Cerebras reported that it had delivered production-scale heterogeneous disaggregated inference. Availability, supported models and deployment options depend on the specific offering. Contact Cerebras to confirm current access and supported configurations.

When should a team evaluate disaggregated inference?

Every team building or scaling production LLM applications should evaluate disaggregated inference early in its infrastructure planning, with a clear path toward adoption. By matching prefill and decode to resources suited to each phase, disaggregated inference can reduce interference and improve responsiveness and throughput. These benefits make it a compelling architecture to pursue as AI applications grow.

Cerebras is helping lead this shift through wafer-scale decode and native support for disaggregated inference in CS-4, which pairs Cerebras token generation with complementary prefill platforms. Teams should move toward implementation by validating a supported configuration against their model-quality, latency and capacity requirements, then scaling deployment as the benefits are demonstratedFrequently asked questions

Is disaggregated inference the same as distributed inference?

No. Distributed inference is a broader term for spreading inference work across devices or systems. Disaggregated inference specifically separates prefill and decode resources. A distributed deployment can still run both phases in the same pool.

Does disaggregated inference require different types of chips?

No. Separate pools can use the same accelerator type. A heterogeneous design uses different types.

Is prefill the same as a model’s reasoning?

No. Prefill processes the input context. In autoregressive reasoning models, generated reasoning tokens are part of token generation. Faster prompt processing alone does not establish faster reasoning or task completion.

Does disaggregated inference always improve latency?

No. Independent tuning can help, but state transfer, queueing and capacity imbalance can offset the benefit. Compare time to first token, inter-token latency and total completion time on the actual workload.

Does Cerebras support disaggregated inference?

Cerebras has announced AMD and AWS collaborations for disaggregated inference, and CS-4 natively supports complementary prefill platforms. Access and model support depend on the specific offering and rollout.

Related glossary terms

Related links

Performance comparisons are based on third-party benchmarking or internal testing. Observed inference speed improvements versus GPU-based systems may vary depending on workload, configuration, date and models being tested.

info@cerebras.ai

1237 E. Arques Ave
 Sunnyvale, CA 94085

© 2026 Cerebras.
All rights reserved.