Your Text Encoder Is Bigger Than Your Model

Nine and a half gigabytes of your memory budget goes to the thing that reads your prompt, and most low VRAM guides never mention it.

Published August 22, 2026 · RealAIGirls · 8 min read

What Quantization Is, In One Paragraph

A model is a pile of numbers. Half precision stores each of those numbers in 16 bits, which is two bytes. Quantization stores them in fewer bits and keeps a small scaling factor per block so the original value can be approximated on the way back out.

That is the whole idea. Fewer bits per weight, smaller file, less memory, some loss. The file size scales almost linearly with the bit width, which is why you can predict most of these numbers before you download anything.

You can see it in the flagship model. The full checkpoint is 23.8 GB at bf16. Divide by two bytes per weight and you get 11.9 billion parameters. Every quantized version below is that same 11.9 billion numbers written more cheaply.

The Diffusion Model Ladder

These are the real published file sizes from the widely used GGUF conversion of the 12 billion parameter model, which the Hugging Face API reports at 135,119 downloads and 1,403 likes over the trailing month.

The K Quants Are Free, So Take Them

Look at Q4_K_S against Q4_0. Twenty megabytes apart. Twenty megabytes out of nearly seven gigabytes, which is a rounding error on any drive made this decade.

The difference between them is not size, it is where the bits went. A legacy quant like Q4_0 gives every tensor in the network the same treatment. A K quant varies the precision per tensor, spending more bits on the layers that are sensitive to rounding and fewer on the ones that are not.

Now The Half Nobody Quantizes

Here is the same ladder for the T5 XXL encoder. Divide its f16 size by two bytes and it comes out at 4.76 billion parameters, which is a serious model in its own right sitting quietly next to the one you are actually thinking about.

Read the Q4_K_S row again. Quantizing the encoder from f16 to Q4_K_S frees 6.79 GB. That is more memory than the entire Q4 diffusion model occupies. The single largest saving available to most people is in the file they never touch.

And the download numbers say most people do not touch it. The GGUF encoder repository reports 56,320 downloads over the trailing month against the diffusion model conversion's 135,119. Those two populations are not identical, because that encoder gets used by other models too, but the ratio is still striking. Fewer than half as many people are quantizing the encoder as are quantizing the model.

Add Up The Whole Stack

A generation needs four files, not one. The diffusion model, the T5 encoder, a small CLIP encoder at 0.25 GB, and the autoencoder at 0.34 GB. Here is what the complete set weighs at three sensible configurations.

Thirty four gigabytes down to ten. That is the actual size of the win, and roughly a third of it came from the encoder.

The fp8 Comparison, Run Honestly

People compare the fp8 checkpoint against a GGUF file and declare a winner, and the comparison is usually rigged by accident.

The commonly used fp8 build is 17.25 GB, and that number is not the diffusion model alone. It is an all in one checkpoint carrying both text encoders and the autoencoder inside it. The honest thing to compare it against is a full Q8 stack, and that comes to 18.36 GB.

So at the 8 bit rung, fp8 is actually the smaller download by about a gigabyte. The GGUF route wins decisively lower down, where 10.14 GB against 17.25 GB is not close, and where fp8 has no equivalent to offer because there is no fp4 checkpoint sitting next to it.

The Encoder Runs Once. Use That.

Here is the part that changes what you should do with all of the above.

The diffusion model runs every step. Twenty steps means twenty passes, and it has to be resident and fast the whole time. The text encoder runs once per prompt, before sampling starts, and then it has nothing to do until you change the words.

That asymmetry is why the standard advice is to force the encoder onto the CPU and leave the GPU entirely to the sampler. The encoder pass gets slower, you feel it once at the top of a run, and it disappears completely on every subsequent generation with the same prompt because the conditioning is cached.

Where The Quality Actually Goes

This section is the soft one and it should be labelled as such. There is no rigorous public benchmark comparing every rung on that ladder for image quality, so what follows is the rough consensus from people generating a lot of pictures, not a measurement, and you should treat it as a starting point rather than a finding.

Q8 is generally reported as indistinguishable from full weight in side by side comparisons. Q6_K and Q5_K_S are where most people settle, with differences that show up in fine texture and small text if you go looking at 100 percent zoom. Q4 is a real trade, and the places it shows first are the places every model is weakest anyway: hands, small text, repeating patterns like lace and chain link.

Below Q4 you are no longer tuning, you are proving something loads. Q2_K exists so that somebody with 6 GB can watch the thing run, and that is a legitimate reason for it to exist and not a reason to generate with it.

The test that settles it for your own eyes takes ten minutes. Fix the seed, fix the prompt, generate once at Q8 and once at Q4 with everything else identical, and put the two side by side at full zoom. If you cannot see it on your subject matter, the lower quant is free for you, whatever anybody else reports.

The Honest Counterpoint

Four things cut against everything above.

File size is not VRAM usage. The weights are only part of what sits on your card during a run. Activations, the latent, attention buffers and whatever your interface is holding all take space on top, and that overhead scales with your output resolution rather than with your quant. A 6.81 GB model does not fit comfortably in 8 GB just because the arithmetic says it should.

Quantization is not always faster. The weights get unpacked back to a usable precision as they are used, and on some hardware that dequantization costs enough per step to cancel out the benefit of the smaller footprint. If you were not swapping to disk before, a lower quant can genuinely make your generations slower.

If you have 24 GB, most of this is not for you. Running Q4 on a card that could hold Q8 comfortably is throwing away quality to solve a problem you do not have. The right quant is the largest one that fits with headroom, not the smallest one that loads.

And LoRAs are the rough edge. Applying a LoRA to a quantized model means the adapter and the quantized weights have to be reconciled at load time, support for that varies by tool and by quant format, and a stack that works perfectly at fp8 can behave differently at Q4. If your workflow leans on five LoRAs, test the quant with them attached rather than on its own.

None of that changes the headline. The prompt reader is 9.53 GB, it runs once, and it is the cheapest thing in your workflow to shrink.