The character sheet describes her face in words. These tools show the model a photo of it instead.
The character sheet describes her face in words. These tools show the model a photo of it instead.
IPAdapter FaceID, InstantID, and PuLID solve a problem your character sheet can't
Twelve renders in, same fifteen-field character sheet pasted into every prompt, and the eyebrow shape has drifted three times. The hair color held. The jacket held. The face did not, because a face isn't fifteen fields, it's thousands of tiny relationships between fifteen fields that no written description captures completely. That gap is where prompting, no matter how disciplined, runs out of road.
A structured JSON character sheet gets you further than a comma-separated prompt ever will, and it is worth building one. But it describes a face. It does not show the model a face. Those are different problems with different tools, and the tools that solve the second one work by an entirely different mechanism: instead of adding more words for the text encoder to parse, they take an actual photo and convert it into a numerical face embedding that rides alongside the prompt, steering the generation toward a specific identity rather than a described one.
Three of these tools cover almost everyone's needs right now: IPAdapter FaceID, InstantID, and PuLID. They are not interchangeable, and the setup friction on each one is real enough that picking wrong costs an evening.
If a ComfyUI install already has the IPAdapter Plus custom node pack from cubiq, FaceID is the cheapest of the three to try because most of the plumbing is already there. The repository's own documentation is blunt about the one hard requirement: FaceID models need insightface installed in the ComfyUI environment, and that install is a separate step from the regular node pack, not something that comes bundled in.
There isn't one FaceID model, there are several, and the README is specific about which does what. Plain FaceID is the baseline. FaceID Plus v2 is the stronger, more widely recommended variant for general identity locking. FaceID Portrait is built for text-driven style transfer around a fixed face, and FaceID Portrait unnorm is called out as SDXL only, described as a very strong style transfer option, meaning it will bend the output aesthetic harder than the others. Pick the wrong variant and you'll either get a weak identity lock or an image that overrides the style you were trying to keep.
On weight, the general IPAdapter guidance carries over directly: the documentation suggests lowering the weight to at least 0.8 and increasing the sampling steps to compensate, rather than running a lighter weight at your normal step count. Push it, don't dilute it.
InstantID's own documentation describes it as a state-of-the-art tuning-free method for ID-preserving generation from a single reference image, and in practice it holds a face tighter than base FaceID does, at the cost of a heavier pipeline. It combines a ControlNetModel that reads facial landmarks with IP-Adapter weights on top, so you're running two conditioning systems at once instead of one.
The setup has a specific trap worth knowing about before you hit it: InstantID's face encoder needs the antelopev2 model, and the project's own README says the default download link for it is invalid, so it has to be fetched manually and placed in a models/antelopev2 folder yourself. Skip that step and the pipeline won't silently work worse, it won't run at all. The example configuration in the documentation pairs the pipeline with wangqixun/YamerMIX_v8 as the base checkpoint, and the whole thing is built around SDXL, not SD 1.5.
That extra weight buys real accuracy. Independent testing referenced across the ComfyUI community consistently places InstantID's facial similarity in the 82 to 86 percent range against the source photo, with eye shape, nose structure, and face shape transferring reliably rather than approximately. If a project genuinely needs the tightest possible identity match and can absorb a slower pipeline, this is the one built for that.
PuLID takes a different design goal entirely, and it's right there in the name: Pure and Lightning ID customization. The method comes out of a NeurIPS 2024 paper by Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, Peng Zhang, and Qian He, and the paper's stated aim is a tuning-free ID customization method that minimizes disruption to the base model while still achieving high identity fidelity, using what the authors call a contrastive alignment loss paired with an accurate ID loss.
What that means in practice is the whole point of choosing PuLID over the other two: the background, lighting, composition, and overall style of a generation are kept as close as possible to what they would have been without any face conditioning at all. IPAdapter FaceID and InstantID both work well but can visibly pull an image's aesthetic toward the reference photo's look. PuLID's entire architecture is built to resist exactly that pull. Code and trained models are published in the ToTheBeginning/PuLID repository for anyone running a local pipeline rather than a paper-only reproduction.
Start with IPAdapter FaceID if you're already in ComfyUI and want to test the concept without new downloads beyond insightface. Move to PuLID specifically when a strong stylized look keeps getting washed out by whichever face tool you're using, since resisting that drift is its actual design goal, not a side benefit. Reach for InstantID only when identity fidelity is the priority above everything else and the project can tolerate the antelopev2 download step and the SDXL-only ControlNet overhead that comes with it.
Here's the honest counterargument, because a technique guide that pretends its subject has no downside is selling something. All three of these add a face analysis model, extra VRAM, and extra load time to a pipeline that ran fine without them. For a character who appears in one image and never again, none of this is worth installing. A good reference photo fed through a plain img2img pass at moderate denoise, alongside the structured prompt from the character sheet approach, gets most of the way there for a fraction of the setup cost.
And none of these three tools replace the character sheet, they sit on top of it. IPAdapter FaceID, InstantID, and PuLID all lock the face. They have no opinion about the wardrobe, the pose, or the lighting in the rest of the frame, which is exactly what the structured prompt is still responsible for describing. Pair a face embedding with a ControlNet OpenPose pass for the body and a labelled prompt block for everything else, and the three problems, face, pose, and scene, finally stop fighting each other for the same solution.
Back to those twelve renders. Swap nothing about the character sheet, keep every field exactly as written, and just add IPAdapter FaceID at a weight of 0.85 with the matched LoRA loaded alongside it. The eyebrow stops drifting on render two. It stops drifting on render twelve. The words were never the problem, they just couldn't do a photo's job.