Global and Regional Reference Images (IP-Adapters)

Global and Regional Reference Image layers use Image Prompt Adapter (IP-Adapter) models to inspire a new image with the content of an input image. You can use add both Global and Regional Reference Images as layers on the ‘Layers’ tab.

How do Reference Images and IP-Adapters work?

While the Image to Image process uses the color and structure to guide the denoising process with your text prompt, the Image Prompt Adapter uses the concepts that the AI can detect in your imagery to inspire new content. This can be a great way to control the diffusion process when you don't know how to articulate or put a visual concept into words - You can let the system do that for you.

IP Adapter Method

The IP Adapter Method is automatically set to ‘Full,’ but can be changed to either ‘Style Only’ or ‘Composition Only’.

• Full — The AI Model will attempt to reference the contents of the image, including style, color, and composition.
• Style Only - The AI Model will only attempt to reference the style and color of the image, ignoring the content and composition in favor of your prompt.
• Composition Only - The AI Model will only attempt to reference the composition and structural components of an image, ignoring the style and color in favor of your prompt.
• 
  

Weight

A Weight of 1 will strongly enforce the concepts of your Image Prompt - It's typically advised to reduce the weight to between .3-.6 if you are attempting to use a Text Prompt alongside your Image Prompt. Image Prompts will also inherit a large number of the model characteristics - If you are using a more Artistic base model,  you'll find outputs to be more artistic in nature.

Image Prompt

Text Prompt

a deer in a glade

Image Weight .25	Image Weight .5	Image Weight .75	Image Weight 1

Begin/End %

The Begin/End setting controls which steps in the generation process are influenced by the Reference Image / IP-Adapter. 0 represents the start of the generation, 1 represents the end.  Portions of your generation that are not influenced by the Reference Image will be solely processed using other control mechanisms and your prompt - This can be a strong way to get direct control over the generation process.

Using a Reference Image

To add a Global or Regional Reference Image, first click on the ‘Layers’ tab on the right-hand side. Then, click the ‘Add Layer’ button and select the ‘Global Reference Image’ or ‘Regional Reference Image’ option. This will create the respective layer, with a default IP Adapter model selected.

Alternatively, from the Gallery, drag an image into the canvas over the “Reference Image” drop area that you’d prefer to use, and it’ll be created as a layer and added to the canvas.

Next, select the IP adapter model you’d like to use - The “Plus” version of the model will have a more powerful effect, and be more literal on exact content/placement.

If you didn’t use an image to create this layer, you can either upload an image directly from your desktop or go to your gallery, drag an image over to the Layers tab, and drop it onto the upload button within the module.

While a Global Reference Image will influence the entire image generation, Regional Reference Images can be used to influence specific parts of an image. To select the part of the image you would like to influence, you can apply a mask using the brush tool in the Control Canvas.

Took see the Image Prompt Adapter in action, take a look at the video below (starting at 9:10):