Using Text to Image Adapter (T2I-Adapter)

Text to Image Adapter (T2I-Adapter) is a feature that allows you guide the image generation by applying another model to the generation process. 

T2I-Adapter is similar in functionality to ControlNet, with different models providing different types of guidance to the generation process. 

Using T2I-Adapter

To use T2I-Adapter, you can simply select the desired model and adjust both the T2I-Adapter and Pre-processor settings to achieve the desired result. You can also use multiple T2I-Adapter models at the same time, allowing you to achieve even more complex effects or styles in your generated images.

Each T2I-Adapter has two main settings that are applied to the T2I-Adapter.

• Weight - Strength of the T2I-Adapter model applied to the generation for the section, defined by start/end.
• Start/End  - 0 represents the start of the generation, 1 represents the end. The Start/end setting controls what steps during the generation process have the T2I-Adapter applied.

Additionally, each T2I-Adapter section can be expanded in order to manipulate settings for the image pre-processor that adjusts your uploaded image before using it in when you Invoke.

T2I-Adapter can be used for all generation methods within InvokeAI; Text2Image,Image2Image, on the Unified Canvas and within Workflows.

Models

T2I-Adapter has a variety of models that can used during the generation process. Each model will provide a different type of guidance, leading to different effects on the resulting image.  

Canny:

When the Canny model is used in T2I-Adapter, Invoke will attempt to generate images that match the edges detected. 

Canny edge detection works by detecting the edges in an image by looking for abrupt changes in intensity. It is known for its ability to detect edges accurately while reducing noise and false edges, and the preprocessor can identify more information by decreasing the thresholds.

Sketch:

When the Sketch model is used in T2I-Adapter, Invoke will attempt to generate images that match the edges detected. 

Sketch works by generating a line based drawing from an input image. 

Color:

When the Color model is used in T2I-Adapter, Invoke will attempt to generate images that match the the color map that is detected. 

Depth:

When the Depth model is used in T2I-Adapter, Invoke will attempt to generate images that match the depth of objects in the input image. Depth works by generating a depth map from an input image.