Skip to content
Keypointguided placement for jewellery and apparel
Inpaintingrather than flat overlay compositing
Open sourcewith Docker packaging

The problem

Virtual try-on needs the generated item to sit in the right place, follow the body, and still look photographic — naive image generation fails all three.

Approach

  • Used Stable Diffusion 2 inpainting so the generated item is composited into the original photo rather than overlaid.
  • Tracked facial and hand keypoints with MediaPipe to position the item before generation.
  • Applied OpenCV spatial alignment and xFormers memory optimisation to keep the pipeline usable.

Stack

  • Diffusers
  • Stable Diffusion 2
  • MediaPipe
  • OpenCV
  • Appwrite
  • xFormers
  • Docker

What these numbers do and do not prove

This is an open-source experiment. Quality depends heavily on the input photo and the item being fitted.

Source

Other work

Start a conversation