HRDiT: Training-Free High-Resolution Image Generation with Off-the-Shelf DiT Models

Abstract

Training-free text-to-high-resolution image generation has recently attracted growing research attention. However, existing studies on this task primarily focus on adapting off-the-shelf U-Net-based diffusion models to high resolutions, with limited progress on adapting off-the-shelf Diffusion Transformer (DiT) models despite their strong text-to-image generation capabilities at limited resolutions. In this work, we find two key challenges particularly hindering the application of off-the-shelf DiT models for high-resolution image synthesis in a training-free manner, namely, spatial disorder and long generation time. To address these challenges, we propose a novel method tailored to adapt off-the-shelf DiT models for high-resolution image synthesis. Extensive experiments show the efficacy of our method.

Compared with FLUX.1-dev

FLUX at 1024×1024

Left is a 1024×1024 image from FLUX.1-dev, scaled up; right is HRDiT. Drag to compare.

HRDiT at 4096 by 4096: A dense hillside city at dusk, thousands of lit apartment windows stacked up the slope, laundry and air conditioners on the balconies

Stock FLUX.1-dev at 1024 by 1024, enlarged: A dense hillside city at dusk, thousands of lit apartment windows stacked up the slope, laundry and air conditioners on the balconies

HRDiT at 4096 by 4096: A spiderweb strung with dew at sunrise, each droplet holding an inverted image of the meadow behind

Stock FLUX.1-dev at 1024 by 1024, enlarged: A spiderweb strung with dew at sunrise, each droplet holding an inverted image of the meadow behind

HRDiT at 4096 by 4096: A fine ink drawing of a brass clockwork orrery, dense crosshatching, gears and rings casting shadow on a wooden desk

Stock FLUX.1-dev at 1024 by 1024, enlarged: A fine ink drawing of a brass clockwork orrery, dense crosshatching, gears and rings casting shadow on a wooden desk

HRDiT at 4096 by 4096: A Moroccan riad courtyard at midday, intricate zellige tilework, carved stucco arches, a still fountain and orange trees

Stock FLUX.1-dev at 1024 by 1024, enlarged: A Moroccan riad courtyard at midday, intricate zellige tilework, carved stucco arches, a still fountain and orange trees

HRDiT at 4096 by 4096: A Gothic cathedral rose window seen from inside, stained glass throwing coloured light across worn stone flagstones

Stock FLUX.1-dev at 1024 by 1024, enlarged: A Gothic cathedral rose window seen from inside, stained glass throwing coloured light across worn stone flagstones

HRDiT at 4096 by 4096: An autumn gorge with a waterfall, wet mossy boulders, birch and maple in full colour, low sunlight through the canopy

Stock FLUX.1-dev at 1024 by 1024, enlarged: An autumn gorge with a waterfall, wet mossy boulders, birch and maple in full colour, low sunlight through the canopy

HRDiT at 4096 by 4096: A coral reef in clear shallow water, branching corals and anemones, a school of small orange fish, sunlight rippling on the sand

Stock FLUX.1-dev at 1024 by 1024, enlarged: A coral reef in clear shallow water, branching corals and anemones, a school of small orange fish, sunlight rippling on the sand

HRDiT at 4096 by 4096: A desert canyon at golden hour, layered sandstone walls in orange and violet, a narrow green river far below

Stock FLUX.1-dev at 1024 by 1024, enlarged: A desert canyon at golden hour, layered sandstone walls in orange and violet, a narrow green river far below

FLUX at 4096×4096

Left is FLUX.1-dev run directly at 4096×4096; right is HRDiT, same prompt and seed.

Stock FLUX.1-dev asked for 4096 by 4096 in one pass: a featureless field.

HRDiT at 4096 by 4096: A young woman in an embroidered folk costume standing in a summer field, dense threadwork and beading, soft overcast light

Method

HRDiT is a training-free framework built from two components, each aimed at one of the two challenges and at the cause we identify behind it. It runs on off-the-shelf weights and needs no fine-tuning.

Off-the-shelf DiT models at 1024x1024 versus direct and I-Max adaptation at 4096x4096. — Off-the-shelf DiT models generate high-quality images quickly at the resolution they were trained on (1024×1024). Pushed to 4096×4096 — either directly or with the DiT-tailored I-Max — they produce **spatial disorder** (red boxes) and take a very **long time** (red text).

Spatial Position Alignment (SPA)

Our analysis traces spatial disorder to the limited expressiveness of the positional embedding mechanism once it is stretched to high resolutions: at high token counts it can no longer keep different pairwise positional signals apart after attention. SPA restores that distinction with two complementary operations. Bundle replaces each token index with a bundle index before it enters the positional encoding, which shrinks the set of pairwise signals the mechanism has to separate. Slide then varies where the bundle boundaries fall and averages over the variants, recovering the positional distinctions that bundling alone would erase inside a bundle.

Illustration of the bundle and slide operations on seven token indices. — The **bundle** and **slide** operations, with T = 7 tokens and bundle size N = 3. (a) Bundling maps the seven token indices onto three bundle indices. (b) Sliding the bundle boundaries produces N variants of the mapping, so tokens that share a bundle in one variant are separated in another.

Head-adaptive Attention Pruning (HAP)

Profiling shows where the time actually goes: multi-head attention dominates the cost at high resolutions, accounting for over 90% of the generation time for both FLUX and Stable Diffusion 3 at 8K. HAP gives every attention head its own attention scope — the range of image tokens that head is allowed to look at — and prunes the attention computed outside it. The scopes are derived once, offline, from the off-the-shelf model, so generation stays training-free.

HRDiT Training-Free High-Resolution Image Generation with
Off-the-Shelf Diffusion Transformer Models