JUICE: Unified Multimodal Modeling in Pretrained Representation Spaces

TL;DR

One process for images and text, in pretrained encoder spaces.

Why one process. Most unified models generate text with a pretrained language model and images with an attached diffusion head, one modality after the other. JUICE asks whether one process, trained on both modalities from scratch, can generate both.
Joint modeling in pretrained representation spaces. One flow matching Transformer models images and text in the continuous spaces of (SigLIP2 or WebSSL-DINO for images, Flan-T5 for text). Each modality has , so text-to-image, captioning, VQA and joint generation are paths through one square of noise levels.
Systematic design studies. Controlled pretraining runs settle four open questions, one takeaway each: continuous text spares images but costs wording, train where you sample, semantics over reconstruction, and scale reorders image spaces for generation, not captioning.
Scaling: one set of weights at 2.1B. At 224×224 and with no LLM initialization, each model handles compositional image generation, generation that needs world knowledge, and visual question answering: 0.88 GenEval (DINO variant); 85.27 DPG-Bench, 0.45 WISE and 73.1 VQAv2 (SigLIP2 variant).
Joint reasoning. An image and its text are generated together. With , a drawn maze route and its written moves are both valid and match 93.7% of the time (54.6% when sampled separately; SigLIP2 model adapted to mazes), within 1.5 points of the better fixed order, without choosing one. On visual spatial planning, JUICE solves 99.8% of maps at the training sizes.

0.88 GenEval DINO variant
85.27 DPG-Bench SigLIP2 variant
0.45 WISE SigLIP2 variant
73.1 VQAv2 SigLIP2 variant
54.6%→93.7% Maze joint success independent → coupled · SigLIP2, maze-adapted

2.1B backbone 224×224 no LLM initialization

JUICE overview. Left: frozen image and text encoders feed one MM-RiT backbone with joint attention over image, text and time tokens, followed by frozen decoders. Right: text-to-image, visual question answering and maze generation use the same interface. — **One backbone, every task.** (a) Continuous image and text states from frozen encoders share one MM-RiT backbone through bidirectional joint attention. (b) Text-to-image, visual question answering and maze solving use the same interface: clean context sits at t = 1 and targets are generated from t < 1. Samples are selected, 224×224.

01 Why one process Most unified models treat the two modalities asymmetrically. JUICE generates both with one process, learned together from scratch. 1A The usual design1B One process for both1C Why encoder spaces
02 Joint modeling in pretrained representation spaces Each modality gets its own noise level, every task is a path through one square, and one backbone reads and writes frozen encoder states. 2A Two clocks, one square2B The frozen encoders2C The MM-RiT block2D Decoders2E Training and sampling
03 Systematic design studies Four open questions, settled by controlled pretraining runs, one takeaway each. 3A Generative frameworks3B Training task mixtures3C Representation spaces3D Scaling
04 Scaling: one set of weights at 2.1B Two 2.1B models, one per image space, each with one set of weights for generation and understanding. 4A Image generation4B Visual understanding4C Selected samples
05 Joint reasoning An image and its text, generated together and made to agree. The order becomes a choice at sampling time. 5A Joint generation5B Mazes with multiple solutions5C Visual spatial planning5D Order at sampling time

Why one process

Most unified models treat images and text asymmetrically. JUICE asks whether both can instead be generated by one process that is trained on them jointly from scratch.

1AThe usual design

In the dominant design, a pretrained autoregressive language model generates the text and drives a diffusion or flow matching head that generates the image; other models quantize images and generate both modalities as discrete tokens. Either way, the two modalities are generated by different processes or through a quantizer, and the joint distribution is factorized in a fixed order: an output that contains both an image and text is produced one modality after the other, and the first cannot be revised in light of the second. Vision is also usually added to a model that was first pretrained on language, although controlled studies of multimodal pretraining find that late integration makes models lean on language priors and that training the modalities jointly from the start works better.

1BOne process for both

JUICE takes the other route: a model that treats both modalities alike, generates them with one process, and learns them together from the start.

1CWhy encoder spaces

Such a model needs two things: a space in which one process can generate both modalities, and a source of knowledge for a backbone trained without a language model. Frozen pretrained encoders provide both. Their spaces are continuous for images and text alike and carry visual and linguistic knowledge learned at scale, and recent studies find them to be good generative targets for vision and for language, although they were never built for generation.

In these spaces, one flow matching process generates both modalities with the same objective, the same features serve as targets and as context, and a model of the joint distribution also contains its conditionals. Giving each modality its own noise level turns this into one square of tasks (§02).

Two ways to build a unified model. Most unified models, as described above, and JUICE.

	Most unified models	JUICE
Text	tokens from a pretrained language model	continuous states of a frozen Flan-T5 encoder
Image	an attached diffusion or flow matching head (or quantized tokens)	continuous features of a frozen SigLIP2 or WebSSL-DINO encoder
Generation	different processes, or a quantizer	one flow matching process for both
Training	vision usually added to a model pretrained on language	both modalities together, from scratch
Image and text output	one modality after the other, in a fixed order	generated jointly; the order becomes a choice at sampling time (§05)

The table sums up the difference. §02 builds the model on frozen encoder spaces; it leaves four design questions open, which §03 settles.

Joint modeling in pretrained representation spaces

One flow matching Transformer, trained from scratch, models the joint distribution of images and text in the continuous spaces of frozen pretrained encoders. With a noise level for each modality, every task is a path through one square, with no fixed generation order.

2ATwo clocks, one square

JUICE learns the joint distribution of an image representation and a text representation, both read from frozen encoders. One network takes the pair of noisy states and predicts both clean ones. Each modality keeps its own clock: an image time t_i and a text time t_t, each running from 0 (pure noise) to 1 (clean).

The two clocks span a unit square, and a task is a path through it. Text-to-image generation holds the text clean (t_t = 1) and runs along the top edge. Captioning and VQA hold the image clean (t_i = 1) and climb the right edge; for VQA the question also stays at t = 1 while only the answer is generated. Joint generation starts in the corner where both streams are noise and crosses the interior to the corner where both are clean.

The joint path hugs the left side because the image schedule keeps the image noisier for longer. Training covers the same ground that sampling uses: conditional denoising on the edges, joint denoising in the interior. The samplers compared in §05 are paths as well: generating one modality first runs along the edges, finishing one stream before conditioning the other on it, while coupled sampling crosses the interior and advances both streams in every backbone call. Separate timesteps per modality follow UniDiffuser; JUICE applies them to frozen encoder states for both modalities, with a backbone trained from scratch.

One backbone over frozen encoders

Under the square sits one network, MM-RiT, a flow matching Transformer that reads and writes the states of frozen encoders. Everything else stays fixed while it trains; the decoders are trained separately.

Architecture: image and text are encoded by frozen encoders, normalized and noised with their own scale and time, processed by MM-RiT blocks with per-modality projections and one joint attention over image, text and time tokens, then mapped back by frozen decoders. — **Architecture.** Flame: trained. Cube: frozen.

“2.1B” counts the trainable MM-RiT backbone only. With the frozen encoders and decoders loaded, the SigLIP2 model has about 3.03B parameters (DINO: 2.91B).

Four choices in this design are not settled by earlier unified models: how text is generated, which tasks are trained, which encoder spaces are read and written, and whether the answers hold as the model grows. §03 settles them with controlled runs.

Systematic design studies

Building JUICE meant answering four questions that existing unified models leave open: whether text should be generated as continuous states or kept as tokens, which denoising tasks to train on, which encoder spaces to read and write, and whether these answers still hold as the model grows. Each is settled with controlled pretraining runs: a 668M reference backbone (179M to 2.1B in the scaling study), 50k updates, one choice changed at a time, each configuration run once.

3AGenerative frameworks

Should text be generated as continuous states, or kept as tokens?

Study setup Four ways to build a unified model

The pretraining study compares JUICE with three frameworks that keep text discrete, all on the same 668M trunk with two streams, the same data, task ratios and schedule: Transfusion-style (autoregressive text, flow matching images), D-DiT-style (masked text diffusion, flow matching images) and MMaDA-style (masked diffusion for both, over vector-quantized image codes). Takeaway 1 below reports what happened.

Four frameworks compared at 668M: (a) JUICE with continuous image and text states; (b) Transfusion-style with autoregressive text; (c) D-DiT-style with masked text diffusion; (d) MMaDA-style with masked diffusion over VQ image codes and text tokens. — **The four frameworks of the pretraining study.** Only (a) generates text as continuous states.

Training curves over 50k updates for four frameworks on GenEval, DPG-Bench, CIDEr and CLIPScore. JUICE is highest on GenEval and DPG-Bench; the Transfusion-style framework with autoregressive text is far ahead on CIDEr. — **Four frameworks over training (668M).** JUICE leads GenEval and DPG-Bench at every checkpoint and ends lowest on CIDEr, far behind autoregressive text.

Of four generative frameworks, only the one that writes text as continuous states keeps GenEval level with a model trained only for text-to-image generation (48.41 vs 47.86 at the same guidance scale). The three frameworks that write text as tokens all fall below it, even with the same frozen T5 condition. The price is exact wording: CIDEr is 88.80 against 160.52 for autoregressive text, while CLIPScore is on par (80.20 vs 80.37).

So the captions describe the image as well as autoregressive ones but word it differently from the references, which is what you would expect when errors in a continuous state drift toward nearby states with a similar meaning. With guidance tuned per model, the text-to-image-only model peaks at 51.23 and leads by 2.8 points.

Takeaway 1 Generative frameworks

Continuous text spares images but costs wording.

3BTraining task mixtures

Which denoising tasks should training cover?

Sampling uses both the edges of the square and its interior, so training has to cover both. With each recipe at its best guidance scale, Mixing (55.6% joint, 22.2% text-to-image, 22.2% image-to-text) reaches 48.41 GenEval, Multi-task (edges only) 44.29, and Joint only (interior only) 31.33. Joint only never sees the exactly clean condition that conditional inference hands it; Multi-task never trains on the interior that joint sampling crosses.

Mixing also supervises each modality more often (77.8% of examples vs 50%), so the study controls for that. Replacing joint examples with matched conditional ones, or blocking cross-modal attention on them, falls 2.51 and 2.92 GenEval short of Mixing. The joint examples give more than extra supervision.

Takeaway 2 Training task mixtures

Train where you sample.

3CRepresentation spaces

Which encoder spaces should the model read and write?

Bar charts at 668M. Image spaces with T5 text: SigLIP2 and DINO far ahead of FLUX VAE latents and raw pixels on all four metrics. Text spaces with SigLIP2 images: Flan-T5 leads Qwen3-Embedding, Qwen3-VL-Embedding and SigLIP2 text features. — **Representation spaces at 668M.** Top: image spaces with T5 text. Bottom: text spaces with SigLIP2 images.

Good reconstruction does not make a good generation space. The FLUX VAE has the best rFID among the learned spaces (0.288 vs 0.435 for SigLIP2), yet GenEval in it drops to 35.42, from 48.41 with SigLIP2; raw pixels reach 24.48. SigLIP2 and DINO are close on generation (48.41 vs 49.07), and SigLIP2's alignment with language shows in captioning, 20.0 CIDEr ahead.

The text side tells a similar story. Flan-T5 states beat larger embedding models (Qwen3-Embedding trails by 5.3 GenEval) and SigLIP2's own text features (33.08 GenEval), even though every one of these spaces lets a decoder recover 97.5–99.2% of the tokens.

Takeaway 3 Representation spaces

Semantics over reconstruction.

3DScaling

Do these preferences survive as the model grows?

Four panels from 179M to 2.1B backbone parameters, SigLIP2 versus DINO. DINO leads GenEval at small scale and SigLIP2 overtakes at 2.1B; SigLIP2 leads CIDEr and CLIPScore at every size. — **Scaling the backbone from 179M to 2.1B** (pretraining, 50k updates). DPG-Bench in this figure uses the time-swap text reference.

At 179M, DINO is far ahead on GenEval (32.90 vs 18.99 for SigLIP2). The gap closes as the backbone grows, the two tie at 1.33B, and at 2.1B the order has flipped (SigLIP2 56.81 vs DINO 52.78). Captioning never flips: SigLIP2 leads CLIPScore at every size and ends at 86.64 against 72.32.

After finetuning, the two 2.1B models differ by less than 0.02 GenEval (DINO 0.88, SigLIP2 0.87), so neither image space keeps a clear generation advantage, while SigLIP2 leads on almost every understanding benchmark.

Takeaway 4 Scaling

Scale reorders image spaces for generation, not captioning.

With these four answers the recipe is fixed: text as continuous states, joint plus conditional training, and semantic encoder spaces (SigLIP2 or DINO for images, Flan-T5 for text). §04 scales it to 2.1B.

Scaling: one set of weights at 2.1B

With these answers in hand, two 2.1B models are trained, one per image space. At 224×224 and with no LLM initialization, each uses one set of weights for compositional image generation, generation that needs world knowledge, and visual question answering.

Each model is pretrained with the recipe above for 60k updates, then finetuned for 15k updates on 20.6M text-to-image and visual understanding examples. Text-to-image uses 25 Euler steps with guidance 5; VQA uses 16 steps with guidance 3, and the parallel decoder reads out all answer positions at once. The tables compare with unified models of 1–3B parameters.

4AImage generation

Image generation. GenEval (0–1), DPG-Bench (%) and WISE (0–1) against unified models of 1–3B. JUICE samples 224×224 images. Its size counts the trainable backbone only (about 3.03B SigLIP2 / 2.91B DINO fully loaded); baseline sizes are as reported.

Model	Size	GenEval ↑			DPG ↑	WISE ↑
Model	Size	Overall	Count	Position	DPG ↑	WISE ↑
Muddit	1B	0.61	0.54	0.19	–	–
OneFlow (Mixed)	1B	–	–	–	80.30	–
Show-o	1.3B	0.68	0.66	0.31	–	0.35^a
JanusFlow	1.3B	–	–	–	–	0.18
UniDisc	1.4B	0.42	0.15	0.13	–	–
FUDOKI	1.5B	0.77	0.56	0.68	83.63	–
Janus-Pro-1B	1.5B	0.73	0.51	0.65	82.63	0.26
Show-o2	1.5B	0.73	0.55	0.46	85.02	0.35
Harmon	1.5B	0.76	0.66	0.74	–	0.41
TUNA	1.5B	0.88	0.83	0.81	86.03	–
D-DiT	2.1B	0.65	0.54	0.32	–	–
JUICE SigLIP2	2.1B	0.87	0.78	0.85	85.27	0.45
JUICE DINO	2.1B	0.88	0.81	0.85	84.55	0.37

GenEval: the DINO variant ties TUNA at 0.88; TUNA still leads on counting and attribute binding. Both JUICE variants have the highest position score (0.85). Overall GenEval moves by at most 0.003 between 25 and 100 sampling steps.
DPG-Bench: 85.27 is above Show-o2 (85.02) and below TUNA (86.03).
WISE: 0.45 is the highest overall score among 1–3B unified models that report one. Two larger references score higher, FLUX.1-dev (12B, text-to-image only) 0.50 and BAGEL 0.52, while Janus-Pro-7B scores 0.35. GPT-4o judge; the 95% CI on JUICE's score is about ±0.02. ^aShow-o at 512².

4BVisual understanding

Visual understanding. Accuracy (%) against unified models of 1–3B. Init: what the model starts from. JUICE sees 224×224 images.

Model	Size	Init	Res.	VQAv2 ↑	GQA ↑	MMBench ↑	AI2D ↑	MMMU ↑
Muddit	1B	T2I	512²	68.2	57.5	–	–	27.6
OneFlow	1B	LLM	512²	67.7	57.8	69.0	58.5	29.8
Show-o	1.3B	LLM	256²	64.7	54.2	–	–	25.0
Show-o	1.3B	LLM	512²	69.4	58.0	–	–	26.7
JanusFlow	1.3B	LLM	384²	79.8	60.3	74.9	–	29.3
Harmon	1.5B	LLM	512²	–	58.9	65.5	–	38.9
FUDOKI	1.5B	LLM	384²	–	57.6	73.9	–	34.3
Janus-Pro-1B	1.5B	LLM	384²	–	59.3	75.5	–	36.3
Show-o2	1.5B	LLM	432²	–	60.0	67.4	69.0	37.1
TUNA	1.5B	LLM	512²	–	61.4	–	71.4	39.1
D-DiT	2.1B	T2I	256²	59.5	55.1	–	–	–
D-DiT	2.1B	T2I	512²	60.1	59.2	–	–	–
JUICE SigLIP2	2.1B	None	224²	73.1	58.6	66.0	34.4	29.6
JUICE DINO	2.1B	None	224²	68.8	55.7	55.8	31.9	31.5

The SigLIP2 variant is competitive on several perception benchmarks (VQAv2 73.1, GQA 58.6, MMBench 66.0) and weak where domain knowledge matters (AI2D 34.4, MMMU 29.6). Every other model here starts from a pretrained language or text-to-image model.
Sizes for JUICE count the trainable backbone; fully loaded with frozen encoders and decoders it is about 3.03B (SigLIP2) or 2.91B (DINO). Baseline sizes as reported.

4CSelected samples

Text → Image

SigLIP2 variant

SigLIP2 sample for the prompt: an astronaut riding a horse on the surface of the moon, Earth rising in the background — an astronaut riding a horse on the surface of the moon, Earth rising in the background

SigLIP2 sample for the prompt: a ukiyo-e woodblock print of a great wave crashing beside a red lighthouse under a full moon — a ukiyo-e woodblock print of a great wave crashing beside a red lighthouse under a full moon

SigLIP2 sample for the prompt: an art nouveau poster of a peacock among white lilies, gold and teal palette, ornate border — an art nouveau poster of a peacock among white lilies, gold and teal palette, ornate border

SigLIP2 sample for the prompt: a giant tortoise carrying a tiny forest with a cabin on its shell, walking across a meadow — a giant tortoise carrying a tiny forest with a cabin on its shell, walking across a meadow

SigLIP2 sample for the prompt: a rain-soaked cyberpunk alley with holographic signs and a steaming noodle stand, teal and magenta light — a rain-soaked cyberpunk alley with holographic signs and a steaming noodle stand, teal and magenta light

SigLIP2 sample for the prompt: a photo of four donuts — a photo of four donuts

SigLIP2 sample for the prompt: a crystal castle on a floating island above the clouds at golden hour, matte painting — a crystal castle on a floating island above the clouds at golden hour, matte painting

SigLIP2 sample for the prompt: a stained-glass window depicting a red fox in an autumn forest — a stained-glass window depicting a red fox in an autumn forest

SigLIP2 sample for the prompt: a photo of a suitcase left of a banana — a photo of a suitcase left of a banana

SigLIP2 sample for the prompt: a glass jar of honey with a wooden dipper beside a honeycomb, backlit by the sun — a glass jar of honey with a wooden dipper beside a honeycomb, backlit by the sun

SigLIP2 sample for the prompt: a loose watercolor of a Venetian canal at dusk with gondolas, soft pastel wash — a loose watercolor of a Venetian canal at dusk with gondolas, soft pastel wash

SigLIP2 sample for the prompt: a photo of a wine glass right of a hot dog — a photo of a wine glass right of a hot dog

DINO variant

DINO sample for the prompt: an astronaut riding a horse on the surface of the moon, Earth rising in the background — an astronaut riding a horse on the surface of the moon, Earth rising in the background

DINO sample for the prompt: a ukiyo-e woodblock print of a great wave crashing beside a red lighthouse under a full moon — a ukiyo-e woodblock print of a great wave crashing beside a red lighthouse under a full moon

DINO sample for the prompt: an art nouveau poster of a peacock among white lilies, gold and teal palette, ornate border — an art nouveau poster of a peacock among white lilies, gold and teal palette, ornate border

DINO sample for the prompt: a giant tortoise carrying a tiny forest with a cabin on its shell, walking across a meadow — a giant tortoise carrying a tiny forest with a cabin on its shell, walking across a meadow

DINO sample for the prompt: a rain-soaked cyberpunk alley with holographic signs and a steaming noodle stand, teal and magenta light — a rain-soaked cyberpunk alley with holographic signs and a steaming noodle stand, teal and magenta light

DINO sample for the prompt: a photo of four donuts — a photo of four donuts

DINO sample for the prompt: a crystal castle on a floating island above the clouds at golden hour, matte painting — a crystal castle on a floating island above the clouds at golden hour, matte painting

DINO sample for the prompt: a stained-glass window depicting a red fox in an autumn forest — a stained-glass window depicting a red fox in an autumn forest

DINO sample for the prompt: a photo of a suitcase left of a banana — a photo of a suitcase left of a banana

DINO sample for the prompt: a glass jar of honey with a wooden dipper beside a honeycomb, backlit by the sun — a glass jar of honey with a wooden dipper beside a honeycomb, backlit by the sun

DINO sample for the prompt: a loose watercolor of a Venetian canal at dusk with gondolas, soft pastel wash — a loose watercolor of a Venetian canal at dusk with gondolas, soft pastel wash

DINO sample for the prompt: a photo of a wine glass right of a hot dog — a photo of a wine glass right of a hot dog

Selected samples, 224×224. The same twelve prompts for both 2.1B variants; 25 Euler steps, guidance 5.

World knowledge

CultureWISE #194

“A mythical creature embodying Singapore's origins”

Implied knowledge: Singapore's founding legend → the Merlion, a lion with a fish body.

TimeWISE #449

“The apple orchard when the farmers are harvesting pumpkins.”

Implied knowledge: Pumpkin harvest → autumn; apples still on the trees.

SpaceWISE #621

“Generate an image of a bird and a dog, with the smaller animal on top and the larger below”

Implied knowledge: The bird is the smaller animal → the bird on top, the dog below.

BiologyWISE #754

“A chameleon perfectly camouflaged against a green leaf”

Implied knowledge: Camouflage → skin color and pattern match the leaf.

PhysicsWISE #900

“The balloon that has been rubbed comes into contact with long hair”

Implied knowledge: Rubbing charges the balloon → the hair is pulled toward it.

ChemistryWISE #935

“A piece of gold that has been exposed to oxygen for decades”

Implied knowledge: Gold does not oxidize → still bright, no corrosion.

Selected samples, 224×224. One WISE prompt per category. The prompt leaves the needed knowledge implicit; the “implied knowledge” line is the benchmark's key, not model output.

Visual question answering

AttributeGQA

Which type of material is used to make the trash bin below the pepper shaker?

SigLIP2: plastic
DINO: plastic
Reference: plastic

CountingVQAv2

How many cows are there?

SigLIP2: 4
DINO: 4
Reference: 4

Spatial relationGQA

Is the green umbrella to the right or to the left of the person who is to the left of the palm tree?

SigLIP2: left
DINO: left
Reference: left

Spatial relationGQA

Are the thick books to the left or to the right of the computer that is to the right of the lamps?

SigLIP2: right
DINO: right
Reference: right

ActionGQA

What's the player doing?

SigLIP2: jumping
DINO: jumping
Reference: jumping

Multiple entitiesGQA

Which material is the clock that is to the left of the tennis ball made of, stainless steel or brick?

SigLIP2: stainless steel
DINO: stainless steel
Reference: stainless steel

Outside knowledgeOK-VQA

What is a male of this animal called?

SigLIP2: ram
DINO: ram
Reference: ram

Object existencePOPE

Is there a baseball bat in the image?

SigLIP2: no
DINO: no
Reference: no

Selected samples. Real benchmark photos, seen by the model at 224×224. Eight questions of seven types, answered by both 2.1B variants (16 steps, guidance 3).

Captions

SigLIP2: Spacious, modern bedroom with white furniture and a large window.
DINO: Modern bedroom with a large bed, mirror, and sliding glass door.

Input photo: people sharing a meal at a restaurant

SigLIP2: Three people enjoy a meal together at a cozy Arabic restaurant.
DINO: Three people enjoy a delicious meal together at a cozy restaurant.

Input photo: a laptop, roses and a cup on a bed

SigLIP2: A cozy workspace with a laptop, roses, and a cup of coffee or cream on a stylish bedspread.
DINO: A cozy workspace setup with coffee, laptop, camera, pink tulips, and tea on patterned fabric.

Input photo: a road through tea plantations

SigLIP2: A serene rural road winds through lush green tea plantations under a cloudy sky.
DINO: A serene road winds through lush green plantations under a bright sky.

SigLIP2: A vintage family portrait in colorful attire, the mid-20th century.
DINO: A family photo dressed in vintage attire, possibly from a historic event.

Input photo: a stone church with a clock tower

SigLIP2: A serene white church with arched windows and a clock tower, set against a sunny day.
DINO: A serene church with an architectural tower with stained glass windows and a clock on it stands under a bright blue sky.

Selected samples. Real photos, seen at 224×224, with the instruction “Describe the image concisely.” Short captions are fluent; long, detailed descriptions are less reliable and contain grammatical errors and the occasional invented word.

Image generation and understanding run along the edges of the square, with one modality held clean. Joint reasoning crosses the interior, where both outputs are generated together (§05).

Joint reasoning

Because JUICE models the joint distribution, it can generate an image and a text together. Some tasks need the two outputs to be correct and to agree, such as a route drawn on a map and the moves that describe it. With several valid routes, agreement is not automatic.

5AJoint generation

Without any condition, both streams start from noise and the model generates an image and a caption together. The text clock runs ahead, so the caption takes shape before the image does, and each stream conditions on the other at every step. In the selected samples below, the caption describes the image it is generated with, so the two streams agree even though neither is given. Step through a sample to watch both form.

Image generated jointly with the text beside it (SigLIP2 variant)

SigLIP2generated together, from noise

The image depicts a charming, cartoon-style character that appears to be a young boy. The boy has light brown hair, adorned with wing-like wings and stands out backward with a cheerful expression. He is dressed in a yellow tunic over a yellow shirt, paired with blue pants that have dotted patterns, a red cape and brown boots. He has a brown belt with a design on it. The character is also showing a peace sign on his forehead and is depicted in a playful, mischievous pose.

SigLIP2generated together, from noise

The image depicts a person standing on the end of a wooden bridge, facing away from the camera, towards a stunning mountainous landscape. The individual is dressed in outdoor attire, including a light gray long-sleeved shirt, dark pants, and black shoes. The bridge appears to be made of well-maintained, with lush green trees and grass on both sides. The landscape reflects the surrounding scenery, including the rugged mountains and overcast sky above.

Image generated jointly with the text beside it (DINO variant)

DINOgenerated together, from noise

The image depicts a tranquil and picturesque park scene with an old European castle in the background, characterized by its multiple levels, turrets, and golden domes. The sky is partly cloudy, suggesting it might be late afternoon or early evening. A stone bridge arches over a body of water adorned with greenery. Surrounding the water are lush green trees and flowers, and a gentle red path winds through the center. The overall atmosphere is serene and idyllic, evoking a sense of history and isolation.

DINOgenerated together, from noise

The image showcases a character that appears to be from a fantasy setting, characterized by a striking use of leather arm bands and a red headscarf. The character has neatly braided hair and is seated at a bar, surrounded by wooden shelves and various bottles filled with beer. The individual is also wearing a black sleeveless top, which adds to the sturdy and rugged aesthetic. The background reveals more shelves and other patrons with blue bottles, suggesting a typical setting in a beer shop.

Selected samples, 224×224. No condition: image and text start from noise and are generated together over 64 steps. In these samples, the caption describes the image it is generated with, so the two streams agree even though neither is given. The text is read out with the autoregressive caption decoder and cut where it starts to repeat; wording errors are left as generated.

Planning tasks test the harder case, in which the outputs must also be correct.

5BMazes with multiple solutions

The mazes here have two to sixteen shortest routes each, so both outputs can be valid and still disagree. Sampling each modality from its own marginal often picks a different valid route in each, a cross-modal mode mismatch, and joint success is only 54.6% for the maze-adapted SigLIP2 model. Generating one modality first avoids it, but the result then depends on the order, because mistakes in the first modality are passed on to the second (95.2% with the image first, 85.3% with the text first). The better order starts with the more reliable modality, here the image, but for a new task, which modality is more reliable is not known in advance. Generating one modality first is the fixed order of §01, here in a task where agreement can be checked.

Coupled sampling computes both velocity fields from the current joint state before either stream moves, so each stream conditions on the other as it forms. Two selected mazes, traced step by step, show it at work.

Who gives way

In both traced mazes, whichever output settles later changes to match the other. In Maze A the drawn route switches to the route of the moves at step 3. In Maze B the moves change from RURRD to RURDR at step 7 to follow the drawing. Because either modality can be the one that yields, coupled sampling is not simply copying one into the other. It does not collapse onto one canonical route either: on solved mazes it returns the stored reference route 43.5% of the time for SigLIP2, close to the 44.1% expected from a uniform pick among shortest routes (DINO: 47.0% vs 44.4%).

Maze joint success (%). Both outputs must be valid shortest routes and show the same route. 500 unseen mazes, 224×224 images, 16 backbone calls, mean of 2 seeds.

Sampler	Order	SigLIP2	DINO
Coupled	none	93.7	81.7
Image first	image → text	95.2	82.7
Text first	text → image	85.3	59.9
Independent	none	54.6	42.7

Coupled sampling lands 1.5 / 1.0 points below the better order and 8.4 / 21.8 above the worse one, without choosing either. Image first is significantly better than coupled for SigLIP2 (paired test, p = 0.017; DINO p = 0.19). Coupled loses at most 0.3 points when the budget is halved to 8 calls.
The maze models are the finetuned 2.1B models adapted for 5k updates on 200k mazes (5×5 to 7×7); a linear head trained for the task reads the moves.

5CVisual spatial planning

VSP asks for a path through a FrozenLake grid from start to goal without stepping into a hole. The SigLIP2 model, adapted for 5k updates on 55.9k maps of sizes 3×3 to 6×6, solves 99.8% of the official maps at those sizes, on par with the fine variant of LatentUM (99%), which plans visually step by step. On larger maps it never saw in adaptation, it solves 94% at 7×7 and 69% at 8×8.

On those unseen sizes the drawn path is right more often than the moves. Of the 37 failures, 29 drop exactly one move while the drawn path is the reference route, and drops grow more frequent with the length of a run of repeated moves. As on mazes, the image is the more reliable modality here, and these dropped moves match the cost in exact wording from Takeaway 1. Coupled sampling can still repair a move string: on nine of all 600 benchmark maps (levels 3 to 8), the moves fail at the first of 16 steps and solve the map by the last.

VSP success (%), 100 official maps per level, 224×224 images. Levels 7–8 are map sizes not seen in adaptation. Baseline numbers are taken from the LatentUM paper, which reports InternVL3.5 and ThinkMorph only as averages; ThinkMorph does not state which levels its average covers.

Model	Size	Task data	Avg. (Lv 3–6)	Lv 7	Lv 8
InternVL3.5	38B	none	20	–	–
ThinkMorph	7B	6k	76	–	–
LatentUM, coarse	4B	6k	85	–	–
LatentUM, fine	4B	18k step-by-step	99	–	–
JUICE SigLIP2	2.1B	55.9k	99.8	94	69

The player below steps through four maps from the paper's VSP figure, decoded at every one of the 16 coupled steps. In two of them the moves solve the map from the first step. The other two are among the nine whose moves fail at first and are corrected during sampling, while the drawn path stays on the reference route throughout.

5DOrder at sampling time

In a model of the joint distribution, the order in which modalities are generated is no longer fixed by the architecture: it becomes a choice at sampling time, and coupled sampling can leave it open. Coupled sampling does not beat the better order, but it comes close (93.7% against 95.2% with the image first on mazes, SigLIP2) without having to know which order that is. On visual spatial planning, JUICE solves nearly all maps of the training sizes (99.8%). Together with the generation and understanding results of §04, pretrained continuous spaces thus let one model understand, generate, and coordinate images and text without putting either one first.

Limitations and next steps

The models run at a native 224×224 resolution, which limits fine visual detail.
Exact wording lags behind autoregressive text. Long, detailed descriptions get the main objects, colors and layout right but contain grammatical errors and the occasional invented word.
Without a language model backbone, JUICE trails models initialized from LLMs on benchmarks that need domain knowledge, such as AI2D and MMMU.
Each pretraining configuration was run once.
The joint reasoning tasks are synthetic planning problems, and each uses its own task-adapted model.

The next steps follow from these limits: higher resolution, more accurate continuous text, and joint generation on real tasks in which several pairs of image and text outputs are valid.

Citation

BibTeX

@article{li2026juice,
  title   = {{JUICE}: Unified Multimodal Modeling in Pretrained Representation Spaces},
  author  = {Li, Jiahan and Su, Huangyuan and Tong, Shengbang and Yang, Shusheng and
             Guo, Ruihan and Huang, Suozhi and Zhao, Hexu and Sharma, Pratyusha and
             Yang, Sherry and Gu, Jiatao and Fergus, Rob and Cho, Kyunghyun},
  journal = {arXiv preprint arXiv:XXXX.XXXXX},
  year    = {2026}
}