ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion

Spend Gaussians where the scene is hard to reconstruct

The dominant design in feed-forward 3DGS is a pixel-aligned formulation: one Gaussian is regressed at every input pixel and lifted along its camera ray, reducing primitive placement to depth estimation. The number and placement of primitives are then determined by image resolution and input viewpoints rather than scene complexity. As a result, simple and textureless regions produce many redundant Gaussians, while challenging regions cannot receive the additional capacity they demand.

ATSplat restores the scene-adaptive capacity allocation of per-scene 3DGS optimization. It recasts three guiding principles of 3DGS–sparse initialization, free 3D placement, adaptive capacity allocation–as feed-forward operations centered on a set of 3D anchor tokens: coarse patches are unprojected into a sparse scaffold, each anchor is regressed into local Gaussians with learnable 3D offsets, and high-uncertainty anchors are expanded during decoding.

ATSplat teaser: sparse-to-adaptive allocation of compact 3D Gaussians — ATSplat reconstructs compact 3D Gaussians from multi-view captures in a single forward pass. Unlike dense pixel-aligned feed-forward methods that place Gaussians according to input image grids, ATSplat starts from sparse 3D anchor tokens and adaptively expands tokens associated with challenging regions. This **sparse-to-adaptive allocation** concentrates representational capacity according to scene complexity, enabling high-quality novel-view rendering with only **311K Gaussians**.

Abstract

3D Gaussian Splatting (3DGS) achieves high-quality novel-view synthesis by optimizing freely placed primitives in 3D and adaptively densifying them in under-reconstructed regions. However, this scene-adaptive capacity allocation is largely lost in existing feed-forward 3DGS methods, which commonly regress Gaussians at input pixels and lift them along camera rays. Such pixel-aligned formulations make the number and placement of primitives depend on image resolution and input viewpoints rather than scene complexity, resulting in dense and often redundant Gaussian sets. We present ATSplat, a feed-forward 3DGS framework that restores the adaptive allocation capability of 3DGS optimization through Adaptive 3D Tokens. ATSplat first lifts coarse patch-level depth and camera cues into sparse 3D anchor tokens, forming a compact scaffold of the scene. Each token is then regressed into local Gaussians with learnable 3D offsets, decoupling primitive placement from input image grids. An Adaptive Token Expansion module predicts a token-level uncertainty score, supervised by rendering error maps, and selectively expands high-uncertainty tokens through learnable expansion layers. This sparse-to-adaptive formulation enables ATSplat to concentrate primitives in challenging regions while maintaining a compact representation. Experiments on two representative datasets, RealEstate10K and DL3DV, show that ATSplat achieves state-of-the-art rendering quality while reducing the number of Gaussians by more than 5.7× compared with dense feed-forward 3DGS methods. From 12 input images at 512×960 resolution, ATSplat completes reconstruction in less than a second using a single commercial GPU, and renders high-quality novel views at 1136 FPS (512×960) with only 311K Gaussians.

Method

Three guiding principles of 3DGS–sparse initialization, free 3D placement, adaptive capacity allocation–recast as feed-forward operations centered on a set of 3D anchor tokens.

01

Sparse anchor tokens

Patch-level depths predicted from frozen DINO features are unprojected along camera rays, assigning coarse patch features to sparse 3D locations that form an anchor scaffold: a scene-conditioned initialization obtained in a single forward pass.

02

Adaptive Token Expansion

A lightweight MLP estimates a per-token uncertainty score at each decoder block, supervised by rendered uncertainty maps to match actual rendering errors. The top-scoring tokens are expanded into child tokens, increasing representational capacity only where the scene is under-reconstructed.

03

Anchors → local Gaussians

Each refined anchor token is regressed into K local Gaussians whose centers are predicted as learnable 3D offsets from the anchor. This decouples primitive placement from the input pixel grid and allows primitives to be freely placed beyond camera rays.

Visualization of uncertainty scores and selected tokens

Predicted uncertainty identifies under-reconstructed regions

A feed-forward model cannot directly access rendering errors during decoding, so ATSplat learns to approximate them. The predicted uncertainty (third column) is high around regions with larger rendering errors (second column), including fine structures and visually complex areas. The selected tokens are concentrated in these uncertain regions (right), indicating that ATE successfully identifies where additional Gaussians are needed.

Results

Novel-view synthesis on RealEstate10K and DL3DV, compared with feed-forward 3DGS baselines and optimization-based 3DGS.

DL3DV · 12 input views · 512×960

ATSplat recovers fine details and thin structures in less than a second with 311K Gaussians, whereas 3DGS requires over 10 minutes of per-scene optimization and 553K Gaussians to obtain these results.

Drag the divider to compare.

ATSplat (Ours) 3DGS

RealEstate10K · 2 input views · 256×256

ATSplat achieves rendering quality comparable to state-of-the-art feed-forward methods while using only 23K Gaussians, a 5.7× reduction compared to the dense pixel-aligned baselines at 131K.

Ground Truth

ATSplat (Ours) · 23K

DepthSplat · 131K

Select a baseline to compare.

DL3DV · 6 input views · 256×448

ATSplat outperforms prior feed-forward baselines while using substantially fewer Gaussians: 120K against 688K for DepthSplat, a 5.7× reduction. Thin structures, intricate details, and view-dependent appearance are faithfully reconstructed.

Ground Truth

ATSplat (Ours) · 120K

DepthSplat · 688K

More scenes

Additional novel views rendered by ATSplat on DL3DV, covering both indoor and outdoor environments with diverse scene layouts and complex geometry (6 input views, 120K Gaussians).

Reconstructed 3D Scenes

DL3DV · 12 input views · 512×960

Explore ATSplat reconstructions in an interactive viewer.

Drag to orbit · right-drag to pan · scroll to zoom

BibTeX

@misc{cho2026atsplat,
      title={ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion},
      author={In Cho and Jeonghwan Cho and Mijin Yoo and Gim Hee Lee and Seon Joo Kim},
      year={2026},
      eprint={2607.20417},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2607.20417},
}