Proto-Objects vs. Image Patches for Visual Reinforcement Learning

Visual reinforcement learning agents must turn high-dimensional images into compact information that a controller can use. One common strategy is to divide every frame into fixed-size patches and let an attention mechanism select the most relevant ones. Our NeuroProto work asks a different question: what if the visual tokens correspond to coherent regions in the scene instead of arbitrary rectangles?

What is the difference between image patches and proto-objects?

Image patches divide the visual field according to geometry. Each token corresponds to a fixed rectangular location, regardless of whether that rectangle contains one object, part of an object, or mostly background.

Proto-objects are intermediate visual structures: coherent regions that share local visual properties before full semantic object recognition. In NeuroProto, image segmentation produces these regions and each region is encoded with a compact descriptor containing properties such as position, shape, size, and color.

Why can fewer visual tokens matter?

Attention mechanisms spend computation deciding which pieces of visual information are relevant. When the input contains hundreds of small patches, the attention system must evaluate many candidate tokens even when large parts of the scene are visually uniform or irrelevant to the task.

A proto-object representation compresses the scene before attention is applied. In the tested environments, the difference was large: 529 patch tokens per frame versus about a dozen proto-object tokens. This changes the scale of the selection problem and allows the attention and controller networks to be smaller.

This reduction is especially relevant to neuroevolution, where every additional parameter expands the search space that evolution must optimize. NeuroProto used 1,406 parameters in the compared proto-object configuration versus 3,667 in the patch-based implementation, a reduction of about 62%.

What happened in Car Racing and Doom Take Cover?

The experiments compared proto-object attention with the patch-based approach that inspired the work. The main result is not that one representation wins every individual comparison; it is that the much more compact proto-object representation remained competitive while substantially reducing the input and model size.

In Car Racing, the reported proto-object configuration exceeded the patch-based score. In Doom, the k=1 proto-object configuration was slightly below the patch baseline, while k=10 was above it. This is why the strongest defensible conclusion is that proto-objects can match or exceed the patch-based approach in these experiments while using a substantially smaller representation.

What about training time?

The study also reported shorter wall-clock training times for the proto-object implementations in the tested setup: 36.5 hours for Car Racing compared with 97 hours for the patch-based implementation, and 33 hours (k=1) or 55 hours (k=10) for Doom compared with 85.5 hours for the patch baseline.

When are image patches still a good choice?

Proto-objects introduce assumptions that patches do not. A patch grid is simple, deterministic, and does not require a segmentation stage. It can therefore be preferable when segmentation is unreliable, when relevant information does not align with coherent regions, or when a general-purpose representation is more important than aggressively reducing the number of tokens.

Patches are also deeply integrated into modern vision architectures, which makes them easy to reuse with established tooling and pretrained models. Proto-objects are better viewed as an alternative representation worth testing when scene structure can provide a useful information bottleneck.

When are proto-objects especially attractive?

• When the environment contains large coherent visual regions and many fixed patches would be redundant.
• When reducing the number of visual tokens can simplify the attention mechanism or controller.
• When interpretability matters and selected regions should correspond more closely to meaningful parts of the scene.
• When neuroevolution or another expensive optimization procedure benefits from a smaller parameter space.

What does this result actually establish?

NeuroProto provides evidence from two visual reinforcement learning environments that proto-object-based attention can be a compact alternative to fixed-patch attention. It does not establish that segmentation-based tokens are universally superior, nor does it test every class of reinforcement learning environment, segmentation method, or modern vision architecture.

The more general research question is therefore still open: when should an agent attend to fixed spatial units, and when should it attend to perceptually grouped regions? The NeuroProto experiments suggest that object-like intermediate representations deserve more attention as a way to build smaller and more interpretable visual agents.

References

1. R. C. Pinto and A. R. Tavares. Neuroevolution of Self-Attention Over Proto-Objects. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2025), 1300–1308.
2. Y. Tang, D. Nguyen, and D. Ha. Neuroevolution of Self-Interpretable Agents. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2020), 414–424.