Leon Sick

2026

1. 

   On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models

   Leon Sick, Lukas Hoyer, Yannick Strümpler, Christian Reisswig, Timo Ropinski, and 1 more author

   arXiv preprint arXiv:2610.32770, 2026

   Abs arXiv HTML

   On-policy self-distillation enables post-training foundation models without relying on stronger external teachers. However, existing approaches rely on ground-truth question-answer pairs, human-curated problem sets, or other supervised signals. In this work, we reformulate on-policy self-distillation as a self-training framework, driven by self-generated question-answer pairs and privileged reasoning demonstrations. We push supervision to the limit: guided solely by coarse, high-level topic descriptions, the model synthesizes its own curriculum of queries and privileged reasoning traces without access to any human-curated prompts or answers. We find that self-distillation on fully synthetic privileged information enables a model to improve across multiple domains while retaining its capabilities on domains it is not self-trained on. We demonstrate the generality of this framework across text and multimodal domains, achieving consistent self-improvement on standard language benchmarks and Visual Question Answering. Our findings demonstrate that foundation models possess rich latent capabilities that can be amplified without external task supervision or curated problem distributions.

2. 

   S2D: Sparse-To-Dense Keymask Distillation for Unsupervised Video Instance Segmentation

   Leon Sick, Lukas Hoyer, Dominik Engel, Pedro Hermosilla, and Timo Ropinski

   In Advances in Neural Information Processing Systems (NeurIPS), Dec 2026

   Abs arXiv HTML Code

   In recent years, the state-of-the-art in unsupervised video instance segmentation has heavily relied on synthetic video data, generated from object-centric image datasets such as ImageNet. However, video synthesis by artificially shifting and scaling image instance masks fails to accurately model realistic motion in videos, such as perspective changes, movement by parts of one or multiple instances, or camera motion. To tackle this issue, we propose an unsupervised video instance segmentation model trained exclusively on real video data. We start from unsupervised instance segmentation masks on individual video frames. However, these single-frame segmentations exhibit temporal noise and their quality varies through the video. Therefore, we establish temporal coherence by identifying high-quality keymasks in the video by leveraging deep motion priors. The sparse keymask pseudo-annotations are then used to train a segmentation model for implicit mask propagation, for which we propose a Sparse-To-Dense Distillation approach aided by a Temporal DropLoss. After training the final model on the resulting dense labelset, our approach outperforms the current state-of-the-art across various benchmarks.

2025

1. 

   CutS3D: Cutting Semantics in 3D for 2D Unsupervised Instance Segmentation

   Leon Sick, Dominik Engel, Sebastian Hartwig, Pedro Hermosilla, and Timo Ropinski

   In International Conference on Computer Vision (ICCV), Oct 2025

   Abs arXiv HTML

   Traditionally, algorithms that learn to segment object instances in 2D images have heavily relied on large amounts of human-annotated data. Only recently, novel approaches have emerged tackling this problem in an unsupervised fashion. Generally, these approaches first generate pseudo-masks and then train a class-agnostic detector. While such methods deliver the current state of the art, they often fail to correctly separate instances overlapping in 2D image space since only semantics are considered. To tackle this issue, we instead propose to cut the semantic masks in 3D to obtain the final 2D instances by utilizing a point cloud representation of the scene. Furthermore, we derive a Spatial Importance function, which we use to resharpen the semantics along the 3D borders of instances. Nevertheless, these pseudo-masks are still subject to mask ambiguity. To address this issue, we further propose to augment the training of a class-agnostic detector with three Spatial Confidence components aiming to isolate a clean learning signal. With these contributions, our approach outperforms competing methods across multiple standard benchmarks for unsupervised instance segmentation and object detection.

2. 

   Masked Scene Modeling: Narrowing the Gap Between Supervised and Self-Supervised Learning in 3D Scene Understanding

   Pedro Hermosilla, Christian Stippel, and Leon Sick

   Conference on Computer Vision and Pattern Recognition (CVPR), Oct 2025

   arXiv HTML

3. 

   Attention-Guided Masked Autoencoders for Learning Image Representations

   Leon Sick, Dominik Engel, Pedro Hermosilla, and Timo Ropinski

   In Oral Presentation, Proceedings of the Winter Conference on Applications of Computer Vision (WACV), Feb 2025

   Abs arXiv HTML Code

   Masked autoencoders (MAEs) have established themselves as a powerful pre-training method for computer vision tasks. While vanilla MAEs put equal emphasis on reconstructing the individual parts of the image, we propose to inform the reconstruction process through an attention-guided loss function. By leveraging advances in unsupervised object discovery, we obtain an attention map of the scene which we employ in the loss function to put increased emphasis on reconstructing relevant objects. Thus, we incentivize the model to learn improved representations of the scene for a variety of tasks. Our evaluations show that our pre-trained models produce off-the-shelf representations more effective than the vanilla MAE for such tasks, demonstrated by improved linear probing and k-NN classification results on several benchmarks while at the same time making ViTs more robust against varying backgrounds and changes in texture.

2024

1. 

   Unsupervised Semantic Segmentation Through Depth-Guided Feature Correlation and Sampling

   Leon Sick, Dominik Engel, Pedro Hermosilla, and Timo Ropinski

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2024

   Abs arXiv HTML Code

   Traditionally, training neural networks to perform semantic segmentation requires expensive human-made annotations. But more recently, advances in the field of unsupervised learning have made significant progress on this issue and towards closing the gap to supervised algorithms. To achieve this, semantic knowledge is distilled by learning to correlate randomly sampled features from images across an entire dataset. In this work, we build upon these advances by incorporating information about the structure of the scene into the training process through the use of depth information. We achieve this by (1) learning depth-feature correlation by spatially correlating the feature maps with the depth maps to induce knowledge about the structure of the scene and (2) exploiting farthest-point sampling to more effectively select relevant features by utilizing 3D sampling techniques on depth information of the scene. Finally, we demonstrate the effectiveness of our technical contributions through extensive experimentation and present significant improvements in performance across multiple benchmark datasets.

2. 

   TTT-KD: Test-Time Training for 3D Semantic Segmentation through Knowledge Distillation from Foundation Models

   Lisa Weijler, Muhammad Jehanzeb Mirza, Leon Sick, Can Ekkazan, and Pedro Hermosilla

   International Conference on 3D Vision (3DV), Jun 2024

   arXiv

3. 

   Weakly Supervised Virus Capsid Detection with Image-Level Annotations in Electron Microscopy Images

   Hannah Kniesel, Leon Sick, Tristan Payer, Tim Bergner, Kavitha Sshaga Devan, and 4 more authors

   International Conference on Learning Representations (ICLR), Jun 2024

   HTML

2023

1. 

   Leveraging Self-Supervised Vision Transformers for Neural Transfer Function Design

   Dominik Engel, Leon Sick, and Timo Ropinski

   Transactions On Visualization and Computer Graphics (TVCG), Jun 2023

   arXiv