Publications

[Paper]

TimeProVe uses lightweight modules to propose candidate answers paired with temporally localized evidence windows. Then it verifies only the most promising candidates with a VLM, drastically reducing the number of expensive calls while preserving temporally grounded reasoning.

[Paper]

UNIEGO shows that egocentric video models need not be limited by what a wearable camera directly observes: through proxy-mediated distillation, a single ego-only encoder can absorb complementary knowledge from exocentric views, depth, skeletons, and foundation models. By converting heterogeneous teachers into a unified proxy space and selectively distilling only reliable supervision, UNIEGO delivers stronger representations across action recognition, retrieval, and temporal segmentation while requiring only egocentric RGB video at inference.

[Paper] [Code]

MS-Temba adapts Mamba-based state-space modeling to Temporal Action Detection by introducing dilated multi-scale SSMs that capture both fine-grained and long-range temporal dynamics in long, untrimmed videos. Through scale-aware supervision and a dedicated multi-scale fusion module, it delivers precise localization of densely overlapping actions and generalizes effectively to long-form Video Summarization.

[Paper]

3D Guided Diffusion for Face Swapping

[Paper] [Code]

LLAVIDAL, a Large Language Vision Model, incorporates 3D poses and relevant object trajectories to understand the intricate spatiotemporal relationships within ADLs.

[Paper] [Code]

SKI models integrate 3D skeletons into vision-language models using SkeletonCLIP, enabling improved generalization to unseen actions in ADL videos. They enhance robustness by not requiring skeleton data during inference and show effectiveness in zero-shot action recognition and video captioning.

[Paper]

This study benchmarks Vision-Language Models (VLMs & VLLMs) on five ADL video tasks across 11 datasets, revealing their struggles with fine-grained action understanding. Despite their web-scale success, these models fall short on real-world, densely labeled, and long-video challenges.