Posts by Collection

[Paper]

This study benchmarks Vision-Language Models (VLMs & VLLMs) on five ADL video tasks across 11 datasets, revealing their struggles with fine-grained action understanding. Despite their web-scale success, these models fall short on real-world, densely labeled, and long-video challenges.

[Paper] [Code]

Multi-scale Temporal Mamba adapts Mamba for action detection in long untrimmed videos by introducing Temporal Mamba (Temba) Blocks with dilated temporal modeling and a Temporal Mamba Fuser for multi-scale feature aggregation. It outperforms SOTA methods on long videos while being significantly more efficient.

[Paper] [Code]

SKI models integrate 3D skeletons into vision-language models using SkeletonCLIP, enabling improved generalization to unseen actions in ADL videos. They enhance robustness by not requiring skeleton data during inference and show effectiveness in zero-shot action recognition and video captioning.

[Paper] [Code]

LLAVIDAL, a Large Language Vision Model, incorporates 3D poses and relevant object trajectories to understand the intricate spatiotemporal relationships within ADLs.