Introduction

We present a video model capable of simulating human movement within a given environment by assuming the parameters of a virtual avatar. Our model, a diffusion transformer applied to the human motion domain, features two key design choices: predicting the sample instead of noise in each diffusion step, and ingesting entire videos to predict complete motion sequences. We focus on rock climbing environments because they are controlled, single-agent, and static, yet encompass many complex biomechanical interactions found in various real-world scenarios. We aptly name our model SABR-CLIMB. Our results indicate that training a virtual avatar generation model on real-world environment videos, leveraging substantial computational resources, and enhancing dataset size and quality are promising strategies for developing general-purpose virtual avatars capable of navigating the world from a human perspective without any reward design or skill primitives.