Abstract

Visual-based human action analysis is an important research topic in the field of computer vision, and has great application prospect in sports performance analysis. Currently available 3D action analysis datasets have a number of limitations in sports application, including the lack of special sports actions, distinct class or score labels and variety of samples. Existing researches mainly use various special RGB videos for sports action analysis, but analysis with 2D features is less effective than 3D representation. In this paper, we introduce a new 3D yoga pose dataset (3D-Yoga) with more than 3,792 action samples and 16,668 RGB-D key frames, collected from 22 subjects performing 117 kinds of yoga poses with two RGB-D cameras. We have reconstructed 3D yoga poses with sparse multi-view data and carried out experiments with the proposed cascade two-stream adaptive graph convolutional neural network (Cascade 2S-AGCN) to recognize and assess these poses. Experimental results have shown the advantage of applying our 3D skeleton fusion and hierarchical analysis methods on 3D-Yoga, and the accuracy of Cascade 2S-AGCN outperforms the state-of-theart methods. The introduction of 3D-Yoga will enable the community to apply, develop and adapt various methods for visual-based sports activity analysis.

The pipeline of our proposed hierarchical yoga pose analysis method.

Data acquisition and examples

The capturing of yoga poses in 3D-Yoga. (a) Scene layout. (b) Samples of yoga pose. (c) Scene elements: 1 and 2 are two indoor scenes; A, B, C, D, E, and F are the textures of the wall; a, b, c, and d are the textures of yoga mates. (d) Proportions of three light types. (e) Proportions of four cloth types.

Hierarchical organization for yoga poses

3D-Yoga dataset contains 117 categories of yoga poses performed by 22 subjects in various indoor environments.

3D human skeleton data fusion

Three-level cascaded 2S-AGCN network

The framework constructed by using 2S-AGCN models and fully connected layers with Dropout and ReLU. The trunk network and branches of Cascade 2S-AGCN are trained together. The input is the skeleton data, and the outputs of each level are the predicted results of Classification I, Classification II, and completion score respectively.