Home » TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition

TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition

In this work, we demonstrate a strong baseline two-stream ConvNet using ResNet-101. We use this baseline to thoroughly examine the use of both RNNs and Temporal-ConvNets for extracting spatiotemporal information. Building upon our experimental results, we then propose and investigate two different networks to further integrate spatiotemporal information: 1) temporal segment RNN and 2) Inception-style Temporal-ConvNet.

Our analysis identifies specific limitations for each method that could form the basis of future work. Our experimental results on UCF101 and HMDB51 datasets achieve state-of-the-art performances, 94.1% and 69.0%, respectively, without requiring extensive temporal augmentation.
[GitHub page]

https://github.com/chihyaoma/Activity-Recognition-with-CNN-and-RNN

[Related Publications]

C. Ma, M. Chen, Z. Kira and G. AlRegib,¬†“TS-LSTM and Temporal-Inception: Exploiting Spatiotemporal Dynamics for Activity Recognition,” arXiv preprint arXiv:1703.10667, 2017.
[PDF] [Bib] [Code]

%d bloggers like this: