GenerationMania: Learning to Semantically Choreograph

  • Zhiyu Lin Georgia Institute of Technology
  • Kyle Xiao Georgia Institute of Technology
  • Mark Riedl Georgia Institute of Technology

Abstract

Beatmania is a rhythm action game where players must reproduce some of the sounds of a song by pressing specific controller buttons at the correct time. In this paper we investigate the use of deep neural networks to automatically create game stages—called charts—for arbitrary pieces of music. Our technique uses a multi-layer feed-forward network trained on sound sequence summary statistics to predict which sounds in the music are to be played by the player and which will play automatically. We use another neural network along with rules to determine which controls should be mapped to which sounds. We evaluated our system on the ability to reconstruct charts in a held-out test set, achieving an F1-score that significantly beats LSTM baselines.

Published
2019-10-08