The Gyrox
The Gyrox is a digital controller that uses inertial movement to control media using MIDI. The current version is an acrylic cube with a slot to insert a standard smartphone. The Gyrox may be used to control music and visuals through a movement based interaction.
The shape of the Gyrox suggests an entirely different set of gestures than that of an unmodified smartphone. Due to the size of its sides, it can now be placed on a table, and rotated orthographically in respect to gravity, something which cannot be done easily with the edges of a smartphone. This leads to the possibility of utilizing different axis as channels for different sounds, distinguished for the user by different colored squares.
Software In Use
The Gyrox uses 4 pieces of software that communicate data through Open Sound Control (OSC) and MIDI data. The accelerometer and gyroscope data is first captured using the smartphone app GyrOSC to send OSC data to Max/MSP (data and signal processing software) over wifi. This signal is then sent via OSC to Nick Gillian's Gesture Recognition Toolkit (GRT) for orientation classification. This classification is then sent back to Max/MSP vis OSC. The gyroscope and accelerometer data is processed in conjunction with the classification provided by GRT to package MIDI signals corresponding to various positions and movement of the cube. Finally, these MIDI signals are received by Ableton Live for sound generation. Note that any software dealing with MIDI could be used for either sound or video manipulation, including MAX/MSP itself.
Movement Classification
Raw data from both the accelerometer and gyroscope are both sent to the Gesture Recognition Toolkit (GRT) for classification. GRT is a program that works with both gesture classification and regression. It uses machine learning, a body of statistical methods that allows software to draw from example data to classify streams of raw data. Gyrox uses 6 classes to identity which surface of the cube is upright. All 3 channels of both the gyroscope and accelerometer data were sent to GRT, however only the gyroscope data is necessary for classifying the orientation. The acceleration data was sent for prototyping purposes, and doesn't effect the orientation classification. For each desired class data was recorded as the cube was moved both on a flat surface and in the air with the desired surface up. After data was captured GRT was “trained” for the data set. At first, the data was relatively effective, though there were some points where GRT would return the incorrect identifier. To fix this more data was recorded, paying attention to the sides that caused recognition issues.
Processing Movement Data
A flow chart of the communications between different software.
Max/MSP uses classifiers received from GRT to gate the incoming data from GryOSC into six streams, one for each side of the Gyrox. The classification is used to send an “on” signal corresponding to each side, as well as allowing MIDI mappings to be organized into six sets. For each side, the gyroscope data is processed slightly differently based on what readings that orientation is returning. Generally one axis provides rotation, while the other two are used to determine when the Gyrox is placed on the table or in the air. The rotational data was directly mapped to the outgoing MIDI data. Because one could rotate the box continuously in one direction, two MIDI signals are mapped to each rotational movement, creating a crossfade between the two. The user can only map one if they would only like one control signal to be modified per side, or two if they would like to cross face parameters.
When the Gyrox is placed on a flat surface, one or two of the gyroscope channels remain relatively constant regardless of rotational orientation. When removed from the table by a user, these channels become less constant. This aspect was used to determine when the cube was placed on the table, as well as provide an additional MIDI signal that accounts for the amount of tilt while the Gyrox is in the air. In it's current version, each side must be calibrated to capture the resting state of the gyroscope in each orientation.