Introduction

This project was part of a course on neuro-inspired engineering. The course explored this topic from two angles: how neuroscience and the brain inspired engineering solutions (e.g. AI), and how engineering can support people who have lost or impaired sensory abilities (e.g. vibrotactile motors to warn visually impaired people about obstacles).

Devices like vibrotactile motors, electromyography (EMG) devices, and Inertial Measurement Units (UMIs) are non-invasive, readily available, affordable, and easy to set up. At the same time, given some creativity, they can also be very useful solutions to aid people with sensory or motor impairments. With this perspective in mind, the course organised a challenge for the students to develop creative solutions with these devices.

In the challenge, every team was given one IMU, one EMG device, two Arduino microcontroller boards, and four vibrotactile motors. The goal was to reach the highest Information Transfer Rate (ITR), which measures how efficiently information can be transmitted. This was measured by transferring 200 numbers (0-9) over the system as quickly (and accurately) as possible. It was up to us to determine how to maximise the ITR.

Execution

The task came down to finding an approach to encoding and decoding the information that is as fast and robust as possible. To this end, we decided to encode the numbers 1 to 8 by making radial movements outward from a central resting point. 0 was encoded by drawing a circular motion with the IMU, and 9 was encoded by lifting the IMU. This makes for very simple and short movements, resulting in fast, robust encoding. While not as intuitive as drawing the digits themselves, the method was still easy to learn. In addition, no machine learning was required, which meant that the system was immediately usable.

The EMG device was not directly used for encoding, in part because it couldn't be used as effectively as the IMU. However, because we can see whether we encoded the right number on the screen, we decided to use it as a correction tool. When a number was encoded incorrectly, the user could tense the muscle to remove it.

Our first idea for decoding the signals was to place the vibrotactile motors in a square and imagine the vibrating motors as an edge, which could then be mapped to a number. However, we found it much easier and faster to see which motors are vibrating than to feel it. The decoding scheme resembled binary code. This was not necessarily intuitive, but was fairly straightforward to learn and had a good accuracy.

Results

In order to get to the final system, we tested multiple variations, two of which stood out. First, ITR is a measurement of both speed and accuracy. So although we found that the EMG device worked very well for correction, it slowed us down enough to reduce the overall ITR. The second finding that stood out to us was that the vibrotactile motors were clearly meant to be used by attaching them to the skin. However, in this challenge, it was not an effective solution. Instead, it was far more effective to see which motors were vibrating. In the end, the maximum ITR that we measured was 107.2 bits per minute. If we prioritised accuracy over speed, we achieved 99.5% accuracy in encoding and 90% in decoding.