Hundreds of millions of people worldwide suffer from hand tremors. As smartphones and tablets become central to everyday life, it’s essential that they remain accessible to these users.

Keyboards — with lots of small controls placed closely together — present a particular challenge to people with hand tremors. While voice input provides an alternative, it isn’t suitable for all situations or all people.

Steady Type is a keyboard designed for people with hand tremors. It allows people with hand tremors — even severe hand tremors — to type accurately and relatively quickly. It provides a solution where none currently exists.

Note: iOS places restrictions on third-party keyboards that limit the ability of Steady Type to provide a full solution to people with hand tremors.

  • They cannot be used to enter passwords.

  • They cannot be used for telephone keypads.

  • They cannot include the iOS voice-to-text control.

  • They can be blocked by apps, and Apple suggests that finance and healthcare-related apps block third-party keyboards.

  • They cannot draw outside the main keyboard rectangle, even temporarily. (This restriction limits Steady Type’s ability to generate large temporary controls that are essential for people with hand tremors.)

In addition, a third-party solution would reach only a tiny fraction of the users who could benefit from Steady Type.

Executive Summary

Hand Tremors

Millions of people suffer from hand tremors. 7 to 10 million people worldwide live with Parkinson’s Disease. As many as 10 million Americans have essential tremor. Tremors can also be caused by multiple sclerosis, stroke, head injury, or as a side effect of prescription and non-prescription drugs.

Hand tremors can make it very difficult to type effectively on touchscreen keyboards. While voice recognition and swiping keyboards are a useful substitute for some people, they don’t work for everyone and they aren’t suitable in all situations.

The existing touch accommodations provided by iOS and Android help some users, but many others still can’t use these devices, even with the features turned on.

As touchscreen devices become central to everyday life, it’s important that they remain accessible to people with hand tremors.

Steady Type is an on-screen keyboard designed for people with hand tremors. It incorporates several unique features that make it easier for people with hand tremors to type quickly and accurately, with less stress. The same principles underlying the design of Steady Type could be used to make the full touchscreen interface as well as much of the web accessible to people with hand tremors.


Typing with Hand Tremors

Typing on a standard touch screen keyboard can be difficult even for people without hand tremors.

  • The keys are small.

  • The keys are close together.

  • As soon as your finger touches the screen, you are committed to typing a letter; you cannot back out and lift your finger without typing a letter.

  • Correcting an error requires an additional interaction, which may itself be error-prone.

All of these difficulties are magnified for people with hand tremors. Additional factors add to these difficulties:

  • Many people have intention tremors. With an intention tremor, the act of attempting a focused goal-oriented movement increases the tremor. The tight focus needed to touch a small key makes the tremor worse and success less likely.

  • Many people with tremors find that they have less control when their hand and fingers are in the air and better control when they are touching something. Unfortunately, the design of standard keyboards requires accurate control before the finger touches the screen.

  • The process of typing with a hand tremor is stressful, and stress often exacerbates tremors, creating an unfortunate feedback loop.

Steady Type addresses all of these issues.


First Principles

Steady Type is designed around a few simple principles:

  • We should provide flexibility about where the finger initially touches the screen and then focus on what happens once it is touching the screen.

  • Whenever possible we should allow the user to back out of actions without leaving a mistake that needs to be corrected through a second action. This avoids the frustrating spiral of errors leading to more errors. Similarly, accidental touches and multi-touches should not degrade the experience.

  • We should make the hit targets as large and widely spaced as possible.

  • We can trade some speed for accuracy. Ultimately this will be faster and more satisfying.

  • We should do what we can to minimize the number of keystrokes needed to write words and sentences. Every keystroke requires effort; we should make each one as productive as possible.

  • We should be careful in the use of timing-dependent functionality. People with tremors and other movement disorders may need to move at their own pace.

  • We should provide the user with configuration options. Customizability improves accessibility.


Basic
Steady Type

Steady Type uses a drag-and-drop action to type letters, rather than the tap-to-type action of standard keyboards or the swiping action of some newer keyboards.

(Ignore the placement of the punctuation/space row or the large gap between the rows of keys. We’ll explain those later.)

01 Key Target.png

When you touch a key, a “target” appears. Keep touching. Don’t lift your finger yet. Note that the target is much larger than the original key. (The size of the target is a user-selectable option.) It is also specially isolated from other potential targets.

02 Drag to Target.png

To type the letter, drag your finger to the target and release.

03 Back Out.png

If you place your finger on the wrong key, simply lift it off the keyboard without typing. Nothing will be typed and there is no error to correct. You can try again to type the right letter.


Use The Background

Steady Type leaves space between rows of keys. To type a letter, you do not have to touch a key directly. Instead you can place your finger on the background and then slide it to a key. Once you reach the key, the target appears. Continue the sliding motion until you reach the target. Then lift your finger to type the letter.

04 Use the Background.png

Place your finger on the background.

05 Use the Background.png

Slide upward to the key. When you reach the key the target appears.

06 Use the Background.png

Continue sliding until you reach the target. Then lift the finger to type the letter.

By giving the user a much larger initial hit area (the background) we increase the likelihood of the finger landing in the right place. The larger area reduces the effect of intention tremors, and also may reduce the stress that accompanies finger placement.

Note: The active hit rectangle of a key is actually smaller than the visual appearance of the key. This effectively makes the background larger, and makes it less likely that the user will accidentally touch a key they didn’t intend. We have implemented screen skins that don’t draw the key outlines; the goal is to decrease the urge to tap the keys directly, something which is hard and stressful for people with tremors.


Multiple Targets

Steady Type’s multi-step keystroke — select a key, slide to a target, release — allows the user to do more than just type a letter. The interface has room to provide more than one target. The user slides to the specific target to type the letter (or perform the command) that they want.

07 Punctuation.png

Basic punctuation marks are placed two-per-key. Touch the Period-Comma key and two targets will be shown.

08 Punctuation.png

Slide to the Period target and lift to type a period. Slide to the Comma target and lift to type a comma.

Several keys provide useful functionality through multiple targets. The Return key provides targets for inserting a single carriage return or two carriage returns. The Shift key provides targets for shift and shift-lock.

09 Delete Key.png

The delete key allows you to delete a single character, an entire word, or an entire sentence with one keystroke. This is a feature I wish I had on every keyboard!

These are some of our first examples of allowing the user to accomplish more with fewer keystrokes.


Two Letters Per Key

For users who want larger keys and more background area we can place two letters on each key.

10.1 Two Letters Per Key.png

The user touches one of the dual-letter keys and a target is shown for each letter.

10.2 Two Letters Per Key.png

The user then slides their finger to the desired letter and lifts it when they see that the target has highlighted.


Word Completion

Adding a second row of targets allows us to provide word completion.

11 Word Completion.png

Word completion targets appear above letter completion targets.

12 Word Completion Yes.png

A single continuous finger motion moves the finger from the background, to the key, to the letter target, and then potentially to the desired word completion.

This continuous motion simplifies motor planning and provides word completion as a natural extension of typing letters. The user’s eye movement from the letter target to the word completion targets is also continuous.

By contrast, traditional word completion interfaces place the completion options above the keyboard; viewing and selecting a word completion on a standard keyboard interrupts the visual and motor planning used to type letters.

Note: We now see why the Steady Type keyboard has the punctuation / space row of keys at the top. Our initial implementation is in iOS, which does not allow third-party keyboards to draw outside the keyboard rectangle, even temporarily. If we were able to display targets outside the keyboard rectangle on top of the application’s content view, we could support word completion and still place the function / space row of keys at the bottom. We would also have more flexibility to make the targets larger and spread them further apart.


Efficiency

Steady Type offers word completions one keystroke earlier than standard keyboard interfaces. This supports our goal of minimizing the number of keystrokes needed to type words and makes typing go more quickly.

13 Word Completion T.png

For example, you see the completions for words that start with the letter “T” when you begin typing the letter “T” rather than after you have typed the letter.

Even with our very simple word prediction algorithm, most core vocabulary words can be typed with one or two keystrokes. It is common to have entire conversations without using more than three keystrokes to type any word. Word completion also inserts a space, so in general it is unusual to use the space key at all. Altogether these features make typing with Steady Type very efficient, and recapture much of the extra time needed for the multi-step keystroke.


Additional Features

Steady Type has several features that allow the user to alter a word after it has been inserted by word completion. For example, you can add an apostrophe and the letter “s” via the Apostrophe-Quote key.

14 Apostrophe Mary.png

In this example, the user has typed the name “Mary”. By selecting the Apostrophe-Quote key, they can change it after the fact to “Mary’s”.

You can also inflect words after the fact, adding “ed”, “ing”, “er”, “s”, and “ly”. You can turn words into related contractions, for example changing “can” to “can’t” or “does” to “doesn’t”.

15 Inflections.png

Here the user has typed the word “Work” using a word completion. Note that several key labels have turned red. These are the keys that can be used to inflect the word that was just typed.

16 Inflections.png

The user selects the “d” key. In addition to the letter target and the word completion targets, we have an inflection target.

17 Inflections.png

The user selects the inflection target.

18 Inflections.png

When the user lifts their finger, the word “Work” is changed to “Worked” and the insertion point is moved to allow them to type the next word.

By allowing inflections and contractions to be added after the fact, we decrease the need to show inflected variations of the same word as potential completions. This lets Steady Type offer a greater number of distinct root words as completions, thereby increasing the likelihood that the user will be offered their desired word quickly. This feature is optional, but in our experience it is very valuable.

Note: inflected variations of words are still sometimes included in the list of word completions, but they have a higher frequency threshold for inclusion.


Touch Smoothing

Touch Smoothing averages the touch point location over a short window of time (between 0.1 and 2.0 seconds, as a user-selectable option). The smoothing creates a slight lag in the user interface, but in return the worst effects of tremor can be mitigated.

Standard Tracking.gif

With standard touch tracking, the touch point follows every movement of the finger on the screen. This creates problems for people with tremors, as the touch point moves erratically from location to location, rapidly switching between touch targets on the screen. This makes it hard for the user to do what they want.

Touch Smoothing.gif

With Touch Smoothing, the touch point is averaged over time. While the tremor may take the user’s finger away from the intended location temporarily, the touch point is maintained closer to the user’s intended location. In the animation, we see see that movement of the touchpoint is smooth, despite the erratic movement of the finger.

Touch Smoothing works well with the context of the Steady Type interface, in which targets are selected after the finger is on the screen, and visual feedback makes clear which target is active at any given time.

Touch Smoothing is an optional feature, with a user-configurable smoothing window.


Summary

Steady Type is a keyboard for people with hand tremors. It uses a drag and drop motion rather than tapping keys to type letters.

Users can back out if they select the wrong key without having to take another action to correct the error.

An expanded background gives users a large hit area when they initially touch the screen. That is when their finger movement is likely to be least accurate.

Large targets provide a visual indicator that the user has selected the correct key. The size and spacing of the targets also makes them easier to select.

By providing multiple targets for a single key, we greatly increase the expressiveness of each keystroke, allowing the user to accomplish more with fewer keystrokes. A continuous motor plan and visual plan brings the user to whichever target is desired. They do not need to look aside and alter their motor plan to reach word completions. They can back out at any time by lifting their finger without leaving an error that needs to be corrected.

The inflection and contraction feature allows a greater number of root words to be presented as completions, thereby decreasing the number of keystrokes needed to type overall.


Future Work

We do not yet have an Android version of Steady Type. To reach the greatest number of people, we should clearly have one.

As noted above, several design improvements would be possible if we could draw outside the keyboard rectangle.

  • Targets could be substantially larger and spaced further apart.

  • The space/punctuation row could be placed in its traditional location at the bottom of the keyboard.

  • Alternatively, the keyboard could be smaller (it is currently somewhat taller than the standard keyboard).

The current implementation uses a very simple word prediction algorithm. A better word prediction system would improve typing efficiency and offer more features.

  • Multi-word completions.

  • Spelling correction as part of the word prediction algorithm.

  • Grammar awareness and other forms of context-sensitive word prediction.

  • User-specific vocabulary, for example based on search history.

  • Application-specific and document-specific vocabulary.

We have not yet implemented support for Emojis or characters with diacritical marks. Several designs are possible.

We have not implemented any support for languages other than English. Ideographic languages would be particularly interesting because of Steady Type’s inherent support for multistage input.

We have not implement support for telephone keypads. Testers have requested this feature.

We have implemented support for haptic feedback and voice feedback when entering/exiting keys and targets but haven’t yet done any user testing of those features.

With some adjustments Steady Type might be helpful for people with other movement disorders. We haven’t explored this.

The drag-and-drop interaction model used by Steady Type could be applied to an entire touchscreen user interface, building on the infrastructure and APIs already in place for other accessibility features such as VoiceOver. For example, VoiceOver on iOS selects and speaks a user interface element when it is tapped. In the same context Steady Touch would display a target when the interface element is touched. The user could complete the command by dragging to the target, or abandon the command by lifting their finger. This could make large portions of a touchscreen UI accessible to people with hand tremors.


 

Copyright 2019 Learning Touch. All rights reserved.