Millions of people suffer from hand tremors. 7 to 10 million people worldwide live with Parkinson’s Disease. As many as 10 million Americans have essential tremor. Tremors can also be caused by multiple sclerosis, stroke, or as a side effect of prescription and non-prescription drugs.
Hand tremors can make it very difficult to type effectively on touch screen keyboards. While voice recognition and swiping keyboards are a useful substitute for some people, they don’t work for everyone and they aren’t suitable in all situations.
As touch screen devices become central to everyday life, it’s important that they remain accessible to people with hand tremors.
Steady Type is an on-screen keyboard designed for people with hand tremors. It incorporates several unique features that make it easier for people with hand tremors to type quickly and accurately, with less stress.
Typing with Hand Tremors
Typing on a standard touch screen keyboard can be difficult even for people without hand tremors.
The keys are small.
The keys are close together.
As soon as your finger touches the screen, you are committed to typing a letter; you cannot back out and lift your finger without typing a letter.
Correcting an error requires an additional interaction, which may itself be error-prone.
All of these difficulties are magnified for people with hand tremors. Additional factors add to these difficulties:
Many people have intention tremors. With an intention tremor, the act of attempting a focused goal-oriented movement increases the tremor. The tight focus needed to touch a small key makes the tremor worse and success less likely.
Many people with tremors find that they have less control when their hand and fingers are in the air and better control when they are touching something. Unfortunately, the design of standard keyboards requires accurate control before the finger touches the screen.
The process of typing with a hand tremor is stressful, and stress often exacerbates tremors, creating an unfortunate feedback loop.
Steady Type addresses all of these issues.
Steady Type is designed around a few simple principles:
We should provide flexibility about where the finger initially touches the screen and then focus on what happens once it is touching the screen.
Whenever possible we should allow the user to back out of actions without leaving a mistake that needs to be corrected through a second action. This avoids the frustrating spiral of errors leading to more errors.
We should make the hit targets as large and widely spaced as possible.
We can trade some speed for accuracy. Ultimately this will be faster and more satisfying.
We should do what we can to minimize the number of keystrokes needed to write words and sentences. When every keystroke requires effort we should make each one as productive as possible.
Steady Type uses a drag-and-drop action to type letters, rather than the tap-to-type action of standard keyboards or the swiping action of some newer keyboards.
(Ignore the placement of the punctuation/space row or the large gap between the rows of keys. We’ll explain those later.)
When you touch a key, a “target” appears.
To type the letter, drag your finger to the target and release.
If you place your finger on the wrong key, simply lift it off the keyboard without typing. Nothing will be typed and there is no error to correct. You can try again to type the right letter.
Use The Background
Steady Type leaves space between rows of keys. To type a letter, you do not have to touch a key directly. Instead you can place your finger on the background and then slide it to a key. Once you reach the key, the target appears. Continue the sliding motion until you reach the target. Then lift your finger to type the letter.
Place your finger on the background.
Slide upward to the key. When you reach the key the target appears.
Continue sliding until you reach the target. Then lift the finger to type the letter.
By giving the user a much larger hit area (the background) we increase the likelihood of the finger landing in the right place. The larger area reduces the effect of intention tremors, and also may reduce the stress that accompanies finger placement.
Note: The active hit rectangle of a key is actually smaller than the visual appearance of the key. This effectively makes the background larger, and makes it less likely that the user will accidentally touch a key they didn’t intend. We have implemented screen skins that don’t draw the key outlines; the goal is to decrease the urge to tap the keys directly, something which is hard and stressful for people with tremors.
Steady Type’s multi-step keystroke — select a key, slide to a target, release — allows the user to do more than just type a letter. The interface has room to provide more than one target. The user slides to the specific target to type the letter (or perform the command) that they want.
Basic punctuation marks are placed two-per-key. Touch the Period-Comma key and two targets will be shown.
Slide to the Period target and lift to type a period. Slide to the Comma target and lift to type a comma.
Several keys provide useful functionality through multiple targets. The Return key provides targets for inserting a single carriage return or two carriage returns. The Shift key provides targets for shift and shift-lock.
The delete key allows you to delete a single character, an entire word, or an entire sentence with one keystroke. This is a feature I wish I had on the standard iOS keyboard!
These are some of our first examples of allowing the user to accomplish more with fewer keystrokes.
Two Letters Per Key
For users who would benefit from more background area or larger keys we can place two letters on each key.
The user touches one of the dual-letter keys, and then slides to the letter they want to type.
Adding a second row of targets allows us to provide word completion.
Word completion targets appear above letter completion targets.
A single continuous finger motion moves the finger from the background, to the key, to the letter target, and then potentially to the desired word completion.
This continuous motion simplifies motor planning and provides word completion as a natural extension of typing letters. The user’s eye movement from the letter target to the word completion targets is also continuous.
By contrast, traditional word completion interfaces place the completion options above the keyboard; viewing and selecting a word completion on a standard keyboard interrupts the visual and motor planning used to type letters.
Note: We now see why the Steady Type keyboard has the punctuation / space row of keys at the top. iOS does not allow third-party keyboards to draw outside the keyboard rectangle, even temporarily. If we were able to display targets outside the keyboard rectangle on top of the application’s content view, we could support word completion and still place the function / space row of keys at the bottom. We would also have more flexibility to make the targets larger and spread them further apart.
Steady Type offers word completions one keystroke earlier than standard keyboard interfaces. This supports our goal of minimizing the number of keystrokes needed to type words and makes typing go more quickly.
For example, you see the completions for words that start with the letter “T” when you begin typing the letter “T” rather than after you have typed the letter.
Even with our very simple word prediction algorithm, most core vocabulary words can be typed with one or two keystrokes. It is common to have entire conversations without using more than three keystrokes to type any word. Word completion also inserts a space, so in general it is unusual to use the space key at all. Altogether these features make typing with Steady Type very efficient, and recapture much of the extra time needed for the multi-step keystroke.
Steady Type has several features that allow the user to alter a word after it has been typed by word completion. For example, you can add an apostrophe and the letter “s” via the Apostrophe-Quote key.
In this example, the user has typed the name “Mary”. By selecting the Apostrophe-Quote key, they can change it after the fact to “Mary’s”.
You can inflect words after the fact, adding “ed”, “ing”, “er”, “s”, and “ly”. You can turn words into related contractions, for example changing “can” to “can’t” or “does” to “doesn’t”.
Here the user has typed the word “Work” using a word completion. Note that several key labels have turned red. These are the keys that can be used to inflect the word that was just typed.
The user selects the “d” key. In addition to the letter target and the word completion targets, we have an inflection target.
The user selects the inflection target.
When the user lifts their finger, the word “Work” is changed to “Worked” and the insertion point is moved to allow them to type the next word.
By allowing inflections and contractions to be added after the fact, we decrease the frequency with which inflected variations of the same word are shown as completion targets. This lets Steady Type offer a greater number of distinct root words as completions, thereby increasing the likelihood that the user will be offered their desired word quickly. This feature is optional, but in our experience it is very valuable.
Note: inflected variations of words can still be included in the list of word completions, but they have a higher frequency threshold for inclusion.
Touch Smoothing averages the touch point location over a short window of time (between 0.1 and 2.0 seconds, as a user-selectable option). The smoothing creates a slight lag in the user interface, but in return the worst effects of tremor can be mitigated.
With standard touch tracking, the touch point follows every movement of the finger on the screen. This creates problems for people with tremors, as the touch point moves erratically from location to location, rapidly switching between touch targets on the screen. This makes it hard for the user to effect their intention.
With Touch Smoothing, the touch point is averaged over time. While the tremor may take the user’s finger away from the intended location temporarily, the touch point is maintained closer to the user’s intended location. In the animation, we see see that movement of the touchpoint is smooth, despite the erratic movement of the finger.
Touch Smoothing works well with the context of the Steady Type interface, in which targets are selected after the finger is on the screen, and visual feedback makes clear which target is active at any given time.
Touch Smoothing can be enabled through Settings/Experimental Features/Touch Smoothing.
Steady Type now supports larger targets. The user can now choose a target size that works best for the magnitude of their hand tremor.
Choosing a larger target sizes currently increases the size of the keyboard rectangle, limiting the amount of screen real estate available for applications. This would not be necessary if Stead Type were able to draw targets outside the keyboard rectangle.
Steady Type is a keyboard for people with hand tremors. It uses a drag and drop motion rather than tapping keys to type letters.
Users can back out if they select the wrong key without having to take another action to correct the error.
An expanded background gives users a large hit area when they initially touch the screen. That is the point at which their finger movement is likely to be least accurate.
Large targets provide a visual indicator that the user has selected the correct key. The size and spacing of the targets also makes them easier to select.
By providing multiple targets for a single key, we greatly increase the expressiveness of each keystroke, allowing the user to accomplish more with fewer keystrokes. A continuous motor plan and visual plan brings the user to whichever target is desired. They do not need to look aside and alter their motor plan to reach word completions. They can back out at any time by lifting their finger without leaving an error that needs to be corrected.
The inflection and contraction feature allows a greater number of root words to be presented as completions, thereby decreasing the number of keystrokes needed to type overall.
As noted above, several design improvements would be possible if we could draw outside the keyboard rectangle.
Targets could be substantially larger and spaced further apart. This could be a user controlled parameter. Update 4/29/19: This has now been implemented.
The space/punctuation row could be placed in its traditional location at the bottom of the keyboard.
Alternatively, the keyboard could be smaller (it is currently somewhat taller than the standard keyboard).
The current implementation uses a very simple word prediction algorithm. A better word prediction system would improve typing efficiency and offer more features.
Grammar awareness and other forms of context-sensitive word prediction.
User-specific vocabulary, for example based on search history.
Application-specific and document-specific vocabulary
We have not yet implemented support for Emojis or characters with diacritical marks. Several designs are possible.
We have not implemented any support for languages other than English. Ideographic languages would be particularly interesting because of Steady Type’s inherent support for multistage input.
We have not (and as a third-party cannot) implement support for telephone keypads. Testers have requested this feature.
We have implemented support for haptic feedback and voice feedback when entering/exiting keys and targets but haven’t done any user testing of those features.
Our implementation is not yet integrated with any other iOS accessibility features.
It should be possible to measure an individual’s tremor through some test screens, and automatically adjust the size of keys, targets and backgrounds accordingly. We haven’t done any work on this yet.
With some adjustments Steady Type might be helpful for people with other movement disorders. We haven’t explored this.
The drag-and-drop interaction model used by Steady Type could be applied to the entire iOS user interface, building on the infrastructure already in place for VoiceOver. Whereas VoiceOver selects and speaks a user interface element when it is tapped, Steady Touch would display a target when the interface element is touched. The user could complete the command by dragging to the target, or abandon the command by lifting their finger. This could make large portions of the iOS UI accessible to people with hand tremors. We have not done any work on this, but would love to.
Copyright 2019 Learning Touch. All rights reserved.