Monday, Aug 23, 2021, 17:49 Software

iPadOS & Handwriting Recognition – How What Once Was Impossible Now Works

Veteran Mac users and Apple customers likely think of the Apple Newton first with respect to Apple and handwriting recognition. The Apple Newton was a product in the 90s designed to recognize handwritten text – however, the feature's reliability was questionable, as the technology available at the time was too limited to truly bring the feature to fruition. Quite a bit of time passed until Apple began to tinker with the possibility again and released a system with support for handwriting input. In fact, it wasn't until recently, thanks to iOS 14 and the Apple Pencil which allowed users to write in fields as opposed to a restriction to typing. The system automatically converts the handwriting into text and in terms of the technology supporting the feature – it's rather complicated.

Machine Learning & A Large Database
Apple's Senior Vice President of Software Engineering Craig Federighi had an interesting interview with Popular Mechanics last year concerning the development of the Apple Pencil and the advances made in the field of handwriting recognition. In comparison to earlier models, current systems with support for the feature can do more than simply recognize common shapes. In the case of more unusual handwriting that deviated from the norm, recognition was not possible in previous systems. Nowadays, such features rely on the support of complicated algorithms based on data from various handwriting samples – in other words, a classic example of machine learning.



Locally Analyzed & Executed
It's not enough to simply analyze handwriting samples to recognize text, the formation of the writing sample itself also has to be taken into account. It's not until data from hand movements, the resulting handwriting, and a dictionary system are combined that a reliable prediction or "recognition" is possible. The evaluation occurs directly on the local device – now possible thanks to the performance capabilities of a modern tablet, in direct contrast to systems in the 90s. Everything takes place in real-time and during the actual writing, the tablet begins to make predictions as to how the movement will continue.

More articles you might enjoy to read: