Wichita State team developing app to help blind and visually impaired readers

Imagine opening the newspaper and seeing the Sunday funnies replaced with just a paragraph of what characters were saying and no pictures.

That’s the reality for the nearly 70,000 Kansans with some form of visual disability. Current approaches to making comics and graphic novels accessible rely on converting the written words into audio, ignoring all of the visual information contained in the strips.

Now, a Wichita State University professor and one of his former students are trying to change that with an app.

“We both had students with visual disabilities in our class, and we were both teaching graphic novels. And so, everything that was happening to translate these or make them accessible wasn’t, didn’t work with what we were trying to teach in a class,” Aaron Rodriguez, a WSU graduate who is now pursuing a doctorate at Florida State University, said. “We’re looking at comics beyond narratives. We wanted to see the actual composition.”

“There wasn’t anything out there that was really addressing this issue,” Darren DeFrain, an English professor at WSU, said. “We have a bigger and bigger mandate at public institutions to make sure that everything we teach is fully accessible. If we’re not able to provide that, that causes a problem for the students and I really wanted to make sure that we were doing what we could to address that.”

The app they are developing, Vizling, will eventually make comics and graphic novels accessible to everyone, not just students.

“There’s not a lot that exists right now,” Michael Lang, the director of Talking Books Service at the State Library of Kansas, said. “I think anything towards accessibility in this level would be great.”

How Vizling makes comic books more accessible

Creating a version of a graphic novel that blind and visually impaired people can read is a time consuming process. DeFrain estimates that it can take 100 hours of work just to make the audio track, the typical way of making a graphic novel accessible.

Even then, much of the information is missing from the version of the text the student receives.

“One of the things that I’ve really emphasized in my class and most people that teach graphic novels emphasize is the importance of seeing how all the elements, visual elements combine with the textural elements,” DeFrain said. “If you have someone explaining to you what’s happening, you’re not really getting the full idea of what the artist is trying to convey.”

To make an accessible version that captures what the artist is trying to convey would require someone to “almost recreate the work” according to Lang.

That’s exactly what DeFrain and Rodriguez are currently doing.

Their app will include the standard audio version of the graphic novel but also make use of haptics and vibrations that are already common in smartphones and tablets to allow readers to freely move between panels.

“As you’re dragging your finger across the page, while you’re in one panel, the output is only the contents of that panel, as soon as you drag your finger over to a new panel, the panel boundary gives you that vibratory response,” Rodriguez said. “As your finger enters into the new panel, it’s reading then what is occurring in that panel.”

In addition, readers can touch anywhere on the page and hear what is there, allowing them to figure out the exact composition of the page and even search for similar objects or characters across books in their library.

The prototype of the app has already been built and DeFrain and Rodriguez are planning to start testing the app with blind and visually impaired readers this fall.

What’s next for the Vizling app

Longer term, DeFrain and Rodriguez hope to make an online platform where anyone can access the accessible versions of graphic novels as well as extend the technology to other visual documents as well. Maps could be one area of intrigue.

“It’s one thing to tell someone yeah there’s a river there and it goes through these counties but to actually touch a surface and get a sense of tracing how that river flows and what counties it cuts through and those kinds of things could be really useful,” DeFrain said.

The team also hopes to develop a drag-and-drop tool so that authors can create accessible versions of their own work, allowing everyone to enjoy their work.

“I would emphasize that this is something that blind, visually impaired, and fully sighted users could really get a lot out of,” DeFrain said. “More and more people are accessing multi-modal texts in online formats all the time, and this is something that I think in terms of search ability, research ability and just general use, that people could really find themselves using pretty regularly.”

To learn more about Vizling, DeFrain and Rodriguez created a comic about their app, which you can read on the Vizling webpage.

This story was originally published July 10, 2021 at 4:03 AM.