Express Activity 3: Exploring Applications on the Monarch

[adresselhaus]

Now that you’ve explored different applications share your thoughts in a discussion post.

Discussion Prompt:

Which application(s) did you explore? Describe your experience and what you learned. How do you see this application being used by students or clients in CIE settings? Think about how these tools improve independence, efficiency, or workplace readiness. What challenges might arise when introducing these applications? How would you help students or clients navigate these barriers?

Optional Extension:

If you’ve used other refreshable braille displays or screen readers, how does the Monarch’s application suite compare? Would you recommend it as a primary tool for students or professionals?

[Ronit_Mazzoni]

I have explored all apps on the monarch now, including the chess game. Given that the monarch can open PDF documents, I wish there was a dedicated PDF reader. That would make it a lot more professional seeming, rather than leaning toward the student market. Although the keyword application can open PDFs, it has to reformat the PDF to be readable. I’d like to be able to open the PDF (including password protected ones) more easily without having to open it in the word processor. It seems a bit redundant to have a keyBRF and KeyWord application, considering you can create BRF documents in either. The KeyBRF I know is more meant as a notetaker which is what I think I’d use most, but I’d personally rather have a word processor that can save as BRF as well as a separate PDF reader. I have also tried adding my work email to the KeyMail app but my workplace has not allowed the app yet, so I am not sure I will be able to access my work email as I was hoping to. I am able to access my personal email though which has been very smooth so far. In terms of what students or people seeking employment would want, I do think that a PDF reader would be wanted by professionals. Also, being able to read powerpoint slides would be a plus. I think the suite of applications currently available is perfect for a high school or college student.

[Jason_White]

I explored the Web browser, the book reader, the word processor and, briefly, the file manager. I wish to emphasize at the outset that the Monarch is a very impressive accomplishment in both hardware and software. To keep my comments focused, however, I shall concentrate on the limitations that I encountered. On the positive side, the applications mostly performed as documented.

Web Browsing

I retrieved and attempted to read a draft document which is under development by the W3C, and which was mostly written by a colleague in the Task Force that I co-facilitate. I was able to navigate by heading, but panning the display to read paragraphs of text proved unreliable, as much of the text would be skipped and the next heading would appear at the top of the display rather than a continuation of the current paragraph being read. I wrote steps to reproduce the issue and submitted a support case last week.

The browser is based on Chromium (excellent for standards support) and has the potential to allow students or professionals to interact with Web-based applications and Web sites on the Monarch. Thus it’s a highly valuable feature.

Book and Magazine Reading

Accessing my Bookshare subscription and reading a book in Victor Reader was easy. Then I discovered a problem. Most of the books I read include footnotes, endnotes, or citations and a bibliography. It is very desirable to be able to navigate quickly from a footnote or endnote number in the text to the corresponding note, and then return to the main text to continue reading. The same holds for citations associated with references in a bibliography.

The book I downloaded to the Monarch included endnotes. In Bookshare Web Reader, the note numbers are presented as links that lead to the endnote text, and there are links from the notes back to the main text. This suggests the markup in the book is correct. On the Monarch, however, using the point and click gesture in an attempt to navigate from a note number in the text to the note at the end of the book simply moved the cursor to the note number but did not navigate. Nothing in the User Guide suggests navigation to footnotes/endnotes has been implemented. I expect this to be an issue for, e.g., humanities students and law students at first-year university level or above, and possibly also for social science students. Professionals in research fields are also affected.

I also discovered that the magazine support isn’t designed for the task of searching for an article by title and reading it. While I could download issues of the magazine I wanted via an NFB Newsline subscription, there was no search functionality, and only the most recent two magazine issues were available, whereas the article I sought was in an earlier issue from last month.

Note also that the DAISY Consortium is currently developing requirements for accessible book/document reading applications, which address the navigational concerns. See pthis draft document](DAISY Reading Apps User Requirements) for details, and the list of open issues.

File Management

I’m accustomed to using the Linux or macOS command line interface for any file management task beyond the basics, so naturally, the file manager on the Monarch was going to be constraining. Nevertheless, it supports all of the basic functions one would expect. I noticed the lack of sorting operations (e.g., by last modification time, size, etc.) of the kind available in Linux, macOS and Windows file managers. Perhaps more importantly, it isn’t possible to access files over a network (e.g., using NFS, SMB, or cloud storage providers). I have worked in an environment in which files were expected to be saved directly to cloud storage and accessed on demand from whatever desktop or mobile device one happened to be using. Although one could use a Web interface for this, it’s additional work compared with having the storage mounted on the device one is using to read and edit documents.

The Monarch supports transfer of files from computers via USB, as documented in the User Guide, but my attempts to make this work have been unsuccessful so far. I can’t find the option to enable USB file transfer, which is supposed to be in Android Settings → Connected Devices. Online resources suggest that on Android devices, the option can appear in the notification area, but I couldn’t find it there either. It’s possible that my computer is at fault, as it’s rather old now and I don’t recall having used its USB-C port before. The newer machine is a MacBook Pro, which doesn’t support file transfer from Android devices by default, although installing Media Transfer Protocol (MTP) tools might solve this problem.

Text Editing and Word Processing

I appreciate the ease of use of the braille navigation and editing commands in the word processor, including the cursor routing function. I think the Monarch’s word processing features could be used for taking notes or writing simple documents, but they aren’t well equipped for writing a research paper, book chapter or graduate-level thesis. For instance, in my own work, automatic citation and bibliography generation has proven invaluable. Headings, footnotes and other document structures are a necessity, and the formatting guidelines of a journal, publisher or academic department need to be followed. I tend to write using a markup format such as LaTeX, Markdown or HTML rather than in a word processor, and this could technically be done on the Monarch, but it doesn’t support features such as markup syntax checking, automatic text indentation, shortcuts for inserting markup, heading navigation, and so on, all of which are very useful.

For those who prefer word processors, the Monarch doesn’t offer a style-based word processor, although there are basic formatting functions - perhaps sufficient for secondary school-level work, but not for undergraduate or graduate writing projects. In the sciences, of course, inline and displayed equations, figures and tables are all essential features.

On the other hand, this word processor should be excellent for taking notes at a meeting, reading a handout at a seminar, or other tasks not involving complex document creation.

Additional Opportunities and Challenges

Obviously, the availability of a terminal mode on the Monarch will overcome the limitations noted above by allowing for the use of any desired application running on another device with a supported screen reader. However, taking both the Monarch and a laptop computer to different meetings, classes, etc., is bound to be inconvenient. If the functionality of an Android screen reader (using system accessibility APIs) were reliably available on the Monarch, it would then be possible to use a variety of accessible Android applications not specifically written for this device. I think such an approach would also help to overcome limitations of the custom-written applications by enabling more fully featured alternatives to be used on the Monarch itself, complementing the custom tools. The general shift toward Web applications in the workplace should also help.

Comparison

This is the well known problem of braille notetakers and similar products: with the specialized applications on these devices, you have the advantages of a user interface designed specifically for braille-based interaction, but you then lose much of the functionality typical of desktop operating systems and applications. Using a screen reader with a mobile or desktop operating system provides the desired application features, but the user interface isn’t optimized for a braille device. A small and dedicated group of software developers can’t possibly implement even a significant subset of the features that would be desirable in specialized applications. So I think a strategic approach will be needed in deciding what to support in specialized applications and how best to enable engagement with the wider world of Android applications and with screen readers running on other devices.

The Monarch’s custom applications are better than any I have used on other braille displays, by a wide margin. The Monarch is among a handful of devices at the forefront of braille technology today. Nothing I’ve written here should detract from an appreciation of how amazing this is.

[Deborah_Armstrong]

I agree with Ronit about the need for a professional PDF reader, especially ability to cope with password protected ones. Publishers tend to deliver eBook versions of their textbooks in PDFS which often have underlying text but can be semi-accessible. This is true whether it’s sold in the college bookstore as an eBook or acquired through disabled student services from the publisher.
PDF eBooks often have bookmarks and links, so the reader would need to be able to navigate with these, for example, enabling the student to go directly to chapter 5.
Another PDF issue which is also true for screen readers is that professors often assign pages to read and the poor student cannot easily locate them in the PDF. “Turn to page 254 then respond to the questions on page 339” is a typical in-class assignment.
Also I agree about the need to be able to read PowerPoint. Like Jason said though, if there’s a terminal mode, maybe college students will simply use Word, or PowerPoint or Outlook and not really bother with the built-in applications.
I agree with Jason that navigation in the web browser is a bit shaky, especially with longer pages and in-page links.
And I agree that links in bookshare books are not working. However, with the FS Reader and JAWS, the same thing occurs; you cannot jump to a footnote or an item in a book’s table of contents in a Daisy book with that tool either. Using an Epub reader on my PC I did have better results, so since the Monarch does support epub, navigating to links within a book should be easily possible.
The limitation of NFB Newsline only giving you the most recent articles is built in to Newsline itself, and probably cannot be changed by the Monarch. My solution is to set my subscription on my other devices to not expire old issues, and every few days I copy all the newsline content off my device to an archive drive. This way I can sort of find older articles, but I doubt the Monarch will ever be able to make this possible. Libraries’ databases are still the best source for finding older magazine and newspaper articles.
I was able to use MTP USB transfer to easily move files from my PC to the Monarch unlike Jason. But I have Windows 11 and several obsolete android phones that I practiced with first. My Fire tablet is equally finicky about file transfer, so it wasn’t my first rodeo. The big problem I have with the file manager is that it insists on copying all these empty Android folders on to a USB flash drive when you transfer files to and from a flash drive, as described in a previous post. Trying to figure out where you are in the file structure is confusing too; in a previous post, I suggested having the pathname always appear as the first line on the display.
The file manager on the Brailliant and NLS eReader, also made by Humanware is much easier to use I think and its interface should be copied, except of course for adding pathnames since there is more screen real estate to work with.
The file manager should also take care of importing any file in to Victor Reader if that’s the app you want to use with a file. There’s no reason to make the user interface less friendly by requiring an extra step for the user.
Regarding Jason’s comments that cloud storage should be supported is another reason for users to have direct access to Android apps. I have apps to do ssh, ftp, NFS, rsync, samba, etc. And of course google provides a google drive app, Microsoft a one-drive app, dropbox its own app … in college, professors do put files on the cloud for students to grab, and every professor has his own idea about which cloud storage he prefers.
I agree with Jason that the word processor is too simple for higher education requirements, but if a student cannot use a computer in college, then they aren’t going to succeed anyway these days. Many of my competent academic students, both those who are blind and those with learning disabilities prepare their first drafts on their phones or tablets, using either iOS or Android apps. Then they complete the formatting on a PC or Mac using Microsoft Word or Apple Pages. So I don’t see that having a simple word processor is truly a problem. I prepared minutes for several meetings this week in KeyWord, moved them to my PC where I cleaned up the formatting, put the tresurers’ report in to a nicely formatted table and it was all good.
Regarding having one editor for both simple note-taking and Word processing, that was what was done on the VarioUltra. It could easily translate between Braille and print though it didn’t have as many features as the Monarch or other Humanware devices’ keyword does. But one really good use for having KeyBRF is the ability to write Braille that cannot be translated, such as Grade 3, music notation, passages in another language intermingled with the primary language or your own personal shorthand.
For example, I’m working through a book on intermediate level Spanish and if I take notes some are in English and some in Spanish. So that I would do these notes in KeyBRF and not KeyWord, because if translated to print it will look like nonsense. For those reading this who do not know Braille, yes there are Spanish tables, but if you are working with multiple languages, you cannot switch tables mid-document with a simple editor.
Regarding Jason’s comment about running Android apps on the device, I agree. I wonder if I’d crash it if I tried side-loading apps. I do that on my Fire tablet all the time and it works fine, but I think this may be too special-purpose. It’s a pity, since education has loads of apps, and of course not all are accessible, but some definitely are.
I still need to try google docs in the browser since it is used extensively in education for all levels and also try filling out some forms. I’ll report on that when I do.
Jason writes:

This is the well known problem of braille notetakers and similar products:…
and I cannot agree more. I encourage my blind students to continue to use their Braille note-takers for just that, taking notes, reading books, but they must learn to use a computer. Specialized apps take you only so far, and regarding CIE, nobody will hire you if you cannot use a computer. Perhaps in a low-level customer service job you could get away with using a specialized device, but you could not get promoted if you had limited skill using a PC or Mac. Of course when you use a Braille device you can be more efficient with certain tasks, which is why they have value.
One workplace trend I’m struggling now with is the tendency during zoom meetings for attendees to collaborate in realtime using google suite or Office on the web. This occurs even if none of us are working remotely. It’s more accessible than everyone collaborating on a whiteboard but not by much. Unfortunately, my co-workers see it as accessible since I am a confident user of Office on the desktop so they don’t understand why I’m struggling.
I know this may not seem Monarch related but ideally, the Monarch could speed my ability to function in these situations. I’m in a meeting and someone pulls up a document and wants everyone to add comments and it’s a huge table. I spend half the meeting just trying to figure out where my comment is supposed to go. I am trying to listen to both my screen reader and people speaking in the meeting and panning frantically on my single line display to find where in this huge table I’m supposed to insert my thoughts. And if I press Tab in the wrong place, I’m now indenting a paragraph instead of navigating to a cell.
It’s even worse with Excel or Google sheets: the boss pops up a spreadsheet and wants us to all enter some data right now in the meeting! I’ve been asked multiple times if I have a learning disability, when it’s my technology that has a learning disability-- not me!
I also want to say that Keymath makes it very easy to keep track of a long list of calculations, something much harder to do in Excel with a screen reader. Though the “Mastering the Monarch” course demoed using it for graphing, which is valuable for students, in an employment situation where you must often calculate rapidly in real-time, it’s superb. Imagine giving a customer a price estimate on a complex wishlist for example, or totaling up expenses for a trip the boss is taking.
I want to explore the web browser more, looking at a variety of pages and web-based apps and online learning platforms. I also want to play with the chess app which I have not yet done.

[Jason_White]

I agree with your excellent points and thoughtful examples. I may take this up further in express activity 5, when I write my comments, but based on an informal survey it’s clear that the information technology environments at universities are increasingly resembling the workplace. Google Workspace and Microsoft 365 are becoming increasingly common for university students, and not only in the United States.

I also agree that the multiple expression support in KeyMath is valuable. When a student needs software that can compute derivatives, integrals, differential equations, etc., then they’ll need to turn to something else, as suggested in my comment on express activity 4.

I find collaborative, real-time editing difficult with a screen reader. The Monarch may be able to help with this, but I suspect it will remain hard, especially if other people are making numerous concurrent changes. Fortunately, I haven’t worked on projects that insist on editing documents during meetings. I always turn text to speech off and use a Braille display exclusively in online meetings, and the Monarch’s multiple lines could be an asset here.

[Jason_White]

I would like to take up one additional point: I agree that those moving into employment generally need to be able to use a computer. At least for some higher-level positions, though, they also need to have the skills in place to learn new software quickly and without assistance. This can be illustrated with the following scenarios. They’re all realistic, since they’re all loosely based on my own experience.

• You want to register for a professional conference and submit a presentation proposal. The conference organizers are using a Web application for this which is completely unfamiliar to you.
• You’ve been invited to participate in a cooperative project with other organizations. The project uses meeting and collaboration software that you’ve never encountered before. There’s a registration link in an invitational message in you’re email, and the project starts tomorrow…
• You want to read some work-related books. The books are hosted in an unfamiliar online reading platform.
• Your employer introduces a mandatory training program for everyone in your department. The training is delivered via software you’ve never heard of, developed by a company that is probably only known to people who are in the business of buying these tools on behalf of organizations, and of course there’s a deadline to complete the training…
• You arrive in a new workplace, enthusiastic about the career opportunity. Then you discover that your new employer is using productivity and collaboration tools that you’ve never experienced before. Perhaps the computer on your desk is running an unfamiliar operating system and screen reader. It’s time to learn fast…
• Your employer introduces new security software that you need to install and set up on your phone before the deadline.

Can the Monarch help with this problem? Perhaps having more of the user interface available at once on the display, instead of having to explore it by navigating to one control at a time would be an advantage in coming to terms with new software rapidly.

[Deborah_Armstrong]

Oh My! Everyone of those scenarios has happened to me in my various jobs. Unfamiliar web apps, employer-mandated training, strange reading platforms protected by DRM, collaboration tools used in realtime, strange interfaces to unfamiliar security software and instructions that tell you to click on the three dots, the blue E or the red star!

I think one issue with this is psychological. If a blind person is seen as competent, say because they can use keystrokes to navigate and format Word documents even more rapidly than sighted co-workers, then it is naturally assumed they will be equally efficient with any computer-based task. A previous supervisor, watching me work with a familiar website, could not understand why our in-house database app was giving me trouble and she insisted on believing I was simply malingering!

And trying to figure out these interfaces is so frustrating especially when you have other work responsibilities and deadlines!

We have some in-house database software built as a front-end on to Microsoft SQL server, that shows student data in grids which unfortunately a screen reader cannot handle with normal table navigation. Because I’m a paraprofessional, I’m expected to deal with data as often as I assist students. And as the end of the fiscal year approaches, I am mostly dealing with data.

I have to poke around the grid with arrow keys and sometimes get dropped to a different row or column because their keyboard navigation is inconsistent. I feel like a bird building a nest, grabbing one little stick of data at a time.

Also when Windows 7 came out, screen readers suddenly had inconsistent access to the mouse. If you’ve used Windows for a while, you know you could reliably navigate an entire active window with the JAWS mouse, or invisible cursor and “click” on anything you wanted. That ability disappeared, and now the information you get when moving the mouse with a screen reader is unreliable. I was told Microsoft limited the ability for third-party software to be able to move the mouse, but I don’t know more about this change. However it greatly limited our ability to work with applications that did not have full keyboard navigation.

And even when you do have full keyboard navigation you are limited to reading what has focus, when the surrounding text might be just as relevant. Of course you can ask a screen reader to read the entire window, often a morass of information, so it’s either the entire thing or the single control with focus.

Plus, screen readers don’t give you any spatial feel for screen layout.

Of course Mac users still can navigate and read everything onscreen, but many applications require Windows.

If the Monarch would be able to let me “see” more of the screen than I can with either the one-dimensional nature of speech or a single-line Braille display, that would be awesome.

[Deborah_Armstrong]

As someone who absolutely loves to read, I am quite happy with Victor Reader, so the below quibbles should be taken in the spirit intended.
I love-love having multiple lines of Braille. I have tried some cookbooks; can get to a recipe much faster. I have tried Science fiction; I am more deeply immersed in the story with more lines at my disposal because I can read rapidly with both hands. All the worlds created for example by S.M. Sterling were something I could previously only get lost in when listening to the book being read with speech. And I’m a fast Braille reader. But now it seems I’m nearly as fast on the Monarch as I am with hardcopy Braille. And because the epub books reflow, they work quite nicely in the Victor Reader, and the speed of being able to read them is helping me enjoy electronic Braille more than speech.
It’s a different story for BRF files. Though they import, I am not sure why reflow, which works so well on the NLS eReader wasn’t implemented.
Also the import feature is kind of klutzy. I have finally gotten a few chapters of some accessible math books in Word, using the equation editor created by another college. They were in zip files, and I couldn’t import them because that file format “wasn’t supported”.
This is plain silly. It is in fact the .zip format it is talking about, because after unzipping them on my PC and moving the unzipped files back to the Monarch, then using import again, it was all good.
But this thing can already unzip files, because epub is just zip with a different file extension!
And an ordinary non-tech users won’t know what that error message means.
The user shouldn’t even need to have to import really, as I said in a previous post, the device should auto-import as soon as you tap on a file, zipped or not.
And it should import directly from the file manager.
Basically the interface I envision is this: you locate a file and tap on it and a configuration setting determines whether it asks “Import in to Victor Reader? (Yes/No)” or it just does it.
And if you import from inside Victor Reader you are presented with the file manager interface and have to poke around and are often unsure where you are in the file system especially since it shows you all those empty and irrelevant android folders. I put all my stuff in a folder labeled Import, but many users don’t know how to create folders in an organized manner. And when you open an empty folder, the one-line message reads “no items” which could completely confuse a beginner. It should instead say:
“Path:
storage\import\mathBooks
This folder is empty.
Press the Space with E or the backspace to return to the previous folder.”
There’s plenty of screen real estate for that.
I spent some time with Treasure island but I’m not seeing that much difference with e-BRL, except that it is better formatted than BRF. I tried to jump to links in the table of contents which is part of the E-BRL standard but it didn’t work, the display didn’t move with point and click. I must be missing something here.
The Help screen for Victor Reader is the same as the help screen for KeyBRF. You have to do chord-M to get all the features.
The heading level navigation seems a little weird. I would have thought it would be better to type numbers for the heading levels, for example 3 for level 3, but it does work.
Victor wastes a line of precious screen real estate using a blank line to mark the start of a paragraph. A 2-space indent would be more efficient and fit more Braille onscreen.
I am very impressed with the flexibility of the bookshare support; especially reading lists, history and that a teacher can assign books to a student and have them see them right there in Victor reader, easily available to grab. It’s better support than any other app to date.
Well it’s the weekend so I’m going back to my science fiction – live long and prosper S.M. Sterling!

[esigmund]

Hey @Jason_White - our developers were able to reproduce the issue you documented in the web browsing section! Just wanted to report back that they are on it. Thanks!

[Ronit_Mazzoni]

I wanted to add some additional feedback, some of which has already been mentioned. I completely agree that while editing in KeyBRF, when you have reached the bottom of the display when writing large blocks of text and then the display must scroll, I’d love it if it could just scroll up from the bottom, allowing you to keep what you previously written in view, rather than completely giving you a blank display to work with once the first “screen” has filled. I always like to see what I’ve written on the previous line.
Secondly, in the file manager, when deleting a file, since there are keystrokes that you can press to do many functions, why is it not possible to type a “y” when asked if you are sure you want to delete or move a file? Even if the default was to put you on the “ok” button, rather than making you scroll to it or point and click, this would be so much faster. On other braille notetakers, you can simply type “y” and it would perform the delete or move. Not having this functionality makes things clunky and slow.
In addition, when putting the monarch to sleep temporarily, I have noticed that sometimes it’ll “wake up” all by itself and start talking. This is quite distracting and always surprising!
In the same line of thinking, it would be great if it had an automatic “go to sleep” after a certain amount of time has passed. I have accidentally left it on a few times and had the battery drain on me unexpectedly. Most other notetakers have the ability to go to sleep after some number of minutes of inactivity.

[Jason_White]

I was also wondering about an inactivity timeout for sleep mode. Perhaps it’s already somewhere in the Android settings. Without it, you have to remember to press the power button once quickly to enter sleep/suspend mode.

I noticed that the Monarch came out of sleep mode unexpectedly overnight while connected to my MacBook Pro via USB. I only discovered it the next morning.

[Deborah_Armstrong]

I keep thinking of more to comment on.
I did get the VisioDesk connected to the Monarch, and yep, people can see magnified images of the display. Now a low-vision tutor who doesn’t know Braille but who has a modern video magnifier should be able to assist a student with the Monarch.
I wanted to read a recipe with the Monarch wile preparing it so I hooked up one of those silicon waterproof rubberized USB keyboards so I could keep the Monarch safely in a plastic bag. And of course I used speech to read.
The keyboard worked great. These keyboards roll up, are cheap and used in factory situations. They would be great for schools because they are small and almost indestructible. One can be stored in a Monarch’s travel bag for a non-Braille user to have a keyboard on the go. I use mine to mainly read email on my PC while eating lunch, but if your hands aren’t clean enough to touch the Braille, or you don’t want to lug around a full-sized keyboard, these are great. I can imagine a child reading some craft directions in Braille, then connecting the keyboard, and while their hands are dirty, using it with speech to remind them of the next step in the process.
Epub is gradually becoming a popular format. Public libraries distribute rental books to patrons in ePub. Library databases like Gale and ebsco let users download articles and entire eBooks in epub as well. Kindle books are actually epub with a DRM wrapper, and DRM is the problem.
Because removing DRM doesn’t violate copyright if you are print-impaired, I have apps to do it for my personal use, and do it liberally. But these vast libraries of books will not be available to the Monarch user without access to Android apps like Adobe digital Editions or the Kindle or google play books apps. And all technical books required to advance in one’s career are accessed with one of these apps.
Now bookshare is great of course, but who knows if they will loose their federal funding and start charging all members for a yearly subscription. Who knows if bookshare will outlast the monarch? Many textbooks are just not available on bookshare, because publishers claim they are “born accessible” and must be read on a DRM-protected but usually web-based platform with an app for offline reading.
Access to Learning Ally, especially now they are promoting a new format that has full-text combined with audio would also be great for elementary grades. On my to-do list I need to try the Learning Ally web interface, but I think it might be embedded in an app.
In my job, procuring alt media is my main duty, and more and more, I get refusals from publishers because their books are already accessible, provided the student signs up for the eBook’s integrated lab. Testing with the web browser, I do see I can access many of these platforms such as vitalsource, RedShelf and Brytwave. So as long as the user can cope with a complex and mostly accessible interface, they can indeed read these DRM-protected textbooks on the Monarch.
So if DRM-protected books could be imported in to Victor reader, that could be truly awesome! The experience would be far easier and more convenient.
Victor reader also needs a way to read Daisy with images. Though most images won’t be comprehended with just the Monarch tactile display, if they can be fed to an AI for a description that would be helpful. I love the JAWS Picture-Smart and in combination with a Daisy reader that displays images it has been so helpful to see those images in books. Could access to google’s gemini somehow be integrated so that an image description could be generated when the image appears on the tactile display?
KeyWord or KeyBrf needs a way you can create columnar data. This was irrelevant with a single-line display but I’d like a way to make tables whose formatting I could actually see. On a PC I can use google sheets or Excel but I’m still seeing just one cel or one row or one column at a time, depending on the sophistication of the screen reader support.
Another cool game I’d like to see for teaching Braille would be a Simon Says. You would hear a character spoken out loud and need to find it on the display and point and click. As you got faster at reading Braille, and finding that hidden character, Simon would get faster too.
Sighted children do “word search” puzzles. They have a list of words and a list of letters and they draw lines from the letters in each word to the corresponding letters in the puzzle. I adapted some of these puzzles for Braille for a blind adult with developmental delays, and she uses crayons to draw lines to matching letters. A Simon game with the multi-line display is a way to make this activity accessible. Crossword puzzles could also be adapted for this display using point and click to draw lines and fill in letters. Scrabble for blind children could increase their vocabulary. And with a dictionary that’s already built in, it should be easy to write word game apps for the Monarch.
Another app idea is to have the equivalent of the scantron for taking multiple-choice tests. The test would be stored in the app, and the user would read the question and point and click on their choice.
And while we are dreaming, I wish the Monarch could run Python. I remember those heady days when the Braille 'N Speak actually had a BASIC interpreter and I wrote lots of little programs for mine. JAWS has a scripting language and WindowEyes used to have user-created apps. Today I code in Python for a hobby.
I’d write an app or two for the Monarch if it could run Python. You could have contests for the best user-created apps. There are also loads of open-source little python scripts. Or it could just have a little macro language, or maybe run Quarom. I know we want to get more blind kids in to coding.
I know this is all off-topic for Express Activity 3, and as you can tell, Express writing is not in my nature. But I love the potential of this device and I can’t help nattering on about it!

[Teisha_Gillespie]

I’ve had the opportunity to explore a handful of applications thus far.
The Word Processor has been tremendous at enabling efficiency. It’s been great in allowing me to take notes (with the screen-reader disabled) while being in a Zoom meeting without feeling overwhelmed by too many sounds or getting distracted by conflicting audio in my ear. Also, with the multi-line design, I’ve been able to quickly skim different documents to locate (or confirm) key information. Another benefit: the “switch document” experience is very user-friendly and allows me to quickly copy-and-paste info from one file to another, or easily validate info that’s located in a different doc. One drawback I’ve noticed, though, when editing a Word doc is that when I use the point-and-click to quickly move to a spot where I need to either make a revision or add more info, there’s quite a long delay in the cursor moving to the spot I need it to, and there’s been a few times where I’m pretty sure the cursor has froze, and so I use the “move to top” or “move to bottom” keyboard short-cut to shock it back to life. My device has been plenty charged in these scenarios, so I don’t think it’s a battery issue. Another challenge I’ve run into is trying to view a downloaded document with tables and/or screenshots. Screenshots that are part of a Word Doc don’t seem to show at all, and this includes screenshots of both diagrams as well as ones that are just text/numbers. The tables do display with all the text information transposed, but the display itself doesn’t feel very user-friendly. And again, this could be just me needing to learn more Computer Braille or other Braille symbols that are used to indicate headers, columns, and rows, and so I’d appreciate any insight others have with this.
I’ve also used the Email app and have logged into my Gmail successfully. Overall, the email experience is pretty simple and intuitive, but a drawback (and perhaps this is an Android thing and not a Monarch one), is that you can’t view emails in a conversation or thread view; you have to view each reply separately, which makes things a little tedious when there are multiple messages that are part of the same email thread.
I’ve briefly played around with the Internet browser, Ecosia. I really like the experience of being able to view multiple links at once, and the UX design of knowing when you are in a form field is really great. One thing I noticed when trying to use the Google search engine, is as I was typing in my search, I kept hearing from the screen-reader that there were X number of results available, but they literally were not showing for me. Perhaps this is a learning curve weakness on my end in needing to acquire more knowledge on navigation, but neither with the point-and-click or with the keyboard navigation strokes could I figure out how to view the auto-complete search results that Google was suggesting based on my initial search query. Google Drive is also a very common app I leverage at work (and even in my personal life), so I tried to view Google Drive using the web browser. If you rely solely on the tactile display without the screen-reader, I’d say you are probably okay. However, I often find it very convenient to use the screen-reader in combination to help with faster navigation. When I was using both the Braille and screen-reader, though, to scroll through my folders and files, it was difficult to know which folder/file was currently in focus because the screen-reader would only say “Google Drive Folder Graphic” or “Google Drive File Graphic.” I had to use the Braille display to read the actual name of the folder/file that was currently selected.
Challenges aside, I’m confident that each of these apps would enable grater equity for BLV professionals when it comes to workplace communication, collaboration, and research.

[Teisha_Gillespie]

Thank you so much for your post! I have not yet tried to connect my work email (which is also through Google), so it’s unfortunate that I may also run into challenges trying to do this. A huge pain-point for me, too, is not being able to open decks/slides/powerpoint documents. They are used so often in my organizations, and I was really looking forward to being able to view these without having to first convert them to PDFs; it’s not a great experience if you need to make edits. Lastly, I’ve noticed that you cannot interact with Excel or Google Sheet documents, which feels a bit odd since these are the primary applications for math-based workflows (accounting, analytics, graph visualization), as well as some project management workflows.

[Teisha_Gillespie]

Jason, I appreciate you raising the issues around cloud storage/real-time collaboration as well as document structure/formatting. I did find Keyword to be great for notetaking, but I do often structure my documents (especially those I plan to share out) with structured headings, bulleted lists, hyperlinks, highlighting, and so on, and i’m not currently confident I could do all these tasks with the current word processor. Additionally, I have not yet found an app version of Google Drive available on the Monarch, even though it’s an Android/Google-based device, and Google Drive is a very common cloud collaboration/storage tool I use on the daily for both work and personal life. I do fear that not being able to edit/save real-time documents to the cloud will become a huge barrier.

[Ronit_Mazzoni]

Totally agree about needing powerpoint viewing capabilities! I also have a huge need for viewing google sheets or google docs more successfully as these are hard to use with my screen reader. I consider myself to be a good screen reader user with JAWS but still struggle with google sheets and google docs online. This si becoming more used and I also supervise students who use this so I need to be able to review their work. Definitely math could be used in google sheets but also just spreadsheets in general.

[Jason_White]

Have you tried any of the Google applications with the Web browser that runs on the Monarch? Of course, the recently announced Braille terminal mode and screen reader support will give you access to your normal Web browsing environment on the Monarch also.

I’m only an occasional user of Google Workspace applications, so I haven’t experimented with them in the Monarch’s Ecosia browser. I have used the browser for other purposes and encountered some issues that have already been discussed here and taken up by technical support.

[Ronit_Mazzoni]

I also have had some issues with the web browser on the monarch so have not felt comfortable trying something like google sheets or docs where I might not be able to accurately read or accidentally edit something that I didn’t want to be. I am trying to become more comfortable with the web browser in general before I try something like that but I do find that it’s a bit clunky and I can’t always read pages accurately.

[Deborah_Armstrong]

I spent some time Tuesday mentoring several folks who needed to read gmail in outlook and it was a real struggle, because for some reason the setup was very tricky. This is part of my volunteer role as education chair for the ACB affiliate BITS (Blind information technology specialists).
On the other hand, connecting gmail to Keymail was trivial. So the Monarch flies high here.
Right now BITS is teaching a class on AI using google classroom, so I’m still working through it using the Monarch browser. Again though google classroom is confusing for the screen reader user, having multiple lines of Braille does make it easier to work with.
I also tried the Canvas course management system using the browser and found that less complex. Our college has a sample Canvas course that’s free for all our students that teaches them how Canvas works and how to take exams, participate in discussions, submit assignments and read course materials. Working with edit fields is still a little strange but I’ll report more when I’ve practiced more with those.
The other project I’m working on is to write a Python program to reflow Braille formatted for one page size for a Braille display of any size. I wrote a simple script in a Unix tool called sed, which does this in a primitive way and when I run BRF files through it, they are already easier to read on the Monarch, but I want to do something more complete and refined. Coding for me though is only a hobby so getting things done there is slow.

[Deborah_Armstrong]

I wrote this up a week ago but just got around to posting it now. I’ve since become a bit more comfortable with Ecosia but it’ still very strange and inconsistent.
So I’m really trying to like Ecosia.
I’d read the documentation two months ago and decided I’d try using it first before referring to the documentation again.
After all, the main point is to view web pages and so the interface should be simple, and intuitive, right?
But in my opinion, it is not.
First, I got some home page all about climate change with nothing on it about the Monarch. And there were links to videos about how much women have contributed to the world – worthy material, but unrelated to APH or the Monarch, plus the Monarch couldn’t even play those videos! “When women thrive, the planet thrives” is a great video, I found it with my PC, but is the Ecosia blog really the first thing a new Monarch user needs to see?
There was a list of common websites like Facebook and MSNBC, which is good, but mixed in with it were sites like Instagram, which seems like it would be of little use to a Braille user. And google was near the end of this list, when it should probably be the first web page listed. After all it’s an android device, and students use google docs and google classroom. Also with these adult sites it’s too easy for a kid to stumble in to material their parents might not want them reading!
I noticed too that the browser interface appeared near the end of a page, but there was nothing to actually distinguish it from the text of the page. It seems to me that it should all be hidden, or beneath menus as you are reading along and suddenly encounter text which makes no sense, like Go to Home Page or “Search and Plant Trees”. It took me an hour of futzing with it before I realized that “Search and Plant Trees” was actually the address bar! Shouldn’t it read “address bar”?
It’s not clear to me why Chrome, which is built in to Android wasn’t used. Chrome is familiar. This Ecosia thing was just weird!
Part of the problem of course is that when websites are presented with Braille and speech it’s just a morasse of text. Visually data is formatted, and there’s color and positioning: for example a navigation panel on the left, buttons on the right and less significant text in a smaller font. I doubt sighted people would read web pages if they were just unrelenting blobs of text!
And with Monarch, for the first time, some simple formatting could be introduced. For example, a line could be drawn between a toolbar and the main region and the footer for each page, links could be indented, and many other formatting things can be done to make the web page easier to skim as sighted people do. There should be a way to “glance” at a web page on a Monarch especially when a toolbar or navigation region takes up two or three screens of the display. The title should appear at the top of the first screen, such as “Ecosia Settings” so it’s easy to see where you arelocated. And web pages should have a banner too, with the URL at the bottom. If you don’t want to waste screen real estate on clues about your orientation, then a where am I keystroke is mandatory.
When I opened it for the second time, the browser interface appeared, not the last page I was reading, which is OK if this is normal behavior, but a little confusing as last time I’d seen it at the end of the text of a page. There are several buttons that do nothing when you point and click such as “open in new tab” or “Customize and Control Google Chrome”. But they do work if you move focus to them with the spacebar and press enter. That too, is inconsistent.
There should be a unique symbol in Braille for a button, a link, or clickable text. These are all different things and there seems to only be symbols for buttons.
Also the concept of “elements” is only clear to someone who has used a screen reader. If the Monarch will be a first device for many with a browser, the hive course and documentation and future tutorials need to do an in-depth explanation of elements, as sighted people look at hundreds of web pages without ever needing to know what an HTML element is.
When you type in the address bar, the speech keeps reading the number of available suggestions which could be very confusing rather than the keystrokes you are typing. How relevant is the number of suggestions, anyway, and is auto-complete that easy for a Braille user to utilize?
When a page loads it says “Web View” with the first line of that page at the top, but no other text. This doesn’t happen with all pages, but for example, is consistent with bookshare.
Why is computer Braille required? And if so, how about letting the user hit a chord H for a cheat sheet of computer Braille symbols for common punctuation like slash, At-sign, question mark, colon and period. As a former software engineer, I do know computer Braille but this is an unnecessary impediment to user friendliness. Having to surf over to Appendix D in the Word document is not exactly intuitive. And at least put that documentation in HTML so someone with a web browser they do happen to know can easily locate information!
Several times “search and plant trees” without an edit box appeared in the middle of a page with text like “AT&T” “Top Stories”, “Previous”, “Next” “Death Toll from Texas Floods” and it was unclear what news site had loaded, since I’d just made a typo in the address bar, then pressed space on “sorry we were unable to find the page you requested”. I hadn’t actually requested a news site, and yet, there a news site had magically appeared.
I read some of the news stories, but the articles were actually smaller than all the other web clutter.
Speech only seems to read the top line of the display which if that’s how it is supposed to work should be documented. In the User guide it read the top line only also but it was the top line of the previous display, the one you had just scrolled past. Either way, it’s a bit strange.
When I pulled up Help and the context menus with Chord-H and Chord-M I could read about keystrokes I needed to know but not about how to actually **DO ** things, like pull up a page I wanted to surf to or activate a link.
When I’m in an edit field for logging in to a site or searching for something there doesn’t appear to be a cursor I can move to edit mistakes. But perhaps it is just very faint. It keeps saying “computer Braille is required” but in other places I’ve read it isn’t so I’m confused. for example the At sign is dots 4-7 in computer Braille but dot 4 followed by dot 1 in UEB. You need to type that so often it should be clearer which table is active. On my iPhone I can just find the “at” sign, which is so much easier!
And my attempts to enter text with an At sign were always read out loud by speech as an underscore, after I pressed enter on the email address I was trying to enter.
Also if I make an error, the error message like “invalid password” appears on a blank screen; I have to press Space to get the rest of the page to load and the edit field to reappear. When it does, it often repeats its prompt three or four times in Braille. It is possible the “invalid password” appeared in a box with an OK button but then if it is in a box, Braille should use regular conventions to show it in a box.
And often a form field that has a prompt which screen readers read fine doesn’t show the prompt at all in Braille or shows it AFTER you’ve entered the information.
Locating a form field is difficult as well. When I point and click I don’t always go there, and my focus remains unchanged. If I navigate to it with the spacebar, it appears I’m focused on that edit field, but when I begin to type, the element single key navigation is still active, so the L goes to the next link instead of typing an L in the edit field. Pressing Enter does seem to consistently activate an edit field, but it took me a while to figure it out.
I’ve just spent an hour trying to log in to bookshare. This should be intuitive, point and click on login, point and click on the edit field that appears, type in username, pres enter, type in password, press enter.
But it’s not working. As I try to type my username I’m sometimes activating the element navigation keys and sometimes I’m typing, But the speech keeps reading “end of field” rather than the text I’m entering so it’s hard to proofread it. I’m not sure which Braille table I’m using. When I press Enter, the prompt for password is sometimes there, sometimes not though I am always in the next field and my typing is replaced by asterisks in computer Braille. So that at least is consistent.
Now after several failures, a web page inviting me to log in to multiple accounts has appeared listing YouTube, Facebook, Yahoo, Twitter, something called “Legato”, Partners, etc with “Sign Up” at the top of the page. This must be some google intelesense, but it’s hardly what I need now because the bookshare page has disappeared. Multiple tabs are all good if you are sighted but not very intuitive here! Plus I don’t even know what URL I am viewing now. Being able to see the URL is crucial for knowing if you are viewing a malicious site. And if I cannot even log in to my bookshare account with my simple email and password, I’m not attempting these other sites with this browser!
Chrome with Talkback even back as far as Android 5 (on an ancient phone I have) is far more reliable!
The element navigation keys are similar to those in JAWS, NVDA and VoiceOver, but not at all consistent with the screen readers. For example every screen reader uses H to go to the next heading and the numbers 1 through 6 to go to headings at that level. The implementers here decided on dot 1 for heading level 1, dot 2 for heading level2 etc. and dots 4-6 chord to move to the next element. I can get used to it of course, but only some of my screen reader knowledge was helpful here.
Humanware did the same thing with the Brailliant, Chameleon and NLS eReader: instead of giving all of them identical chords, each one uses a slightly different set of keystrokes, which is quite maddening. If you teach folks, you really want consistency and intuitiveness and not to have to constantly refer to a keystroke cheat sheet.
One thing really nice about the Brailliant and eReader that doesn’t seem implemented on the Monarch is that if you read about a keystroke under the context-sensitive help, pressing a routing button on it activates it. So for example, if I want to select text, and I look up how to do it, when I press a routing button on the help line for “highlight” it automatically starts or ends a selection. If I want to enter a quick bookmark, and I find quick bookmark in the context-sensitive help, I just focus on it and hit enter or the cursor routing button. With point and click on the Monarch, this should be a snap to invoke commands without having to memorize them, especially since you might be using multiple devices and don’t want to cram your head full of keystrokes you seldom need.
This would have been useful in the help especially for Ecosia, because a student isn’t taking this home due to its expense.
I think the use case for the Monarch is a student might use it for only an hour or two in a computer lab or classroom. Unlike the Braille 'N Speak, which the previous generation of kids took to bed with them, and thus easily memorized its hundreds of keystrokes because it was a constant companion, the Monarch isn’t going to be used on one’s commute or while waiting for Mom to finish her shopping. So it has to be especially intuitive!
There are many places like this in the Monarch interface that could have been more intuitive. For example, in Ecosia you have three different ways to get to three separate menus, chord W, Chord H and Chord M. Why not have one keystroke to pull up one long context menu for everything a user could possibly do! And make every help line active, so that pointing and clicking “makes it so”. This is how Keysoft is working in general and I’m unsure why they deviated from the proven keysoft interface!
And a page that belongs to the Monarch interface and not to the page you are browsing could have a clear banner saying what it is. Speech says “pop up window” which is a firm grasp in the obvious since it’s already obvious a window has popped up. What’s not obvious is what window this one is! It could read “browser menu” for example.
I also noticed that links aren’t underlined or in any way indicated and neither are clickable elements. So as I work through a page it is hard to know what I can point and click on and what is just static text. It’s possible there are Braille symbols for each I’m missing. For example, it looks like the Braille OW (backwards O) sign might indicate a link. But all that should be documented especially in the context sensitive help! I can find all elements reliably with the element navigation shortcuts. But reading a page is a better way to encounter them and you tend to use those shortcuts only after you know a page well, otherwise you miss important information.
I understand this is beta software and the developers worked hard to implement this on a Braille tablet. All I’m saying here, is it needs to go further to be appealing to average keep it simple users.
There’s a wonderful book on this subject:
“The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity (2nd Edition)”
The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity (2nd Edition) 2, Cooper, Alan, eBook - Amazon.com
about how software developers tend to not think like ordinary users.
When I worked at TeleSensory, four decades ago, I met Jim Halliday, former TeleSensory sales guy who was just founding Humanware and pledged to write software that ordinary folk could easily use. I hope this is still their mission.