GUI Automation Tools for Mac OS X

I've received a lot of inquiries recently from people who are interested in controlling Mac OS X's GUI. First, let me say that there are some limitations inherent to most GUI automation processes. Here are the top five that I see most often...

  1. GUI automation is interface-reliant. It expects the interface to always be the same. If it changes, the process must be updated.
  2. GUI automation can sometimes be error prone. For example, changing the window layering on screen, accidentally clicking the mouse, and so forth, can cause some GUI automation tools to become confused.
  3. Tracking down the cause of a GUI automation error or problem is not always straightforward. Locating "button 12" of "window 500", for example, isn't as easy as it sounds.
  4. GUI automation tools typically have a limited ability to perform branching, i.e. looking at something, and making a decision about how to proceed.
  5. While GUI automation tools may be able to click around, select menus, and type keystrokes, they aren't good for "processing" actual data. For example, don't expect GUI automation to retrieve content from a database, bring it into an InDesign document, construct a catalog, output it to PDF, and send it to your printer.

For simple tasks, GUI automation may work just fine. For complex tasks, it may not. In these situations, you may need something much more robust, such as AppleScript, or AppleScript+GUI automation, or some other tool.

So, strictly with regard to GUI automation, there are a number of tools that can help you get the job done…

Automator - Automator has the ability to record manual tasks, and play them back as part of a workflow. Here are some resources I've released in the past to get you started…

Third-Party Automator Actions - My Automator Extension Action Pack includes several GUI automation actions, including:

  • Click Safari Web Form Button
  • Click Safari Web Form Checkbox
  • Click Safari Web Form Radio Button
  • Set Safari Web Form Text or Popup Field
  • Type Keystroke

Learn more or download a demo of my Automator actions here.

QuicKeys - An automation tool that can simulate user interaction by clicking buttons, selecting menus, and more. Essentially, a macro utility. Learn more on the QuicKeys website and in the following episode of my podcast...

iKey - Another macro utility for Mac OS X, allowing you to automate keystrokes, mouse clicks, and more. Learn more on the iKey website and in the following episode of my podcast...

Sikuli - An interesting visual automation tool for Mac OS X, which works from screenshots. Learn more on the Project Sikuli website.

Fake - A browser for automating web tasks, such as filling out forms, testing websites, and more. Learn more on the website.

Eggplant - A visual, screen-shot-based automated testing tool, designed for automating processes across multiple machines (even PCs) through VPN. Learn more on the Eggplant website.

AppleScript Editor - Used to write AppleScripts, but also includes the ability to record manual tasks in certain applications as a script for later playback. Unfortunately, recordable applications are few and far between, but some include BBEdit, Fetch, and the Mac OS X Finder. Learn more in the following episode of my podcast...

If you are into AppleScript, check out my MacTech column on User Interface Scripting.

Be Sociable, Share!