Overview

Can modern computer vision techniques be applied to automatically score character states from images?  We are exploring this question by building a taxonomy of the visual manifestations of characters, then identifying the computer vision problems that must be solved, and finally extending existing computer vision methods to solve these problems.

Some characters concern the presence or absence of a particular part or process.  The computer vision problem is “part detection”.  The computer vision method that can be applied is Object Recognition.

Other characters concern relationships between parts, so the computer vision problem is “part modeling”, and the computer vision method that can be applied is the Deformable Part Model. However, this requires extension to handle parts of varying size and to extract the relationships among the parts.

The figure below shows parts of a bat skull detected by a deformable parts model.

avatol_cv

avatol_cv is our system for using computer vision to score characters. The initial release is a questionnaire which allows the biologist to communicate the nature of the character to the system, to help select which algorithm can best be applied. This version is available for download here. It will act as a survey tool so that we can get a sense of the data that might be given to our algorithms. Once the algorithms are hooked into the next release of avatol_cv, there will be three segments: questionnaire, running an algorithm, and results review. The system will be able to work with data in the format downloaded from Morphbank

Bat_Characters

Bat_Characters is a tool for teeth character localization and is available on the download page.

CV image

Image courtesy A. Cirranello, N. Simmons, and P. Velazco, American Museum of Natural History, for the AVATOL project