Overview of a brief field test of the Merlin SoundID app
I downloaded the Merlin app to my iPad and installed the SoundID module. I tested SoundID identification in a bird community typical of Willamette Valley residential areas in July; birds were reasonably vocal but in general not actively singing at this stage of the breeding season. I am familiar with all of the bird species expected at this location. This site has light to moderate ambient noise (traffic and machinery) - generally this is not an issue but it may briefly interfere with audio detections. SoundID recommends working with audio clips of no more than 2 minutes in length. For this test I conducted five 10-minute point counts - during each of which I collected five sequential 2-minute audio clips in SoundID. During these counts I allowed Merlin to access iPad location information, so that suggested identifications were informed by appropriate geographical location. During each point count I recorded all birds that I detected, whether they were identified visually or by sound, and estimated numbers of each species. If Merlin provided a species identification on any of the five audio clips during a count, I scored the species as identified by SoundID. SoundID identifications that I did not myself detect during the point count were noted in the checklist comments.
Each point count was recorded as an eBird checklist:
https://ebird.org/checklist/S91876620
https://ebird.org/checklist/S91878345
https://ebird.org/checklist/S91880634
https://ebird.org/checklist/S91887506
https://ebird.org/checklist/S91887743
Summarizing species detected, with total number of counts on which each was confirmed to have occurred, broken down by three identification methods:
Species #Counts Visual Heard SoundID
Eurasian Collared-Dove 4 3 3 2
Vaux's Swift 3 3 1 1
Anna's Hummingbird 5 3 5 1
Turkey Vulture 2 2 0 0
Cooper's Hawk 0 0 0 1
Great Horned Owl 0 0 0 2
California Scrub-Jay 3 2 2 0
American Crow 3 0 3 1
Violet-green Swallow 3 1 2 2
European Starling 2 2 2 0
Cedar Waxwing 1? 0 0 1
Evening Grosbeak 1 1 1 0
House Finch 3 3 3 2
American Goldfinch 4 2 3 2
Pine Siskin 0 0 0 1
Finch sp. 2 1 1 0
For the species detected only by SoundID during a count, I reviewed the audio clip to assess whether I could identify the call. The Great Horned Owl and Cooper's Hawk identifications appeared to be concurrent with Eurasian Collared-Dove calls. The Pine Siskin identification appeared to be concurrent with a European Starling call; the Cedar Waxwing identification was a high-pitched whistle that might have been correctly identified but was likely a European Starling call.
Total number of confirmed species x count detections was 35, including two finches not confirmed to species. Of these, 23 were visually detected, 26 were heard, and 12 were correctly detected by SoundID. In addition, there were 4 species x count detections by SoundID that appeared to be incorrectly identified, and one (Cedar Waxwing) for which the accuracy of the identification was undetermined.
My thoughts:
1) I enjoyed using this app. Even though the successful identification rate was fairly low, it was impressive that in many cases SoundID successfully picked up an identification within a few seconds. Indeed, in a few cases where I was not paying full attention, the ID popped up slightly before I had mentally processed a sound. Because the app can often provide nearly real-time feedback and does not get distracted, this could be a useful training tool to help with learning bird identification by sound.
2) The accuracy of identification is by no means adequate to be relied upon for a comprehensive bird list or to document rarities. Nonvocal species like Turkey Vultures will of course never be picked up. Moreover, vocalizations that are nearby and loud are much more likely to be detected; in several cases even where a species was successfully ID'd, the birds had been identifiably calling at a moderate distance for several minutes before SoundID successfully made an identification. Possibly a better microphone system than the iPad default would allow more species to be detected. The rate of incorrect identifications was significant, so using the system even for learning purposes certainly needs to be tempered with more than a grain of salt.
3) Evidently starlings are hard! These five point counts included 20 minutes of loud and actively communicating social starlings, about 20 meters away, but SoundID never provided a European Starling ID. Since their vocalizations are so diverse, this might be unsurprising, but it was a significant miss.
4) On other trials where I set the iPad to not allow Merlin access to location information, SoundID did the best it could but was nowhere near as realistic. (A notification did pop up indicating that the identifications did not incorporate location info and might be inaccurate). A wide variety of unlikely North American species were proposed, including Common Redpoll, Flammulated Owl, American Avocet, and Yellow-crowned Night Heron. Enabling location info is necessary to give a fair test of the system.
