An essential tool for a bird enthusiast, in particular a beginning one, is a good field guide. A field guide will help you determine what birds you are seeing and hearing in your yard and in your local area. Without a good field guide, even the birds in your backyard can form an overwhelming variety.
Field guides can be found in your local bookstore in the Wild Bird or Audubon section. Field guides are generally region-specific and sometimes species-specific. At your local bookstore, you can usually be assured that there will be general guides available for your region. You can also purchase field guides via online bookstores such as Amazon.Com and Barnes & Noble. If you go this route, be sure you are getting the right one for your region.
An example of a good field guide series are the Stokes Field Guides.
Once you have your field guide, take some time to browse through it. Mark the birds you have seen or the birds that you are likely to see in your area. You can use removable colored tape tabs for this if you don't want to write in your book. Then take your guide out with you into your yard and put it to work.
A field guide will do what its name implies: guide you in identifying all the various birds in your area and keep you from getting lost in their variety.