The Academy was founded in 1898, to honor artists and their achievements. Members of the Academy pick the artists, and every year there are three major exhibitions shown. The fields that are shown could be from literary works, paintings, photographs, manuscripts and musical scores. The Academy admits artists that have excelled in their own field. The artists covet the admission to the Academy as one of the highest honors that can be bestowed on them.

The Academy has over 25,000 book and 2000 original works of art that include photographs, paintings and manuscripts.

The Academy, which no occupies the property of John Audobon, is in the same location as the American Numismatic Society, and the Hispanic Society of America. The building was financed by Archer M. Huntington, and was constructed in the 1920 's by Mc Kim, Mead, and White. Past Members include Mark Twain, Allen Ginsberg, and Duke Ellington, William Faulkner, Dizzy Gillespie, John Steinbeck, and Frank Lloyd Wright.