The earliest mention of kaparos is from the Geonim. They speak about taking a chicken and circling your head and then giving it to feed the poor. Among the reasons given for this minhag include a remembrance of the Sa'ir Laazazel (Machzor Vitri), the chicken gets the four Misos Bais Din instead of us (Shela HaKadosh), a chicken is called "Gever" which is also used for "man" so the chicken dies instead of us.
The minhag originally called only for a chicken or an animal for wealthy people. The Levush says that you can use a fish if no chickens are available and Rav Yaakov Emden says that you can uses vegetation since the pasuk says "KI Ha"adam EItz HaSadeh" the man is like the tree of the field. The Chayei Adam says you can use money.
Kaparos ran into some fierce opposition as many rishonim held that it is an Issur of Nichush, superstition. Other reasons given is that with the mass amount of chickens slaughtered in a short period of time, many became neveilos but in all the commotion were eaten anyway. Among the opposition included the Ramban (one of the great early Mekubalim), the Rashba who proudly proclaimed he stopped the minhag in his city, and the Shulchan Aruch who according to some early girsos wrote in the Shulchan Aruch that it is a "Minhag Shel Shtus". Our girsa says simply that the minhag should be stopped.
The Rema however argues and says it is an old minhag that should be continued and in fact the Arizal and the Shela HaKadosh did kaparos. Today the minhag has prevailed in both the Askenazic and Sephardic communities despite the great protest of the Shulchan Aruch and the great Sephardic rishonim.
It is worthwhile when doing kaparos to keep it in proper perspective and remember the words of the Chayei Adam (144:4) about those who mistakenly exaggerate the forgiving powers of kaparos. "... it is rooted in the hearts of the masses that the entire Kapara of Yom Kippur depends on this, and that kaparos and eating matza (on Pesach) are both equally considered a mitzva..."