Shane Bernard, the McIlhenny Company archivist, suspects the tale got started because people confused Lee's daughter, Mildred (Millie) Childe Lee, with Margaret (Minnie) Henshaw Avery, daughter of Judge Daniel Dudley Avery, patriacrch of the family for whom the island is named.
Minnie Avery is buried on Avery Island and has Confederate connections. She was married to William Preston Johnston. He was the son of Confederate Gen. Albert Sidney Johnston, for whom Johnston Street in Lafayette was named and who was right up there with Lee among the dieties of the South.
William Johnston and Minnie Avery lived in New Orleans but she was widowed by the time of her death and chose to be buried on the island. Millie Lee was a friend of the McIlhenny and Avery families and visited them in New Orleans and at Avery Island, but there is no record that she died there.
Adding to the confusion, socialite Margaret Rose Anthony Julia Josephine Catherine Cornelia Donovan O'Donovan, who thankfully went by the simpler name of Daisy Breaux after her mother married into that family, wrote in her 1930 "Autobiography of a Chameleon" that Mildred Lee died at Minnie Avery Johnston's home, "her death occurring suddenly after she had been to the French opera one evening." If Mrs. Breaux is correct, Millie Lee's death would have been in New Orleans, not on Avery Island.
Magruder Drake, who attempted to teach me history many years ago at USL, told me some years back that Lee and all of his family are buried at the Lee Chapel on the Washington & Lee University campus in Lexington, Va. Dr. Drake was on the faculty there at one time.
William Preston Johnston taught at that school from 1867 to 1877, and that may be how Millie Lee and Minnie Avery got to know each other in later years. Robert E. Lee was president of school from 1865 to 1870, so Johnston was on the campus at the same time as the Lee family.
Minnie Avery was William Johnston's second wife and probably did not meet Millie Lee until after their marriage in April 1888. By that time William had moved to Louisiana, spending three years in Baton Rouge as president of LSU, then moving to New Orleans to become president of Tulane University.
According to an account in "The Robert E. Lee Family Cooking and Housekeeping Book" (University of North Carolina Press, 1997) by Anne Carter Zimmer, who was a great-granddaughter of Robert E. Lee, "Mildred used to visit the family of ... William Preston Johnston and his first wife in Louisiana after he became president of Tulane, and the youngest Lee daughter died while staying with the widowed second Mrs. Johnston, the former Margaret Avery."
The Zimmer book does make the claim that Millie Lee died on Petite Anse Island, as Avery Island was once called, but practically everyone else seems sure that Millie was visiting Minnie in New Orleans at the time of her death.
Zimmer's book also contains a gingerbread recipe given to Millie Lee by Margaret Avery which appears on notepaper headed "Petite-Anse Island." That letterhead may have led Zimmer to think the visit and death were on the island.
You can contact Jim Bradshaw at firstname.lastname@example.org or P.O. Box 1121, Washington LA 70589.