When AI tools first came out, in late 2022, I was curious to see how it would work, mainly because I spent three decades as a writing teacher. I tried using ChatGPT, but decided that I really didn’t want to open a new account with yet another password to remember. So I decided to give Google’s new AI tool, Bard, a shot, since I already had a Google account.
My first attempt at using Bard involved having it write a college-level essay on why Americans are afraid that the government is going to come and take away their possessions. Bard responded with a simplistic essay that could have been written by a high school sophomore and lacked citations (which I had specifically requested). A second attempt, with a better prompt, was a little bit better, but not very impressive. (I know now a lot more about how to prompt these AI tools than I did in 2022.)
I was not impressed, but not quite ready to give up, so I gave Bard another chance. This time I asked it to write an obituary for someone I know very well—myself—and I gave it more helpful prompts: “…including his family background, education, occupations, publications, hobbies, and next of kin.” Good old Bard spat out an incredible piece of fiction—nothing it wrote was true, except for my major in college. The rest was just a complete fabrication. Bard, I realized, was hallucinating!
When I told this story to my granddaughter last week, she had an interesting take. “Why didn’t I try to see if Bard can do fact-checking?” It was a clever idea and worth a shot. After all, it’s now 2024, Google says that Bard is now “improved” and called Gemini, so why not see if it can catch factual errors.
I pulled two paragraphs from a biography from a reliable website:
“John N. Laycock was born in Methuen, Massachusetts on 18 May 1892; he attended primary and secondary school in that city and was subsequently appointed to the Naval Academy in 1910. He graduated in 1914 and then served aboard the battleship USS VIRGINIA. His ship was at Vera Cruz, Mexico, during the occupation of July-December 1914. Following his tour with the VIRGINIA, ENS Laycock began postgraduate work at Rensselaer Polytechnic Institute and received a bachelor of engineering degree in 1917. On 6 June 1917 he was promoted to lieutenant, junior grade, and transferred to the Civil Engineer Corps.
“During the 1920s Laycock's career was much like that of his fellows; between 1917 and 1927 he performed public works functions at the Naval Air Station, Cape May, New Jersey; the Navy Yard, Charleston, South Carolina; the Fifteenth Naval District (Canal Zone); and the Naval Station, Newport, Rhode Island. In 1927, the President nominated Laycock, now a lieutenant commander, as Treaty Engineer of the Republic of Haiti and the President of Haiti appointed him as such. In this position, LCDR Laycock served as Director of Municipal Engineering from 1927 to 1928, and as Executive Officer of the Public Works Administration of the Republic of Haiti from 1928 to 1931. Laycock was the last such Treaty Engineer and was made a Commander of the Order of Honor and Merit by the Haitian government for his services.”
Then I re-wrote it and deliberately altered some of the facts (changes are shown in red):
John N. Laycock was born in Madison, Wisconsin on 18 July 1901; he attended Madison War Memorial High School and was drafted into the US Navy in 1918. After the war he was appointed to the Air Force Academy and served aboard the aircraft carrier USS Vermont. His ship was at Cancun, Mexico, during the occupation of July-December 1919.
Ensign Laycock began postgraduate work in Theology and Criminal Law at Rensselaer Polytechnic Institute. Laycock wrote his dissertation on “The Role of the US Navy in the Battle Of Gettysburg” and received a bachelor of engineering degree in 1917. On 6 June 1917 he was promoted to lieutenant, junior grade, and transferred to the Civil Engineer Corps.
During the 1920s Laycock performed public works functions at the Naval Air Station, Cape May, New Jersey; the Navy Yard, Brooklyn, New York; the Fifteenth Naval District (Canal Zone); and the Naval Station, Newport, Rhode Island.
In 1927, President Roosevelt nominated Laycock, now a lieutenant commander, as Treaty Engineer of the Republic of Haiti and the President of Haiti appointed him as such. Commodore John N. Laycock served as Director of Municipal Engineering from 1927 to 1938, and as Executive Officer of the Public Works Administration of the Peoples Republic of China from 1928 to 1931. Laycock was made a Commander of the Order of Honor and Merit by the Haitian government for his services.
I ran the revised error-filled biography past two AI tools, with a prompt asking them to fact-check it.
The first response, from You.com, was encouraging:
Based on the search results, there is no evidence to support the claim that John N. Laycock was born in Madison, Wisconsin on 18 July 1901, attended Madison War Memorial High School, or was drafted into the US Navy in 1918. The search results provide information about a different individual named John N. Laycock who was born in Methuen, Massachusetts on 18 May 1892 and attended primary and secondary school in that city before being appointed to the Naval Academy in 1910 1 . Therefore, the information provided in the claim is not accurate.
Additionally, there is no information in the search results about John N. Laycock serving aboard the aircraft carrier USS Vermont or being in Cancun, Mexico during the occupation of July-December 1919. The search results do not mention John N. Laycock's postgraduate work in Theology and Criminal Law at Rensselaer Polytechnic Institute, his dissertation on "The Role of the US Navy in the Battle Of Gettysburg," or his appointment to the Civil Engineer Corps 1 .
Furthermore, the search results do not provide any information about John N. Laycock performing public works functions at the Naval Air Station, Cape May, New Jersey; the Navy Yard, Brooklyn, New York; the Fifteenth Naval District (Canal Zone); or the Naval Station, Newport, Rhode Island during the 1920s.
Lastly, there is no evidence in the search results to support the claim that John N. Laycock was nominated by President Roosevelt as Treaty Engineer of the Republic of Haiti, served as Director of Municipal Engineering from 1927 to 1938, or served as Executive Officer of the Public Works Administration of the Peoples Republic of China from 1928 to 1931. There is also no mention of John N. Laycock being made a Commander of the Order of Honor and Merit by the Haitian government 1.
In conclusion, the information provided in the claim about John N. Laycock is not supported by the search results.
============
A second response to a request for fact-checking to Perplexity.ai was even better and more specific:
The claims made in the query about John N. Laycock's biography appear to be inaccurate based on the information provided in the search results. Here are the issues:
- There is no mention of John N. Laycock being born in Madison, Wisconsin in 1901 or attending Madison War Memorial High School in the search results.
- The results do not indicate that he was drafted into the US Navy in 1918 or that he attended the Air Force Academy after the war.
- There are no details about him serving aboard the USS Vermont or being in Cancun, Mexico in 1919.
- His supposed postgraduate work in Theology and Criminal Law at Rensselaer Polytechnic Institute, writing a dissertation on the Battle of Gettysburg, and receiving a bachelor's degree in engineering in 1917 are not corroborated.
- The claims about his promotions, assignments, and roles in Haiti, the Canal Zone, China, and as Treaty Engineer are unsubstantiated by the provided information.
- There is no evidence in the results of him being made a Commander of the Order of Honor and Merit by the Haitian government.
In summary, the biographical details presented in the query about John N. Laycock do not align with the information found in the search results, which focus primarily on Thomas Laycock, a 19th century physician, and the experiences of Black Seabees during World War II. Without additional credible sources, the claims about John N. Laycock's life and career cannot be verified as factual.
I decided to give Bard/Gemini a shot at it:
Bard couldn't do it!
No comments:
Post a Comment