ChatGPT Said It Was My Best Article. It Was My Worst.

AI's siren song

In the Odyssey, the Sirens’ enchanting voices lure sailors toward their rocky shores. By the time they realize the danger, it’s too late to avoid being shipwrecked.

This edition marks the one-year anniversary of my newsletter. Every word is mine. The hard part is keeping it that way. AI is incredibly helpful, and every week, it tempts me to do more.

Here’s what I’ve learned after a year of hearing AI’s Siren song.

“Want me to write this for you?”

Every Saturday morning, I ask ChatGPT for feedback on my outline. It always closes with the offer: “If you want, I can turn this into a newsletter with a hook, examples, quotes, and a conclusion.”

Boy is that tempting! I spend fifteen hours on each edition. One weekend I was especially under the gun, and took ChatGPT up on its offer. It wrote the whole article in fifteen seconds. I thought it wasn’t bad, but my wife didn’t like it.

So I started editing. The writing was technically correct, but it was littered with short sentences and words like “delve” and “reshape” that I never use. I ended up rewriting almost every sentence, and it took longer than starting from scratch.

People can still spot AI writing, and most everybody hates it. Even when it gets better, publishing AI’s work as my own would feel like paying someone to qualify for the Boston Marathon on my behalf.

Siren Defense #1: Don’t let AI write for you. Ask it for feedback on what you’ve written.

“This is your best article yet!”

Once my draft is ready, I ask ChatGPT for feedback. “This is strong. I would forward this to my colleagues.”

Now that’s music to my ears. My writing is making massive leaps. Here comes the next Ethan Mollick!

But then I gave it an article that I knew wasn’t ready. It loved it. 🤔

I was suspicious, so I had it rank the same article versus my last 10 editions. It came in dead last. 🤦

AIs will always find a way to be positive, and this hollow praise gives us confidence we haven’t earned. We play into this temptation by asking generic questions instead of pinning it down.

Siren Defense #2: Never ask AI if your work is good. Ask where it’s weakest, or to compare it with your past writing.

“Just a few small edits”

One week I asked ChatGPT for feedback. I made some changes, and fed back the revised section. “That’s a lot better. But your third section needs work.”

I fixed those things, and it gave me more, and more after that. Eventually the suggestions began contradicting earlier ones. After an hour, it had lost the point of what I wanted to say in the first place.

AIs are trained to be endlessly helpful, always offering another suggestion. This temptation plays on our desire for perfection, especially on writing we care about. The trick is knowing when it’s good enough.

Siren Defense #3: Cut off AI revisions after two rounds and ask your spouse for their take. Remember to ask nicely!

Don’t Forget How to Sail

The biggest risk isn’t hitting the rocks. It’s forgetting how to sail.

I’ve learned a ton about writing over the past year. How to keep a theme throughout an article. How to balance stories, citations and visuals. How to say something in 800 words instead of 1,200. How to avoid putting too many ideas in a list. 😉

Each edition takes me about 15 hours, and some have been a lot more. The temptation to let AI do it is always there. If I gave in, I wouldn’t have learned any of those lessons.

Odysseus had his crew plug their ears and tie him to the mast so he could hear the Sirens without steering into the rocks.

AI is a Siren’s call for you and me.

Hear it, but don’t let it replace your own voice. You might not get it back.

---

Dad Joke: Why did ChatGPT take English literature class? Because it wanted to learn the “write” stuff.