🔀 Differences between ChatGPT and human thinking

Currently, many people using ChatGPT easily fall into two misconceptions. One is thinking ChatGPT is too amazing, as it can chat so smoothly, understands quantum mechanics, and can even discuss philosophy with me. They wonder if they can hand over all their work and learning content to it, but when they actually do so, they encounter many problems. The other type thinks ChatGPT is just overhyped by course sellers and internet marketers, pointing out that it doesn't even know "Lu Xun" and "Zhou Shuren" are the same person when asked about their differences, and it gives wrong results for simple multiplications like 89x78, calling it utterly useless.

These two views, considering ChatGPT either too amazing or too stupid, actually belong to the same category. They are like blind men touching an elephant, understanding only a small part of ChatGPT's features and extrapolating them to the whole. The problem with this thinking is that we are evaluating and understanding ChatGPT based on the general abilities and common sense that humans possess, because ChatGPT's logical and expressive abilities displayed during conversations with humans are too human-like.

However, ChatGPT is not human. Its thinking patterns and its strengths and weaknesses are vastly different from humans. Let's first talk about what human thinking patterns are like.

Human Thinking Example - Common Sense, Reflection, Tool Use Suppose you need to write a financial article, and for some reason, you need to write this sentence: "As of June 28, Pinduoduo's market value is xx times that of Bilibili."

When writing this dialogue, you would first activate your reflective ability. You know you need to compare the market values of Pinduoduo and Bilibili, and it's the latest market value difference. You estimate that no one has written this exact data before, so the best way is not to directly search for how many times Pinduoduo's market value is compared to Bilibili's, but to find their respective market values as of yesterday and then divide.

Next, you know your brain definitely doesn't have this data, but you know that Xueqiu (a financial platform) probably has this information. You go there and find that as of yesterday, Pinduoduo's market value is $93.28 billion, and Bilibili's market value is $6.36 billion.

Then you would note these two figures down. You know it's not easy to calculate 93.28/6.36 in your head, so you might use a calculator and find it's 14.7 times.

At this point, you might think that you didn't expect Pinduoduo to be worth so much more than Bilibili, but you double-check your information source and numbers, and they seem correct.

Then you start writing this sentence: "Pinduoduo's stock price is 13.7 times more expensive than Bilibili's."

You would reflect in real-time that this sentence is not accurate. You readjust your wording, add the time constraint, and make the description more precise: "As of June 28, Pinduoduo's market value is 14.7 times that of Bilibili."

This is the entire process you might go through when writing such a simple sentence. But for ChatGPT, it's not like this. It would only decompose the input information into layers of relationships, then start predicting what the next word should be to be more accurate, and every time it writes a character, it would incorporate that character into its thinking, then continuously produce the next character, kua, kua, kua, forming complete sentences. You can learn more about ChatGPT's text generation mechanism from my previous video on how ChatGPT was trained.

So in the end, the thinking process of humans and ChatGPT is vastly different.

The essence of prompt engineering is to bridge this difference, maximizing ChatGPT's advantages while avoiding its defects.

ChatGPT's continuous text generation process is actually very similar to the "System 1" thinking system of humans. If you've read "Thinking, Fast and Slow" by Nobel Prize winner Daniel Kahneman, you probably know that human thinking includes both "System 1" intuitive fast thinking and "System 2" more deliberate slow thinking. For some complex tasks, you can make ChatGPT adopt the slow thinking mode. There are several ways to make ChatGPT think slowly. Since ChatGPT essentially thinks through tokens, as long as you let ChatGPT have more tokens to complete the final answer, it's equivalent to giving it ample thinking time. Here are a few different ways to make ChatGPT slow down and think:

Break down complex tasks into multiple small tasks, so that each task has enough tokens to think about. For example, you can break down the question about comparing Pinduoduo and Bilibili's market values into asking what Pinduoduo's market value was as of yesterday, what Bilibili's market value was, and so on.

Use few-shot prompting to let ChatGPT mimic this thinking process. For example, you write "Beijing's permanent population is 3 times that of Hong Kong," and then tell ChatGPT how you arrived at this result. It will then mimic this process.

Tell the model to "think step by step," making it output its thinking process step by step. This is equivalent to making each token bear less thinking task, which can also achieve better results.

Finally, ChatGPT has very rich knowledge, having learned a large amount of publicly available information on the internet, with probably over 80% being of relatively average quality. So usually, to generate text content that's close to the quality of its training corpus, it won't provide you with the best results, although it actually can. OpenAI co-founder Andrej Karpathy has a vivid way of putting this: ChatGPT doesn't want to succeed, but you can demand success. The specific approach is already familiar to many people: based on the task you expect ChatGPT to complete, tell ChatGPT it's an expert in this field and ask it to output results from an expert perspective.

I hope it will inspire you.