To mitigate the effects of the mistakes, the diversity of demonstrations matter. This automatic process can still end up with mistakes in generated chains.
(2022) (opens in a new tab) propose an approach to eliminate manual efforts by leveraging LLMs with 'Let's think step by step' prompt to generate reasoning chains for demonstrations one by one. This manual effort could lead to suboptimal solutions. When applying chain-of-thought prompting with demonstrations, the process involves hand-crafting effective and diverse examples.
This is particularly useful where you don't have too many examples to use in the prompt.
It's impressive that this simple prompt is effective at this task.
