self-consistency · reasoning · lamda-pt, gpt-3
recommended confidence: moderate no check recorded
majority-voting over sampled chain-of-thought reasoning paths improves reasoning accuracy over single-path greedy decoding
- Self-consistency improved arithmetic reasoning accuracy over greedy single-path chain-of-thought decoding, with absolute gains of +10.6% on GSM8K, +14.4% on SVAMP, and +23.9% on MultiArith. arXiv:2203.11171
- Self-consistency also improved commonsense reasoning, with +2-5% absolute accuracy improvement on StrategyQA and CommonsenseQA, +4.0% on the ARC easy set (from 75.3% to 79.3%), and +4.7% on the ARC challenge set (from 55.1% to 59.8%). arXiv:2203.11171
- The gains generalized to GPT-3: with 40 sampled reasoning paths, self-consistency outperformed greedy decoding by +8.8% on GSM8K and +23.2% on MultiArith. arXiv:2203.11171
Self-Consistency Improves Chain of Thought Reasoning in Language Models · arXiv preprint, 2022 not read in full