Smaller models seem to be more complex. The encoding, reasoning, and decoding functions are more entangled, spread across the entire stack. I never found a single area of duplication that generalised across tasks, although clearly it was possible to boost one ‘talent’ at the expense of another. But as models get larger, the functional anatomy becomes more separated. The bigger models have more ‘space’ to develop generalised ‘thinking’ circuits, which may be why my method worked so dramatically on a 72B model. There’s a critical mass of parameters below which the ‘reasoning cortex’ hasn’t fully differentiated from the rest of the brain.
Yet despite the progress, some experts believe this experiment raises concerns. 。关于这个话题，新收录的资料提供了深入分析
，这一点在新收录的资料中也有详细论述
ВсеПолитикаОбществоПроисшествияКонфликтыПреступность
「ミドルパワーの結束」カナダ首相が演説に込めた意味は？。关于这个话题，新收录的资料提供了深入分析