Is Chain-of-Thought Reasoning of LLMs a Mirage? ... Our results reveal that CoT reasoning is a brittle mirage that vanishes when it is pushed beyond training distributions. This work offers a deeper understanding of why and when CoT reasoning fails, emphasizing the ongoing challenge of achieving genuine and generalizable reasoning. ... Our findings reveal that CoT reasoning works effectively when applied to in-distribution or near in-distribution data but becomes fragile and prone to failure even under moderate distribution shifts. In some cases, LLMs generate fluent yet logically inconsistent reasoning steps. The results suggest that what appears to be structured reasoning can be a mirage, emerging from memorized or interpolated patterns in the training data rather than logical inference. ... Together, these findings suggest that LLMs are not principled reasoners but rather sophisticated simulators of reasoning-like text.

NYTimes: Columbia and Brown to Disclose Admissions and Race Data in Trump Deal It took an existential threat from Trump to get them to release this data. Years ago a faculty committee of the University of California system examined just this kind of data and produced the graphs below. If you look at the data you will understand the monstrous nature of affirmative action as practiced in the US for ~50 years. NYT: ... Columbia and Brown will have to maintain “merit-based admissions policies,” according to their settlements, which codify the administration’s broader aims in legally binding language. The universities “may not by any means unlawfully preference applicants based on race, color or national origin in admissions throughout its programs,” both agreements state in identical language. “No proxy for racial admission will be tolerated.” ... “The Department of Justice will put an end to a shameful system in which someone’s race matters more than their ability,” Chad Mizelle, the acting associate attorney general, said in March. “Every college and university should know that illegal discrimination in admissions will be investigated and eliminated.” The language used in the settlements with Columbia and Brown hammers home contested assertions about the Supreme Court admissions case that the Trump administration has been making since February. It insists that the decision goes beyond admissions and bars any consideration of race in university life.

Support ATOM! Open source AI is good! Below, another 💀 graph. Things are MUCH worse than this graph indicates bc most of the China model downloads probably don't happen on HuggingFace. With ~50% of AI developers in PRC, and ~100% of their downloads being open models made by Chinese labs (Llama is not competitive, to be honest), the crossing point has already happened at a global level.

America needs to take open models more seriously. This summer the early lead in open model adoption of the US via Llama has been overtaken by Chinese models. With The American Truly Open Models (ATOM) Project we're looking to build support and express the urgency of this issue.