Research

Efficacy, effectiveness, evaluations, evidence, UNESCO

Language models (LMs), such as GPT-4, are at the forefront of natural language processing, offering capabilities that range from crafting complex prose to solving intricate computational problems. Despite their advanced functionalities, these models need fixing, sometimes yielding inaccurate or conflicting outputs. The challenge lies in enhancing their precision and versatility, particularly in complex, multi-faceted tasks. A key issue with current language models is their occasional inaccuracy and limitation in handling diverse and complex tasks. While these models excel in many areas, their efficacy could improve when confronted with tasks that demand nuanced understanding or specialized knowledge beyond their general capabilities.

We introduce meta-prompting, an effective scaffolding technique designed to enhance the functionality of language models (LMs). This approach transforms a single LM into a multi-faceted conductor, adept at managing and integrating multiple independent LM queries. By employing high-level instructions, meta-prompting guides the LM to break down complex tasks into smaller, more manageable subtasks. These subtasks are then handled by distinct "expert" instances of the same LM, each operating under specific, tailored instructions. Central to this process is the LM itself, in its role as the conductor, which ensures seamless communication and effective integration of the outputs from these expert models. It additionally employs its inherent critical thinking and robust verification processes to refine and authenticate the end result. This collaborative prompting approach empowers a single LM to simultaneously act as a comprehensive orchestrator and a panel of diverse experts, significantly enhancing its performance across a wide array of tasks. The zero-shot, task-agnostic nature of meta-prompting greatly simplifies user interaction by obviating the need for detailed, task-specific instructions. Furthermore, our research demonstrates the seamless integration of external tools, such as a Python interpreter, into the meta-prompting framework, thereby broadening its applicability and utility. Through rigorous experimentation with GPT-4, we establish the superiority of meta-prompting over conventional scaffolding methods: When averaged across all tasks, including the Game of 24, Checkmate-in-One, and Python Programming Puzzles, meta-prompting, augmented with a Python interpreter functionality, surpasses standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multipersona prompting by 15.2%.

An early look into the future of AI with General World Models.

The State of AI Report analyses the most interesting developments in AI. Read and download here.

The impact of digital technologies and importance of high quality education materials

The idea that the increased ubiquity of digital devices negatively impacts neurodevelopment is as compelling as it is disturbing. This study investiga…

Editor’s note: This essay is an entry in Fordham’s 2023 Wonkathon, which asked contributors to answer this question: “How can we harness the power but mitigate the risks of artificial intelligence in our schools?” Learn more.

How can policymakers credibly reveal and assess intentions in the field of artificial intelligence? Policymakers can send credible signals of their intent by making pledges or committing to undertaking certain actions for which they will pay a price—political, reputational, or monetary—if they back down or fail to make good on their initial promise or threat. Talk is cheap, but inadvertent escalation is costly to all sides.

A study pitting human children against large language models in basic reasoning problems revealed AIs to have one big blind spot: innovation.

We may already be in the era of ‘peak humanity’, a time where we have the greatest levels of education, reasoning, rationality, and creativity – spread out amongst the greatest number of us. A brilliant result of the massification of universal basic education and the power of the university. But with the rapid advancement of Artificial Intelligence (AI) that can already replicate and even exceed many of our reasoning capabilities – there may soon be less incentive for us to learn and grow. The grave risk is that we then become de-educated and de-coupled from the driving seat to the future. In all the hype about AI, we need to properly assess these risks to collectively decide whether the AI upsides are worth it and whether we should ‘stick or twist’. This paper aims to catalyse the debate and reduce the probability that we sleepwalk to a destination that we don’t want and can’t reverse back out of. We also make 13 clear recommendations about how AI developments could be regulated - to slow things down a little and give time for informed choices about the best future for humanity. Those potential long-term futures include: (1) AI Curtailment; (2) Fake Work; (3) Transhumanism; and (4) Universal Basic Income – each with very different implications for the future of education.

Anthropic is an AI safety and research company that's working to build reliable, interpretable, and steerable AI systems.

In a survey conducted for the Junior Achievement organization in July 2023, of the 1,006 respondents between the ages of 13 and 17 who were polled, nearly half of them said they intend to use AI this fall to do their classwork for them. But most teens consider doing this to be “cheating.”

Can the online encyclopedia help teach A.I. chatbots to get their facts right — without destroying itself in the process?

What happens if you fill a virtual town with AIs and set them loose? Turns out, they brush their teeth and are very nice to one another!

Reflections on AI and the future of human flourishing

We’ve observed agents discovering progressively more complex tool use while playing a simple game of hide-and-seek. Through training in our new simulated hid...

Abrupt Emergence Capabilities - https://arxiv.org/pdf/2202.07785.pdfModel Evaluation For Extreme Risk - https://arxiv.org/pdf/2305.15324.pdf#page=17&zoom=100...