Newest first. Each card says what the paper showed, in plain words, and who among its authors later joined Anthropic. Each square is one author, in the paper's order; orange marks someone who later worked there.
2020 · GPT-310 of 31 authors later joinedIntroduced GPT-3, a 175-billion-parameter language model that could take on new tasks from just a few examples written in the prompt.
Tom Brown, later at AnthropicBen Mann, later at AnthropicNick RyderMelanie SubbiahJared Kaplan, later at AnthropicPrafulla DhariwalArvind NeelakantanPranav ShyamGirish SastryAmanda Askell, later at AnthropicSandhini AgarwalAriel Herbert-VossGretchen KruegerTom Henighan, later at AnthropicRewon ChildAditya RameshDaniel Ziegler, later at AnthropicJeffrey Wu, later at AnthropicClemens WinterChristopher HesseMark ChenEric SiglerMateusz LitwinScott GrayBenjamin ChessJack Clark, later at AnthropicChristopher BernerSam McCandlish, later at AnthropicAlec RadfordIlya SutskeverDario Amodei, later at Anthropic20208 of 19 authors later joinedShowed that the scaling laws found for language models also hold for models of images, video, math, and paired images and text.
Tom Henighan, later at AnthropicJared Kaplan, later at AnthropicMor KatzMark ChenChristopher HesseJacob JacksonHeewoo JunTom Brown, later at AnthropicPrafulla DhariwalScott GrayChris HallacyBen Mann, later at AnthropicAlec RadfordAditya RameshNick RyderDaniel Ziegler, later at AnthropicJohn Schulman, later at AnthropicDario Amodei, later at AnthropicSam McCandlish, later at Anthropic2020 · Scaling laws6 of 10 authors later joinedFound that language models improve in a predictable way as they get bigger and are trained with more data and computing power.
Jared Kaplan, later at AnthropicSam McCandlish, later at AnthropicTom Henighan, later at AnthropicTom Brown, later at AnthropicBenjamin ChessRewon ChildScott GrayAlec RadfordJeffrey Wu, later at AnthropicDario Amodei, later at Anthropic2020 · Summarize from feedback3 of 9 authors later joinedTrained models to summarize text using human feedback; people preferred their summaries to the human-written reference summaries.
Nisan StiennonLong OuyangJeffrey Wu, later at AnthropicDaniel Ziegler, later at AnthropicRyan LoweChelsea VossAlec RadfordDario Amodei, later at AnthropicPaul Christiano2020 · Zoom In2 of 6 authors later joinedArgued that a neural network can be understood piece by piece, as circuits of features a person can interpret.
Chris Olah, later at AnthropicNick CammarataLudwig SchubertGabriel GohMichael PetrovShan Carter, later at Anthropic2019 · Fine-tuning from preferences4 of 8 authors later joinedApplied learning from human preferences to language models, teaching them to continue and summarize text the way people preferred.
Daniel Ziegler, later at AnthropicNisan StiennonJeffrey Wu, later at AnthropicTom Brown, later at AnthropicAlec RadfordDario Amodei, later at AnthropicPaul ChristianoGeoffrey Irving2019 · GPT-22 of 6 authors later joinedShowed that a 1.5-billion-parameter language model could translate, summarize and answer questions without being trained for those tasks.
Alec RadfordJeffrey Wu, later at AnthropicRewon ChildDavid LuanDario Amodei, later at AnthropicIlya Sutskever2018 · Building Blocks2 of 7 authors later joinedBuilt visual tools for seeing what an image-recognition model has learned and how it reaches its answers.
Chris Olah, later at AnthropicArvind SatyanarayanIan JohnsonShan Carter, later at AnthropicLudwig SchubertKatherine YeAlexander Mordvintsev2017 · RL from human preferences3 of 6 authors later joinedTrained AI agents from people's choices between pairs of short clips instead of a hand-written goal, the approach later called reinforcement learning from human feedback (RLHF).
Paul ChristianoJan Leike, later at AnthropicTom Brown, later at AnthropicMiljan MarticShane LeggDario Amodei, later at Anthropic2016 · Concrete Problems3 of 6 authors later joinedSet out five practical problems in making AI systems behave safely, such as gaming the goal they are given or causing unwanted side effects.
Dario Amodei, later at AnthropicChris Olah, later at AnthropicJacob SteinhardtPaul ChristianoJohn Schulman, later at AnthropicDan Mané