{"id":54392,"date":"2021-12-09T13:04:23","date_gmt":"2021-12-09T11:04:23","guid":{"rendered":"https:\/\/forklog.com\/en\/?p=54392"},"modified":"2025-09-03T20:11:39","modified_gmt":"2025-09-03T17:11:39","slug":"deepmind-unveils-a-280-billion-parameter-language-model","status":"publish","type":"post","link":"https:\/\/u1f987.com\/en\/deepmind-unveils-a-280-billion-parameter-language-model\/","title":{"rendered":"DeepMind unveils a 280-billion-parameter language model"},"content":{"rendered":"<p>British AI lab DeepMind has developed a large language model, Gopher, with 280 billion parameters. Researchers say that the larger the model, the more accurate its performance.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<blockquote class=\\\"twitter-tweet\\\">\n<p lang=\\\"en\\\" dir=\\\"ltr\\\">Today we\u2019re releasing three new papers on large language models. This work offers a foundation for our future language research, especially in areas that will have a bearing on how models are evaluated and deployed: <a href=\\\"https:\/\/t.co\/TV05K4zptv\\\">https:\/\/t.co\/TV05K4zptv<\/a> 1\/ <a href=\\\"https:\/\/t.co\/SyWb8qIDk0\\\">pic.twitter.com\/SyWb8qIDk0<\/a><\/p>\n<p>\u2014 DeepMind (@DeepMind) <a href=\\\"https:\/\/twitter.com\/DeepMind\/status\/1468613620280004614?ref_src=twsrc%5Etfw\\\">December 8, 2021<\/a><\/p><\/blockquote>\n<p> <script async=\\\"\\\\\\\" src=\\\"https:\/\/platform.twitter.com\/widgets.js\\\" charset=\\\"utf-8\\\"><\/script>\\n\\n\\n\\n<\/p>\n<p>Researchers, through their own study, confirmed the hypothesis that a language model\u2019s accuracy depends on its size. As the number of parameters increases, Gopher\u2019s performance improves on the most common benchmarks, such as sentiment analysis and generalization.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<blockquote class=\\\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\\\">\n<p>\\&#8221;One of the key findings of the paper is that progress and capabilities of large language models are still increasing. This is not the area that has plateaued,\\&#8221; said DeepMind researcher Jack Rae.<\/p>\n<\/blockquote>\n<p>\\n\\n\\n\\n<\/p>\n<p>However, researchers identified a number of shortcomings of this approach. According to Rae, there are many scenarios in which the model can fail:<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<blockquote class=\\\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\\\">\n<p>\\&#8221;Some of these failure modes relate to the model simply not sufficiently understanding what it reads.\\&#8221;<\/p>\n<\/blockquote>\n<p>\\n\\n\\n\\n<\/p>\n<p>Rae believes that the problem of misunderstanding context can be addressed by increasing the training data and scaling up the models.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<p>He added that there are other issues, such as entrenching stereotypical biases, the spread of misinformation, or toxic vocabulary. DeepMind believes that scaling up will not remove these shortcomings.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<blockquote class=\\\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\\\">\n<p>\\&#8221;In these cases, language models will require &#8216;additional training procedures&#8217;, such as human feedback,\\&#8221; Rae noted.<\/p>\n<\/blockquote>\n<p>\\n\\n\\n\\n<\/p>\n<p>Whether Gopher will be released publicly remains unknown. DeepMind says it will continue studying language models to make AI applications safer and more transparent.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<p>Earlier in October, Microsoft and Nvidia introduced the Megatron language model with 530 billion parameters.<\/p>\n<p>\\n\\n\\n\\n<a href=\"https:\/\/u1f987.com\/en\/news\/israeli-startup-develops-an-affordable-alternative-to-gpt-3\">developed an accessible alternative to GPT-3<\/a>. The largest version of the model contains 178 billion parameters.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<p>In January, researchers from Google Brain introduced a language model with 1 trillion parameters.<\/p>\n<p>\\n\\n\\n\\n<\/p>\n<p>Subscribe to ForkLog news on Telegram: <a href=\\\"https:\/\/t.me\/forklogAI\\\" target=\\\"_blank\\\" rel=\\\"noreferrer noopener nofollow\\\">ForkLog AI<\/a> \u2014 all the news from the world of AI!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>British AI lab DeepMind has developed a large language model, Gopher, with 280 billion parameters. Researchers say that the larger the model, the more accurate its performance.<\/p>\n","protected":false},"author":1,"featured_media":54393,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"select":"1","news_style_id":"1","cryptorium_level":"","_short_excerpt_text":"","creation_source":"","_metatest_mainpost_news_update":false,"footnotes":""},"categories":[3],"tags":[438,1474],"class_list":["post-54392","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news-and-analysis","tag-artificial-intelligence","tag-deepmind"],"aioseo_notices":[],"amp_enabled":true,"views":"41","promo_type":"1","layout_type":"1","short_excerpt":"","is_update":"","_links":{"self":[{"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/posts\/54392","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/comments?post=54392"}],"version-history":[{"count":1,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/posts\/54392\/revisions"}],"predecessor-version":[{"id":54394,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/posts\/54392\/revisions\/54394"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/media\/54393"}],"wp:attachment":[{"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/media?parent=54392"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/categories?post=54392"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/u1f987.com\/en\/wp-json\/wp\/v2\/tags?post=54392"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}