via Shutterstock
Large language models (LLMs) like GPT-3 capture the imagination of data scientists around the world, thanks to their advanced ability to understand and generate text. Now, Salesforce researchers have leveraged an LLM to create CodeGen, which can understand source code and even generate its own code in different programming languages.
To learn more about CodeGen, we turned to Selling power Chief Scientist Silvio Savarese, who was kind enough to answer a few questions about the research project, its development, and his potential role with Salesforce in the future.
The original inspiration for CodeGen came over a year ago, when Savarese’s team “imagined a conversational system capable of having a chat with the user to solve a coding problem or task”, Savarese said via a Q&A email with datanami.
“With CodeGen, that discussion takes the form of a textual English speech between a human and a machine,” he continued. “The result is AI-generated code that solves the problem discussed.”
Just as language models have demonstrated the ability to understand the writings of William Shakespeare and even generate prose that closely resembles that of The Bard, CodeGen has the ability to understand the various textual components of a programming language and generate code that matches the syntax, rules, and constraints of that language.
“CodeGen bridges natural language and programming language,” Savarese said. “CodeGen can help democratize programming like low-code tools do by lowering the barrier to entry for non-developers.”
CodeGen’s model has a GPT-style architecture and was trained from scratch on Google‘s TPU-v4, said Savarese. At this point, CodeGen remains primarily a research project, according to Savarese, although it is being tested with a small group of users.
So far, CodeGen demos have mostly focused on interactive data science scenarios, such as working with Jupyter notebooks. It has also been used with contextual code completion in common development environments, he said.
“Because CodeGen is a flexible and foundational model, it can be applied at scale,” Savarese said. “For example, it helps us better understand the existing code. It helps detect bugs in code that humans have written, estimate risks, and even summarize the functionality of a code to help new developers understand it. CodeGen even translates code into all programming languages, another advantage when dealing with legacy code that may still have value but is difficult to maintain. »
CodeGen excels in frequently used programming patterns, Savarese said, such as “known efficient implementations of algorithms, file operations, data manipulation, custom analysis tools on platforms such as Tableau, web development and design, or building larger programs consisting of several ‘simple’ steps,” he continued. “Programs for which entirely new algorithms for solving a problem are needed or code written in less common programming languages may be less accessible.”
The model could allow inexperienced users to develop simple programs, Savarese said, while more complex programs would still require some development experience. It could speed up the progression of IT users with some experience, such as administrators, who want to become full-fledged developers, while it could save time for expert coders who want to eliminate redundant or repetitive tasks, he said. -he declares.
CodeGen is available on GitHub.
Related articles:
Google’s massive new language model can explain jokes
10 NLP predictions for 2022
“Deep-Speare” emulates The Bard with AI
Comments are closed.