Robot Technology News
ROBO SPACE
System based on light may yield powerful, efficient large language models
Artist's rendition of a computer system based on light that could jumpstart the power of machine-learning programs like ChatGPT. Blue sections represent the micron-scale lasers key to the technology. Credits:Image: Ella Maru Studio
System based on light may yield powerful, efficient large language models
by Elizabeth A. Thomson for Materials Research Laboratory
Boston MA (SPX) Aug 24, 2023

ChatGPT has made Machine-learning system based on light could yield more powerful, efficient large language modelss around the world with its ability to write essays, email, and computer code based on a few prompts from a user. Now an MIT-led team reports a system that could lead to machine-learning programs several orders of magnitude more powerful than the one behind ChatGPT. The system they developed could also use several orders of magnitude less energy than the state-of-the-art supercomputers behind the machine-learning models of today.

In the July 17 issue of Nature Photonics, the researchers report the first experimental demonstration of the new system, which performs its computations based on the movement of light, rather than electrons, using hundreds of micron-scale lasers. With the new system, the team reports a greater than 100-fold improvement in energy efficiency and a 25-fold improvement in compute density, a measure of the power of a system, over state-of-the-art digital computers for machine learning.

Toward the future
In the paper, the team also cites "substantially several more orders of magnitude for future improvement." As a result, the authors continue, the technique "opens an avenue to large-scale optoelectronic processors to accelerate machine-learning tasks from data centers to decentralized edge devices." In other words, cellphones and other small devices could become capable of running programs that can currently only be computed at large data centers.

Further, because the components of the system can be created using fabrication processes already in use today, "we expect that it could be scaled for commercial use in a few years. For example, the laser arrays involved are widely used in cell-phone face ID and data communication," says Zaijun Chen, first author, who conducted the work while a postdoc at MIT in the Research Laboratory of Electronics (RLE) and is now an assistant professor at the University of Southern California.

Says Dirk Englund, an associate professor in MIT's Department of Electrical Engineering and Computer Science and leader of the work, "ChatGPT is limited in its size by the power of today's supercomputers. It's just not economically viable to train models that are much bigger. Our new technology could make it possible to leapfrog to machine-learning models that otherwise would not be reachable in the near future."

He continues, "We don't know what capabilities the next-generation ChatGPT will have if it is 100 times more powerful, but that's the regime of discovery that this kind of technology can allow." Englund is also leader of MIT's Quantum Photonics Laboratory and is affiliated with the RLE and the Materials Research Laboratory.

A drumbeat of progress
The current work is the latest achievement in a drumbeat of progress over the last few years by Englund and many of the same colleagues. For example, in 2019 an Englund team reported the theoretical work that led to the current demonstration. The first author of that paper, Ryan Hamerly, now of RLE and NTT Research Inc., is also an author of the current paper.

Additional coauthors of the current Nature Photonics paper are Alexander Sludds, Ronald Davis, Ian Christen, Liane Bernstein, and Lamia Ateshian, all of RLE; and Tobias Heuser, Niels Heermeier, James A. Lott, and Stephan Reitzensttein of Technische Universitat Berlin.

Deep neural networks (DNNs) like the one behind ChatGPT are based on huge machine-learning models that simulate how the brain processes information. However, the digital technologies behind today's DNNs are reaching their limits even as the field of machine learning is growing. Further, they require huge amounts of energy and are largely confined to large data centers. That is motivating the development of new computing paradigms.

Using light rather than electrons to run DNN computations has the potential to break through the current bottlenecks. Computations using optics, for example, have the potential to use far less energy than those based on electronics. Further, with optics, "you can have much larger bandwidths," or compute densities, says Chen. Light can transfer much more information over a much smaller area.

But current optical neural networks (ONNs) have significant challenges. For example, they use a great deal of energy because they are inefficient at converting incoming data based on electrical energy into light. Further, the components involved are bulky and take up significant space. And while ONNs are quite good at linear calculations like adding, they are not great at nonlinear calculations like multiplication and "if" statements.

In the current work the researchers introduce a compact architecture that, for the first time, solves all of these challenges and two more simultaneously. That architecture is based on state-of-the-art arrays of vertical surface-emitting lasers (VCSELs), a relatively new technology used in applications including lidar remote sensing and laser printing. The particular VCELs reported in the Nature Photonics paper were developed by the Reitzenstein group at Technische Universitat Berlin. "This was a collaborative project that would not have been possible without them," Hamerly says.

Logan Wright, an assistant professor at Yale University who was not involved in the current research, comments, "The work by Zaijun Chen et al. is inspiring, encouraging me and likely many other researchers in this area that systems based on modulated VCSEL arrays could be a viable route to large-scale, high-speed optical neural networks. Of course, the state of the art here is still far from the scale and cost that would be necessary for practically useful devices, but I am optimistic about what can be realized in the next few years, especially given the potential these systems have to accelerate the very large-scale, very expensive AI systems like those used in popular textual 'GPT' systems like ChatGPT."

Chen, Hamerly, and Englund have filed for a patent on the work, which was sponsored by the U.S. Army Research Office, NTT Research, the U.S. National Defense Science and Engineering Graduate Fellowship Program, the U.S. National Science Foundation, the Natural Sciences and Engineering Research Council of Canada, and the Volkswagen Foundation.

Research Report:"Deep learning with coherent VCSEL neural networks"

Related Links
Materials Research Laboratory
All about the robots on Earth and beyond!

Subscribe Free To Our Daily Newsletters
Tweet

RELATED CONTENT
The following news reports may link to other Space Media Network websites.
ROBO SPACE
Sidus Space acquires Edge Artificial Intelligence company, Exo-Space
Cape Canaveral FL (SPX) Aug 24, 2023
Sidus Space (NASDAQ: SIDU) reports its strategic acquisition of Exo-Space, a cutting-edge California-based firm specializing in Edge Artificial Intelligence (AI) software and hardware for space applications. This transaction signals Sidus's determination to tap into the growing AI sector and expand its offerings in the Earth and Space Observations services market. Pursuant to the terms of the acquisition agreement Sidus has acquired Exo-Space's assets in a combination of cash, stock options, and p ... read more

ROBO SPACE
Three PKK members killed in Turkish drone strike in Iraq: authorities

Moscow region hit in sixth straight night of drone attacks

Elbit Systems awarded a $55M contract for Dutch Counter UAS solution

Iran unveils its latest attack drone

ROBO SPACE
Japan slams China harassment over Fukushima water release

MIT engineers use kirigami to make ultrastrong, lightweight structures

Observation of metal healing itself confirms researcher's prediction

Droplets unite!

ROBO SPACE
Scientists develop fermionic quantum processor

DNA chips as storage media of the future: What challenges need to be overcome

Chip giant Nvidia rides AI wave as profits soar

British chip champion Arm files to go public in US

ROBO SPACE
Sweden to clear obstacles for new nuclear reactors

Ukraine nuclear plants fully operational for winter: operator

No explosives found on Zaporizhzhia nuclear plant roofs: IAEA

Niger coup raises questions about uranium dependence

ROBO SPACE
Iraq says arrested IS member for gathering intel

Syrians recall 'apocalypse' chemical attack, 10 years on

Guantanamo judge rejects torture-derived confession

U.S. sanctions four linked to 2020 poisoning of Alexei Navalny

ROBO SPACE
British energy regulator Ofgem cuts energy bills to lowest since late 2021

European energy firms doing nothing to tackle climate change, says Greenpeace

Campaigners urge debt cancellation to cut fossil fuel reliance

UK lagging in switch to green energy, study warns

ROBO SPACE
Alumnus' thermal battery helps industry eliminate fossil fuels

Jeep owner Stellantis invests $100 mn in US lithium

DoE announces $112 million for research on computational projects in fusion energy sciences

US lab repeats nuclear fusion feat, with higher yield

ROBO SPACE
From rice to quantum gas: China's targets pioneering space research

China to launch "Innovation X Scientific Flight" program, applications open worldwide

Scientists reveal blueprint of China's lunar water-ice probe mission

Shenzhou 15 crew share memorable moments from Tiangong Station mission

Subscribe Free To Our Daily Newsletters




The content herein, unless otherwise known to be public domain, are Copyright 1995-2024 - Space Media Network. All websites are published in Australia and are solely subject to Australian law and governed by Fair Use principals for news reporting and research purposes. AFP, UPI and IANS news wire stories are copyright Agence France-Presse, United Press International and Indo-Asia News Service. ESA news reports are copyright European Space Agency. All NASA sourced material is public domain. Additional copyrights may apply in whole or part to other bona fide parties. All articles labeled "by Staff Writers" include reports supplied to Space Media Network by industry news wires, PR agencies, corporate press officers and the like. Such articles are individually curated and edited by Space Media Network staff on the basis of the report's information value to our industry and professional readership. Advertising does not imply endorsement, agreement or approval of any opinions, statements or information provided by Space Media Network on any Web page published or hosted by Space Media Network. General Data Protection Regulation (GDPR) Statement Our advertisers use various cookies and the like to deliver the best ad banner available at one time. All network advertising suppliers have GDPR policies (Legitimate Interest) that conform with EU regulations for data collection. By using our websites you consent to cookie based advertising. If you do not agree with this then you must stop using the websites from May 25, 2018. Privacy Statement. Additional information can be found here at About Us.