Home › Posts tagged nvidia

Tag: nvidia

Posted on Posted in Digital transformation, General topics Tagged with , , , , ,

Nvidia’s transformation from an accelerator of video games to an enabler of artificial intelligence (AI) and the industrial metaverse didn’t happen overnight — but the leap in its stock market value to over a trillion dollars did.

It was when Nvidia reported strong results for the three months to April 30, 2023, and forecast its sales could jump by 50% in the following fiscal quarter, that its stock market valuation soared, catapulting it into the exclusive trillion-dollar club alongside well-known tech giants Alphabet, Amazon, Apple, and Microsoft. The once-niche chipmaker, now a Wall Street darling, was becoming a household name.

Investor exuberance waned later that week, however, dropping the chip designer out of the trillion-dollar club in short order, just as former members Meta and Tesla before it. But it was soon back in, and in mid-June, investment bank Morgan Stanley forecast Nvidia’s value could continue to rise another 15% before the year is out.

By late August, Nvidia had more than justified its earlier optimism, reporting a quarter-on-quarter increase in revenue of 88% for the three months to July 30, driven by record sales of data center products of over $10 billion, with strong demand from AWS, Google, Meta, Microsoft, and Oracle. Its stock price, too, continued to climb, bumping up against the $500 level Morgan Stanley forecast. Unlike most of its trillion-dollar tech cohorts, Nvidia has less consumer brand awareness to go on, making its Wall Street leap more mysterious to Main Street. How Nvidia got here and where it’s going next sheds light on how the company has achieved that valuation — a story that owes a lot to the rising importance of specialty chips in business, and accelerating interest in the promise of generative AI.

Graphics driver

Nvidia started out in 1993 as a fabless semiconductor firm designing graphics accelerator chips for PCs. Its founders spotted that generating 3D graphics in video games — then a fast-growing market — placed highly repetitive, math-intensive demands on PC central processing units (CPUs). They realized those calculations could be performed more rapidly in parallel by a dedicated chip rather than in series by the CPU, an insight that led to the creation of the first Nvidia GeForce graphic cards.

For many years, graphics drove Nvidia’s business; even 30 years on, its sales of graphics cards for gaming, including the GeForce line, still make it the biggest vendor of discrete graphics cards in the world. (Intel makes more graphics chips, though, because most of its CPUs ship with the company’s own integrated graphics silicon.)

Over the years, other uses for the parallel-processing capabilities of Nvidia’s graphical processing units (GPUs) emerged, solving problems with a similar matrix arithmetic structure to 3D-graphics modelling.

Still, software developers seeking to leverage graphics chips for non-graphical applications had to wrangle their calculations into a form that could be sent to the GPU as a series of instructions for either Microsoft’s DirectX graphics API or the open-source OpenGL (Open Graphics Library).

Then in 2006 Nvidia introduced a new GPU architecture, CUDA, that could be programmed directly in C to accelerate mathematical processing, simplifying its use in parallel computing. One of the first applications for CUDA was in oil and gas exploration, processing the mountains of data from geological surveys.

The market for using GPUs as general-purpose processors (GPGPUs) really opened up in 2009, when OpenGL publisher Khronos Group released Open Computing Language (OpenCL).

Soon, hyperscalers such as AWS added GPUs to some of their compute instances, making scalable GPGPU capacity available on demand, thereby lowering the barrier of entry to compute-intensive workloads for enterprises everywhere.

AI, crypto mining, and the metaverse

One of the biggest drivers of demand for Nvidia’s chips in recent years has been AI, or, more specifically, the need to perform trillions of repetitive calculations to train machine learning (ML) models. Some of those models are truly gargantuan: OpenAI’s GPT-4 is said to have over 1 trillion parameters. Nvidia was an early supporter of OpenAI, even building a special compute module based on its H100 processors to accelerate the training of the large language models (LLMs) the company was developing.

Another unexpected source of demand for the company’s chips has been cryptocurrency mining, the calculations for which can be performed faster and in a more energy-efficient manner on a GPU than on a CPU. Demand for GPUs for cryptocurrency mining meant that graphics cards were in short supply for years, making GPU manufacturers like Nvidia similar to pick-axe retailers during the California Gold Rush.

Although Nvidia’s first chips were used to enhance 3D gaming, the manufacturing industry is also interested in 3D simulations, and its pockets are deeper. Going beyond the basic rendering and accelerating code libraries of OpenGL and OpenCL, Nvidia has developed a software platform called Omniverse — a metaverse for industry used to create and view digital twins of products or even entire production lines in real-time. The resulting imagery can be used for marketing or collaborating on new designs and manufacturing processes.

Efforts to stay in the $1t club

Nvidia is driving forward on many fronts. On the hardware side, it continues to sell GPUs for PCs and some gaming consoles; supplies computational accelerators to server manufacturers, hyperscalers, and supercomputer manufacturers; and makes chips for self-driving cars. It’s also in the service business, operating its own cloud infrastructure for pharmaceutical firms, the manufacturing industry, and others. And it’s a software vendor, developing generic libraries of code that anyone can use to accelerate calculations on Nvidia hardware, as well as more specific tools such as its cuLitho package to optimize the lithography stage in semiconductor manufacturing.

But interest in the latest AI tools such as ChatGPT (developed on Nvidia hardware), among others, is driving a new wave of demand for Nvidia hardware, and prompting the company to develop new software to help enterprises develop and train the LLMs on which generative AI is based.

In the last few months the company has also partnered with software vendors including Adobe, Snowflake, ServiceNow, Hugging Face, and VMware, to ensure the AI elements of their enterprise software are optimized for its chips.

“Because of our scale and velocity, we’re able to sustain this really complex stack of software and hardware, networking and compute across all these different usage models and computing environments,” CEO Jensen Huang said during a call on August 23 to discuss the latest earnings.

Nvidia is also pitching AI Foundations, its cloud-based generative AI service, as a one-stop shop for enterprises that might lack resources to build, tune, and run custom LLMs trained on their own data to perform tasks specific to their industry. The move, announced in March, may be a savvy one, given rising business interest in generative AI, and it pits the company in direct competition with hyperscalers that also rely on Nvidia’s chips.

Nvidia AI Foundations models include NeMo, a cloud-native enterprise framework; Picasso, an AI capable of generating images, video, and 3D applications; and BioNemo, which deals in molecular structures, making generative AI particularly interesting for accelerating drug development, where it can take up to 15 years to bring a new drug to market. Nvidia says its hardware, software, and services can cut early-stage drug discovery from months to weeks. Amgen and AstraZeneca are among the pharmaceutical firms testing the waters, and with US pharmaceutical firms alone spending over $100 billion a year on R&D, more than three times Nvidia’s revenue, the potential upside is clear.

Pharmaceutical development is moving faster, but the road toward widespread adoption of another of Nvidia’s target markets is less clear: self-driving cars have been “just around the corner” for years, but testing and getting approval for use on the open road is proving even more complex than getting approval for a new drug.

Nvidia gets two bites at this market. One is building and running the virtual worlds in which self-driving algorithms are tested without putting anyone at risk. The other is the cars themselves. If the algorithms make it out of the virtual world and onto the roads, cars will need chips from Nvidia and others to process real-time imagery and perform myriad calculations needed to keep them on course. This is the smallest market segment Nvidia breaks out in its quarterly results: just $253 million, or 2% of overall sales, in the three months to July 30, 2023. But it’s a segment that’s been more than doubling each year.

When it reported its results for the three months to April 30, Nvidia made an ambitious forecast: that its revenue for the following fiscal quarter, ending July 30, would be over 50% higher — and it went on to beat that figure by a wide margin, reporting revenue of $13.5 billion. Growth in gaming hardware sales was also up 22% year on year, and 11% quarter on quarter, which would be impressive for most consumer electronics companies, but lags far behind the recent growth in Nvidia’s biggest market — data centers. The proportion of its overall revenue coming from gaming has shrunk from over one-third in the three months to April 30 to just under one-fifth in the period to July 30. Nevertheless, Nvidia still sees opportunity ahead, as less than half of its installed base has upgraded to graphics cards with the Geforce RTX technology it introduced in 2018, CFO Colette Kress said during the call.

Huang and Kress both talked up how clearly Nvidia can see future demand for its consumer and data center products, well into next year.

“The world is transitioning from general-purpose computing to accelerated computing,” Huang said. With around $250 billion in capital expenditure on data centers every year, according to Huang, the potential market for Nvidia is enormous as that transition plays out.

“Demand is tremendous,” he said, adding that the company is significantly expanding its production capacity to boost supply for the rest of this year and into next.

Nevertheless, Kress was more reserved in her projections for the three months to October 30, saying she expects revenue of between $15.7 billion and $16.3 billion, or quarter-on-quarter growth between 16% and 21%.

All eyes will be on the company’s next earnings announcement, on November 21.

Artificial Intelligence, C Language, Cryptocurrency, GPUs, Manufacturing Systems, Nvidia, Software Deployment, Software Development

Posted on Posted in Digital transformation, General topics Tagged with , , , , , ,

As the generative AI bandwagon gathers pace, Nvidia is promising tools to accelerate it still further.

On March 21, CEO Jensen Huang (pictured) told attendees at the company’s online-only developer conference, GTC 2023, about a string of new services Nvidia hopes enterprises will use to train and run their own generative AI models.

When they hit the market, they’ll open up more options along the build-vs-buy continuum for CIOs called upon to support AI training workloads.

This doesn’t mean CIOs can just hand off responsibility for AI infrastructure, said Shane Rau, a research VP covering data processing chips for IDC. 

“CIOs should already understand that AI is not one-size-fits-all,” he said. “The AI solution stack varies according to how it will be used, which implies one must have intimate understanding of the AI use case — your AI workers and the end domain in which they work — and how to map the needs of the use case to the silicon, software, system hardware, and services.”

Nvidia is offering as-a-service solutions to those problems at several levels. Nvidia AI Foundations is a family of cloud services with which enterprises will be able to build their own large language models (LLMs), the technologies at the heart of generative AI systems, and run them at scale, calling them from enterprise applications via Nvidia’s APIs.

There will be three Foundations services at launch, still in limited access or private preview for now: NeMo for generating text, Picasso for visuals, and BioNeMo for molecular structures. Each offering will include pre-trained models, data processing frameworks, personalization databases, inference engines, and APIs that enterprises can access from a browser, Nvidia said.

Generative AI in action

Financial data provider Morningstar is already studying how it can use test-based NeMo to extract useful information about markets from raw data, drawing on the expertise of its staff to tune the models, according to Nvidia.

The Picasso service will enable enterprises to train models to generate custom images, videos, and even 3D models in the cloud. Nvidia is partnering with Adobe to deliver such generative capabilities inside Adobe’s tools for creative professionals such as Photoshop and After Effects.

Nvidia is seeking to clean up graphical generative AI’s reputation for playing fast and loose with the rights of the artists and photographers on whose works the models are trained. There are concerns that using such models to create derivative work could expose enterprises to lawsuits for breach of copyright. Nvidia hopes to allay those concerns by striking a licensing deal with stock image library Getty Images, which says it will pay royalties to artists on revenue generated by models trained on the works in its database.

Nvidia is working with another library, Shutterstock, to train Picasso to create 3D models in response to text prompts based on licensed images in its database. These 3D designs will be available for use in industrial digital twins running on Nvidia’s Omniverse platform.

The third AI Foundations service, BioNeMo, deals not in words and images but in molecular structures. Researchers can use it to design new molecules and predict their behavior. Nvidia is targeting it at pharmaceutical firms for drug discovery and testing, fine-tuning it with proprietary data. It named biotechnology company Amgen as one of the first users of the service.

AI infrastructure additions

Nvidia’s AI Foundations software services will run in DGX Cloud, a new infrastructure-as-a-service offering.

DGX is the name of Nvidia’s supercomputer in a box, one of the first of which was delivered to OpenAI, developer of ChatGPT, in 2016. Half of all Fortune 100 companies now have their own DGX supercomputers, according to Nvidia.

Cloud providers such as Oracle and Microsoft are beginning to offer access to the H100 processors on which the DGX is built, and Amazon Web Services will soon join them, Nvidia said.

Later this year, enterprises that don’t want to buy their own DGX supercomputer will have the option of renting clusters of those H100 processors by the month through the DGX Cloud service, which will be hosted by Nvidia’s hyperscaler partners.

CIOs are likely already using hardware, software, and perhaps services from Nvidia to support AI-enabled applications in the enterprise, but the company’s move deeper into the as-a-service market raises new questions, said IDC’s Rau.

“Having Nvidia as a service provider will likely mean a single source of responsibility for the service and a familiar underlying solution architecture,” he said. “But relying on Nvidia for service and the underlying solution architecture offers the prospect of costly lock-in should other service and solution architecture providers innovate faster on some measure, like performance or cost.”

Artificial Intelligence, Cloud Management, Infrastructure Management, IT Leadership

Posted on Posted in Digital transformation, General topics Tagged with , , , , , , , , , , ,

GPU manufacturer Nvidia is expanding its enterprise software offering with three new AI workflows for retailers it hopes will also drive sales of its hardware accelerators.

The workflows are built on Nvidia’s existing AI technology platform. One tracks shoppers and objects across multiple camera views as a building block for cashierless store systems; one aims to prevent ticket-switching fraud at self-service checkouts; and one is for building analytics dashboards from surveillance camera video.

Nvidia isn’t packaging these workflows as off-the-shelf applications, however. Instead, it will make them available for enterprises to integrate themselves, or to buy as part of larger systems developed by startups or third-party systems integrators.

“There are several of them out there, globally, that have successfully developed these kinds of solutions, but we’re making it easier for more software companies and also system integrators to build these kinds of solutions,” said Azita Martin, Nvidia’s VP of retail.

She expects that demand for the software will drive sales of edge computing products containing Nvidia’s accelerator chips, as latency issues mean the algorithms for cashierless and self-checkout systems need to be running close to the checkout and not in some distant data center.

In addition to tracking who is carrying what items out of the store, the multiple camera system can also recognize when items have been put back on the wrong shelf, directing staff to reshelve them so that other customers can find them and stock outages are avoided, she said.

“We’re seeing huge adoption of frictionless shopping in Asia-Pacific and Europe, driven by shortage of labor,” said Martin.

Nvidia will face competition from Amazon in the cashierless store market, though, since while Amazon initially developed its Just Walk Out technology for use in its own Amazon Go and Amazon Fresh stores, it’s now offering it to third-party retailers, too. The first non-Amazon supermarket to use the company’s technology opened in Kansas City in December.

Assessing cost control

The tool to prevent ticket switching is intended to be integrated with camera-equipped self-service point-of-sale terminals, augmenting them with the ability to identify the product being scanned and verify it matches the barcode.

The cost of training the AI model to recognize these products went beyond the usual spending on computing capacity.

“We bought tens of thousands of dollars of products like steak and Tide and beer and razors, which are the most common items stolen, and we trained these algorithms,” said Martin.

Nvidia kept its grocery bill under control using its Omniverse simulation platform. “We didn’t buy every size of Tide and every packaging of beer,” she adds. “We took Omniverse and created synthetic data to train those algorithms even further for higher accuracy.”

Beer presents a particular challenge for the image recognition system, as it often sells in different-size multipacks or in special-edition packaging associated with events like the Super Bowl. However, the system continues to learn about new product formats and packaging from images captured at the checkout.

While implementation will be left up to retailers and their systems integrators, Martin suggested the tool might be used to lock up a point-of-sale terminal when ticket switching is suspected, summoning a member of staff to reset it and help the customer rescan their items.

Nvidia is touting high accuracy for its algorithms, but it remains to be seen how this will work out in deployment.

“These algorithms will deliver 98% accuracy in detecting theft and shutting down the point of sale and preventing it,” she said.

But that still leaves a 2% false positive rate, so CIOs will want to carefully monitor the potential impact on profitability, customer satisfaction, and frequent resets to prevent ticket switching.

A $100 billion problem

A 2022 survey by the National Retail Federation found that inventory shrink amounted to 1.44% of revenue — a relatively stable figure over the last decade — and in 2021, losses due to shrink totaled almost $100 billion, the NRF estimated.

Of that, survey respondents said 26% was due to process or control failures, 29% due to employee or internal theft, and 37% due to external theft.

But Nvidia suggests that its loss prevention technology could eliminate 30% of shrinkage. That, though, would mean it could prevent four-fifths of all external retail theft, even though in addition to ticket switching, that category also includes shoplifting and organized retail crime activities such as cargo theft, and the use of stolen or cloned credit cards to obtain merchandise.

Plus, potential gains must be weighed against the cost of deploying the technology, which, Martin says, “depends on the size of the store, the number of cameras and how many stores you deploy it to.”

More positively, Nvidia is also offering AI workflows that can process surveillance camera video feeds to generate a dashboard of retail analytics, including a heatmap of the most popular aisles and hour-by-hour trends in customer count and dwell time. “All of this is incredibly important in optimizing the merchandising, how the store is laid out, where the products go, and on what shelves to drive additional revenue,” Martin said.

Artificial Intelligence, IT Strategy, Retail Industry