In recent months, the landscape of artificial intelligence (AI) trading has witnessed a significant shift, with the focus moving towards custom Application-Specific Integrated Circuits (ASICs). Market analysts speculate that the potential growth of ASICs could far surpass that of commercially available Graphics Processing Units (GPUs). This has had a notable impact on companies such as NVIDIA, which has seen a stagnation in its stock price, while Advanced Micro Devices (AMD) has also struggled to maintain its position in the marketHowever, a report released by Morgan Stanley paints a different picture, asserting that the expectations surrounding ASICs may be excessively optimistic, and suggests that ASICs may not possess the long-term capability to disrupt the market dominance currently held by GPUs.

On February 12, Morgan Stanley's lead analyst Joseph Moore, along with his team, published findings that equate ASICs and GPUs as two different avenues leading to the same outcome, rather than one being inherently superior to the otherMoore argues that while ASICs indeed show impressive performance in specialized applications, they remain heavily reliant on the unique customization demands of specific clientsThe development costs associated with ASICs may be lower, but their overall system and software deployment expenses can be significantly higher, resulting in a total cost of ownership (TCO) that may exceed that of widely deployable GPUs.

The CUDA ecosystem developed by NVIDIA has matured into a robust framework, allowing clients to deploy and manage various workloads with relative easeThis contributes further to the overall ownership cost for enterprises considering an investment in ASICsAccording to Morgan Stanley, unless there are extraordinary shifts in the current market dynamics, NVIDIA is likely to maintain its leadership position in the semiconductor industry.

When comparing ASICs to commercial GPUs, it becomes clear that the application scope of custom ASICs is rather narrow

Advertisements

Morgan Stanley indicates that ASICs can indeed be appealing in niche scenarios—especially when tailored for specific cloud service providers or enterprise customersThis niche specialization can yield greater performance and efficiency, allowing ASICs to maintain a competitive edge in certain market segmentsA prime example of such success is Google's Tensor Processing Unit (TPU), which stemmed from the development of the transformative Transformer technology that Google pioneered, alongside collaboration with Broadcom to create a chip specifically engineered for optimizing these applicationsThis partnership has since brought in revenues exceeding $8 billion for Broadcom.

Nonetheless, NVIDIA is actively working to enhance its GPUs in response to the evolving needs of the market, particularly to cater to transformative models like GPTIn many sectors of cloud computing, commercially available GPUs tend to outshine ASICs in terms of competitivenessLooking into the future, Morgan Stanley predicts that the advantages conferred by custom ASICs might only hold sway in traditional workloadsThis is particularly relevant given that NVIDA’s focus increasingly lies on training multi-modal artificial general intelligence (AGI) models, which may prove to be excessive for certain legacy applications.

However, it is worth noting that when it comes to delivering cutting-edge training capacities, NVIDIA appears to remain the unassailable leaderMorgan Stanley elaborates on the cost dynamics, focusing on how while ASICs might be cheaper on a hardware level, the associated systems costs can be considerably exorbitant, negating any initial financial advantages.

For instance, certain ASICs may come with hardware costs as low as $3,000, while NVIDIA's H100 GPU is priced around $20,000—a disparity that makes ASICs look financially alluring on the surfaceHowever, the operational costs for ASIC deployments can, in fact, escalate beyond those for general-purpose GPUs

Advertisements

As discussed in the report, the costs involved in creating an ASIC cluster can surpass the expensive setup NVIDIA employs, which is mainly built upon a copper-based NVLINK domain consisting of 72 GPUsThe optical technologies typically used by ASICs often introduce additional expenses that could tilt the scales significantly in favor of NVIDIA.

In terms of high-bandwidth memory (HBM), though the costs remain comparable, NVIDIA stands to gain more due to its monopolistic procurement powers on new HBM technologies, as well as control over CoWoS (Chip-on-Wafer-on-Substrate) packaging technologiesThe software framework is another significant aspect, with NVIDIA boasting a mature CUDA ecosystemThis readiness facilitates clients' ability to deploy and run various workloads almost seamlesslyBy contrast, adopting ASICs or alternative solutions may require clients to expend considerable time and additional resources on software adaptation, leading to higher total ownership costs.

As an illustration, Databricks anticipates that deploying systems using Trainium will take "weeks or even months" before they are fully operationalOne executive from a cloud service provider recently conveyed to Morgan Stanley that "every two years, the technology delivered by my ASIC team is 2-3 years behind that of NVIDIAFrom an economic perspective, this is not particularly beneficial."

Consequently, even with the introduction of lower-priced chips like the L4 and L40 by NVIDIA, the market still gravitates towards high-performance graphics cards that carry steeper price tags, given the significant advantages these GPUs offer in performance and ecosystem supportMorgan Stanley concludes that while many lower-cost processors may initially attract certain customers, a lack of a mature ecosystem and long-term support often drives these clients back towards NVIDIAThe TPU, Trainium, and AMD MI300 represent notable exceptions to this trendNonetheless, the overarching takeaway seems to be that cheaper processors often fail to realize the market share that initial expectations would suggest they would.

The market presence of NVIDIA remains solid, with Morgan Stanley highlighting that its dominance in AI chips results from a combination of formidable technological prowess and a well-developed ecosystem bolstered by sustained research and development investments

Advertisements

Advertisements

Advertisements