NVIDIA L4 24GB Review The Versatile AI Inference Card

Aug 03, 2023

Today we get a fun review, taking a look at the NVIDIA L4. While not NVIDIA’s fastest GPU by any means, the L4 is going to be a popular card. We know this, because it is effectively an update to the NVIDIA Tesla T4 (then they were still “Tesla” branded) we reviewed in 2019. The NVIDIA T4 has been wildly popular, and we expect the L4 to be equally so building on a proven format.

Before we get to the review, we just wanted to give a quick thanks to PNY for actually getting us a card for review. NVIDIA GPUs are very popular and PNY is NVIDIA’s major partner for professional cards. With that, let us get to the hardware.

The NVIDIA L4 is a low-profile, half-height GPU. That is perhaps the most important spec of the card since it allows the card to be installed in all kinds of servers.

The backplate is another small but important feature. These often get installed in risers that are removed for service. Having the backplate, helps ensure the card stays safe during service.

With NVIDIA’s Ampere and later design elements, the gold card is easily recognizable. Passive cooling on the lower-power card also helps ensure it can be used in a wide array of systems since it can just use chassis airflow.

Since this is a GPU designed primarily for AI inference (although there are some other data center GPU use cases it services) we do not have display outputs. As these cards get densely packed in servers due to their small size, having more display outputs can be an issue in some OSes as we found many years ago doing 8x GPU systems.

The new design has retention bracket mounting holes on the front of the card. One item that is not present is a GPU power connector. Being solely PCIe slot powered is another feature that helps this class of GPU be easily integrated into many types of servers.

Next, given the NVIDIA (Tesla) T4 popularity, we wanted to do a quick side-by-side so you can see differences given how the ease of integration into servers is a major feature.

Here are the two cards. We did not have a T4 with a low profile bracket. The four we found all had full height. The low-profile brackets were all installed in data centers.

Here is the back side. We can see the NVIDIA L4 has much less NVIDIA branding than the old “NVIDIA Tesla T4”.

Both are a similar size, however, the ribbing on the L4 seems to make the front of the card slightly wider.

Here is a look at the airflow view if that helps. One can see that the new card has an improved thermal solution.

Next, let us get to key specs and then performance and power.