AI Hardware Performance Rankings

    Welcome to our AI Performance Rankings page. Here, we evaluate the components based on their AI processing power, measured in TOPS (Tera Operations Per Second) – a critical metric indicating the computational throughput, particularly for AI tasks.

    The first column shows peak performance for INT8/FP8 precision, which is the most widespread metric for evaluating AI inference capabilities. We exclude Sparsity to provide a more accurate reflection of AI performance in dense computation scenarios where sparsity optimizations may not be applicable. The second and third columns show the performance with Sparsity, and FP4 TFLOPS, when supported.

    For SoCs, the results reflect the peak performance of the integrated NPU. Additionally, it’s important to note that, according to Microsoft, a NPU must have at least 40 TOPS of AI computing power for the PC to be considered “AI-capable.”

    #TypeGPU / CPU (NPU)TOPS INT8/FP8
    No Sparsity
    TOPS INT8/FP8
    Sparsity
    TFLOPS FP4
    Sparsity
    Prices from...
    1.mobile NVIDIA GeForce RTX 5090 (Laptop)3637261452 $3818.09
    2.mobile NVIDIA RTX 5000 Ada Generation (Laptop)341682 $3579.00
    3.mobile NVIDIA GeForce RTX 4090 (Laptop)318636 $2349.00
    4.mobile NVIDIA GeForce RTX 5080 (Laptop)2815621124 $2249.99
    5.mobile NVIDIA GeForce RTX 4080 (Laptop)271542 $2099.00
    6.mobile NVIDIA RTX 4000 Ada Generation (Laptop)258516 $3098.00
    7.mobile NVIDIA Quadro RTX 6000 (Laptop)238 Check Price
    8.mobile NVIDIA GeForce RTX 5070 Ti (Laptop)209418836 $1299.93
    9.mobile NVIDIA RTX A5500 (Laptop)198 $3667.24
    10.mobile NVIDIA GeForce RTX 3080 Ti (Laptop)189378 $1499.99
    11.mobile NVIDIA RTX A5000 (Laptop)174348 $3399.99
    12.mobile NVIDIA GeForce RTX 5070 (Laptop)173346692 $1199.99
    13.mobile NVIDIA GeForce RTX 3080 (Laptop)168336 $1250.00
    14.mobile NVIDIA GeForce RTX 4070 (Laptop)160320 $760.15
    15.mobile NVIDIA GeForce RTX 2080 SUPER (Laptop)153 $2729.00
    16.mobile NVIDIA Quadro RTX 5000 (16GB GDDR6, Laptop)152 $7999.00
    17.mobile NVIDIA GeForce RTX 2080 (Laptop)150 $2599.00
    18.mobile NVIDIA RTX A4000 (Laptop)142284 $1669.00
    19.mobile NVIDIA GeForce RTX 3070 Ti (Laptop)140280 $1089.99
    20.mobile NVIDIA Quadro RTX 5000 Max-Q (16GB GDDR6)133 $3999.77
    21.mobile NVIDIA GeForce RTX 5060 (Laptop)133266532 $949.99
    22.mobile NVIDIA GeForce RTX 3070 (Laptop)133266 $999.99
    23.mobile NVIDIA Quadro RTX 4000 (Laptop)128 $1929.93
    24.mobile NVIDIA RTX 3500 Ada Generation (Laptop)127254 $2576.77
    25.mobile NVIDIA RTX 3000 Ada Generation (Laptop)125250 $2799.77
    26.mobile NVIDIA GeForce RTX 4060 (Laptop)116232 $809.99
    27.mobile NVIDIA GeForce RTX 2070 SUPER (Laptop)113 $1099.00
    28.mobile NVIDIA Quadro RTX 4000 Max-Q113 $2479.77
    29.mobile NVIDIA GeForce RTX 2070 (Laptop)107 $998.79
    30.mobile NVIDIA RTX 2000 Ada Generation (Laptop)105210 $1500.00
    31.mobile NVIDIA GeForce RTX 3060 (Laptop)105210 $699.00
    32.mobile NVIDIA RTX A3000 (Laptop)102204 $1969.77
    33.mobile NVIDIA GeForce RTX 4050 (Laptop)97194 $589.99
    34.mobile NVIDIA GeForce RTX 2060 (Laptop)96 $649.99
    35.mobile NVIDIA Quadro RTX 3000 (Laptop)86 $2008.07
    36.mobile NVIDIA Quadro RTX 3000 Max-Q85 $4176.96
    37.mobile NVIDIA RTX A2000 (Laptop)74148 $1159.77
    38.mobile NVIDIA GeForce RTX 3050 Ti (Laptop)69138 $699.99
    39.mobile NVIDIA RTX A1000 (Laptop)60 $743.38
    40.mobile NVIDIA RTX A500 (Laptop)58 $949.93
    41.mobile NVIDIA GeForce RTX 3050 (Laptop)57114 $554.85
    42.mobile NVIDIA GeForce RTX 2050 (Laptop)4896 $537.00
    43.mobile NVIDIA GeForce MX5704896 $970.77
    Subscribe
    Notify of
    guest
    7 Comments
    Inline Feedbacks
    View all comments
    Rado Petov
    Rado Petov
    1 year ago

     Great ranking, will there be any for phones ?

    DebuggingWizard
    DebuggingWizard
    1 year ago

    Is it a good idea to buy RTX 4090 for a server, instead of NVIDIA L40, given the much lower price, considering the similar performance?

    CacheFlow07
    CacheFlow07
    1 year ago

    This is gold, I hope that you’ll keep it updated!
    Please consider adding more info about the GPUs.

    Albert Jaeke
    Albert Jaeke
    11 months ago

    would love the option to exclude same gpus.
    instead of dozens of rtx4090, for example:

    nvidia 4090 (max) 800 400 …
    nvidia 4090 (min) 780 390 …

    Gfgjnbvg ghjn
    Gfgjnbvg ghjn
    5 months ago

    No AMD?