The Cutting Edge of AI… Everything Matters!

Running a marathon in under two hours requires shoes engineered for the task. AI is no different.

Growth of data is exponential. In five years, humans & machines will produce 10x more than we did this year. And more than 70% will be created at the Edge. Only half will move to public clouds — the rest will be processed, stored and analyzed at the Edge.

Why? Several reasons…bandwidth availability & cost, data sovereignty, security and the increasing need for real-time processing. Processing that volume of data, at that velocity, requires a new approach…an approach that is significantly more efficient than current options. Which is why we are creating purpose-built products & tools for our most demanding customers.

AI is rapidly being adopted at the Edge by companies of every size, across every industry — from Smart Cities to Industrial to Retail and Health & Life Sciences. Our Edge portfolio has been tremendously popular with customers and developers because it makes the existing infrastructure, investments, and experience they have on Intel more valuable.

Capturing the full benefit of AI requires making software open and easy, designing products & tools for the unique demands of the Edge, and investing in the next generation of talent.

Earlier this week at the Intel® AI Summit, I spoke about hardware and software innovations that not only accelerate Edge AI performance further, but do so while making performance easy to attain. We disclosed our next-gen Movidius Vision Processing Unit (VPU), codenamed Keem Bay. Keem Bay is a low power (4–15 watts), high-performance Edge Inferencing product purpose-built for Deep Learning, Vision & Media. This is the work of over 1,000 incredibly talented people from ten teams across seven countries. We have early customers testing it and will launch in the first half of 2020.

This architecture is highly optimized for Edge inference, with ground-breaking leaps forward in performance. And it’s a workhorse. With flexible form factors, customers can adopt it chip-down for cameras, on compact M.2 cards for things like kiosks and robotics, all the way up to full-power PCIe cards that will be able to run multiple VPUs in parallel for high-density, scalable Edge AI server acceleration. This will enable the full range of Edge experiences our customers are asking for.

Compared to alternatives in the market, Keem Bay really shines. In early testing, KMB is fast — up to 4X the performance of NVIDIA’s comparable TX2. Nvidia’s Xavier part is actually a tier above KMB — 30W and 5x the size — but KMB delivers there as well, being on par1 with raw performance @ only 1/5th power.

And that’s the key…at the Edge, raw performance is only part of the equation — customers also care about power, size and latency. Keem Bay shines here as well. From a power point of view, Keem Bay is “green” — delivering over 6x the inference performance per watt over NVIDIA’s TX2.

And it’s small: measured by inferences per mm2, Keem Bay is 8.7x the TX2; with Xavier also far behind.

And finally, it’s efficient, because it is engineered for Edge Inference. There are no wasted TOPS, with 4x the inferences per second per TOPS than NVIDIA’s Xavier.

Rather than taking products designed for another purpose, we are engineering specifically for Edge Inference.

Just as Nike’s Vaporfly helped Kipchoge break the 2 hour marathon record, purpose-built silicon will enable a new wave of AI at the Edge.

But it’s not just about silicon innovation — we believe in democratizing AI. Easy-to-use tools like our open source OpenVINO™ toolkit are key to a level playing field.

OpenVINO is a dev tool & runtime to enable customers to write once and deploy across a wide variety of Intel silicon at maximum performance. With OpenVINO, highly performant systems are accessible to people who don’t need to understand hardware architecture and electrical engineering.

We also launched a new companion to OpenVINO — the Intel Dev Cloud for the Edge. Here’s how it works:

First, using their favorite standard frameworks, developers run their model through the OpenVINO model optimizer to accelerate for the Edge on any Intel compute, automatically.

Next, because developers have asked how to know which Intel product will provide the right performance for their deep learning model use case, Dev Cloud for the Edge lets them test their models for free against the full range of Intel Edge silicon. It informs their hardware landing zone decision and lets them test before they buy. Over 2,700 customers have been using the beta — and loving it.

When they are ready to deploy, the OpenVINO runtime takes into account the capabilities of the full system and can allow customers to achieve automatic, software-defined parallelism by load balancing across CPU cores, iGPU, accelerator, or other hardware.

Building a solution or algorithm is only one step — scaling a solution or algorithm to the Edge takes an ecosystem of partners, and our AI: In Production program pairs solution developers, ODMs, System Integrators and more to enable easy deployment, at scale.

Oh, and one more thing…for too long deep learning courseware has been focused on AI in the cloud. We announced this week a partnership with Udacity to launch an Edge AI Nanodegree that will train the next generation of developers on how to do AI where the data is generated: at the Edge. We’ll be awarding 750 scholarships for this course.

So, key takeaways:

• Keem Bay’s groundbreaking architecture will deliver superior raw performance and efficiency when it launches 1H’20, at a fraction of the power, a fraction of the size & a fraction of the cost.
• It complements our full portfolio of products, tools and services purpose-built for the Edge.
• This portfolio — including OpenVINO and our new Dev Cloud for the Edge — makes AI accessible to everyone, not just the experts and companies with fleets of data scientists.
• And they can get started with the first Edge AI nanodegree we’ve launched with Udacity.
• Each of these products individually demonstrate extraordinary performance and value to our customers. When employed collectively, they are an unparalleled set of resources that aim to make Edge AI available to every customer, at every size, across every industry without massive investments in talent or capital.

Cheers —

JB