e-con Systems has proven expertise in integrating our cameras into the NVIDIA platform, including Jetson Xavier NX / Nano / TX2 NX, Jetson AGX Xavier, Jetson AGX Orin, and NVIDIA Jetson Orin NX / NANO. Find out about how our cameras are integrated into the NVIDIA platform, their popular use cases, and how they empower you to solve retail challenges.

In the retail industry, there are numerous challenges, including security risks, inventory management, and enhancing the shopping experience. NVIDIA-powered cameras are helping to address these challenges by providing retailers with real-time data and insights. In addition, these cameras are being used to enhance store security, optimize store layout and staffing, etc.

So, by leveraging the power of the NVIDIA platform, retailers can better understand their customers while improving operations and ultimately providing a more satisfying shopping experience.

In this blog, let’s discover more about the role of e-con Systems’ cameras integrated into the NVIDIA platform, how they help solve some major retail challenges, and their most popular use cases.

Read: e-con Systems launches 3D time of flight camera for NVIDIA Jetson AGX Orin and AGX Xavier

A quick introduction to NVIDIA and e-con Systems’ cameras

NVIDIA has been involved in developing camera sensors for various applications, focusing on AI-powered edge computing and autonomous vehicles. One of their most notable releases is the Jetson Nano Developer Kit (released in 2019). This System-on-Module (processor) is designed for AI-powered edge computing applications like object recognition and autonomous shopping.

As you may already know, e-con Systems has proven expertise in integrating our cameras into the Nvidia platform. We support the entire NVIDIA Jetson family, including Jetson Xavier NX / Nano / TX2 NX, Jetson AGX Xavier, Jetson AGX Orin, and NVIDIA Jetson Orin NX / NANO. e-con Systems’ popular camera solutions come with advanced features, such as dedicated ISP, ultra-low-light performance, low noise, wide temperature range, LED flicker mitigation, bidirectional control, and long-distance transmission.

Benefits of using cameras powered by the NVIDIA platform

• • They work seamlessly with their powerful GPUs, which are optimized for processing large amounts of data in real time. This allows for advanced image processing and analysis, making it possible for machines to “see” and understand their surroundings with greater accuracy and speed.
  • They are capable of capturing high-quality data that can be used to train deep neural networks. So they can then be used for tasks such as object detection and recognition.
  • They are designed to be low-power and compact, making them ideal for use in embedded vision applications. This is particularly important for applications such as smart trolleys and smart checkout systems.
  • They are highly customizable, letting developers tailor them to specific applications and use cases. This flexibility makes it possible to create embedded vision solutions that are optimized for specific tasks and environments, providing better performance and reliability.

Read: Popular embedded vision use cases of NVIDIA® Jetson AGX Orin

Major retail use cases of NVIDIA and e-con Systems

Smart Checkout

e-con Systems’ cameras, powered by the NVIDIA platform, are transforming smart checkout systems by enabling faster, more accurate, and more efficient checkout experiences for customers. Firstly, they can be used to enable contactless checkout, reducing the risk of transmission of infectious diseases. So, customers can avoid touching checkout equipment and interacting with cashiers, reducing the risk of transmission.

These smart checkout systems usually refer to a camera-enabled automated object detection system at the billing or checkout counter. They can operate autonomously with limited supervision from human staff – offering benefits like effective utilization of the retail staff, enhanced shopping experience, data insights on shopping patterns, and more. The integrated camera is equipped with smart algorithms to detect a wide variety of objects in a retail store.

Read: Key camera-related features of smart trolley and smart checkout systems

Smart Trolley

NVIDIA cameras are changing the game for retailers by providing real-time insights into customer behavior and preferences through the use of smart trolleys. These trolleys equipped with cameras and sensors help identify products or the barcode on each item – enabling the customers to pay in the same cart. This can greatly reduce wait times and improve overall customer satisfaction.

Moreover, the data collected by these cameras can enable retailers to offer personalized product recommendations and promotions based on past purchases and interactions. This personalized approach can increase sales and customer loyalty.

Another significant advantage of NVIDIA cameras in smart trolleys is enhanced store security. The cameras can detect and track suspicious activity in real time, such as items being removed from trolleys without payment or abandoned trolleys blocking store aisles.

Read: How embedded vision is contributing to the smart retail revolution

Other retail use cases include:

• • Optimized store operations and improved inventory management: With real-time data on store traffic and product placement, retailers can make informed decisions about store layout, staffing, and inventory management, leading to more efficient operations and reduced costs.
  • Personalized shopping experiences for customers: By analyzing customer behavior through imaging detail and preferences, retailers can offer personalized product recommendations and promotions. In turn, this leads to increased sales and customer satisfaction.

As the technology continues to evolve, it is likely that we will see even more innovative applications of NVIDIA-powered cameras in the retail industry.

NVIDIA and e-con Systems: An ongoing multi-year Elite partnership

NVIDIA and e-con Systems together have formed a one-stop ecosystem – providing USB, MIPI, GMSL, GigE, and FPD Link camera solutions across several industries and significantly reducing time-to-market. This multi-year Elite partnership started with Jetson Nano (40 TOPS) and continues strong with AGX Orin (100 TOPS).

Explore our NVIDIA Jetson-based cameras

If you are looking for an expert to help integrate NVIDIA cameras into your embedded vision products, please write to camerasolutions@e-consystems.com. You can also check out our Camera Selector page to get a full view of e-con Systems’ camera portfolio.

Ranjith Kumar
Camera Solution Architect, e-con Systems

The post How NVIDIA and e-con Systems are Helping Solve Major Challenges In the Retail Industry appeared first on Edge AI and Vision Alliance.

Built around SONY and PROPHESEE’s cutting-edge EVS technology, this developer kit simplifies the prototyping process and helps companies reduce time to market.

Event-based Vision Sensing (EVS)

Unlike conventional sensors that transmit all visible data in successive frames, the EVS sensor captures only the changed pixel data, specifically luminance changes. Each event package includes crucial information: pixel coordinates, timestamp, and polarity, resulting in efficient bandwidth usage.

By reducing the transmission of redundant data, this technology lowers energy consumption and optimizes processing capacities, reducing the cost of vision solutions.

EVS sensors provide high-speed and low-latency data output. They give outstanding results in monitoring vibration and movement in low-light conditions.

The FSM-IMX636 Development Kit consists of an IMX636 Event-based Vision Sensor board with a lens, all necessary adapters, accessories, and drivers, crafted into a comprehensive, easy-to-integrate solution for testing EVS in embedded applications systems on NVIDIA® Jetson AGX Xavier and NVIDIA® Jetson AGX Orin platforms.

The PROPHESEE Metavision® Intelligence Suite provides machine learning-supported event data processing, analytics, and visualization modules.

FRAMOS’ new Development Kit is an affordable, simple to use, and intelligent platform for testing, prototpying, and faster launch of diverse EVS-based applications in in a wide range of fields, including industrial automation, medical field, automotive and mobility, and IoT and monitoring.

For more information, visit this link.

About FRAMOS

FRAMOS® is the leading global expert in vision systems, dedicated to innovation and excellence in enabling devices to see and think.

For more than 40 years, the company has supported clients worldwide in building pioneering vision systems.

Throughout all phases of vision system development, from hardware and software solutions to component selection, customization, consulting, prototyping, and mass production, companies worldwide rely on FRAMOS proven expertise.

Thanks to its engineering excellence and a large base of loyal clients, the company operates successfully on three continents.

Over 180 experts working in Munich, Ottawa, Zagreb, and Čakovec offices commit themselves to developing cutting-edge imaging solutions for various applications across various industries.

For more information, please visit www.framos.com or follow us on LinkedIn, Facebook, Instagram or Twitter.

The post FRAMOS Launches Event-based Vision Sensing (EVS) Development Kit appeared first on Edge AI and Vision Alliance.

Leveraging Sony ISX031’s sub-pixel HDR technology, STURDeCAM31 has been fine tuned to provide impressive HDR performance of up to 120 dB and LFM, thereby providing a solution for capturing dynamic scenes without motion blur. With the reliable GMSL2 interface, this rugged camera meets IP69K standards, ensuring durability and protection against dust, water, high temperature, high pressure, heavy vibration and shock.

e-con Systems’ STURDeCAM31 is compatible with NVIDIA Jetson AGX Orin system on modules, offering synchronized multi-camera solutions that can support up to eight cameras through the GMSL2 interface. This powerful combination of STURDeCAM31 and NVIDIA Jetson Orin platform for edge AI and robotics is a game changer in the autonomous mobility industry, especially for ADAS, delivery robots, autonomous agriculture vehicles, etc.

“In a rapidly evolving market with surging demand for autonomous mobility in challenging outdoor lighting conditions, whether it be robotics or automotive vehicles, STURDeCAM31 emerges as the perfect fit. Through its superior HDR and LFM capabilities, we are transforming mobility and enhancing global safety. Our IP69K-rated cameras set a new benchmark for reliability, ruggedness, and performance, effortlessly enduring the rigors of high vibrations, shocks, dust, and water environments. In collaboration with NVIDIA, we stand at the forefront of pioneering a safer future through cutting-edge imaging technology.“ said Gomathi Sankar, Business Unit Head-Industrial Cameras at e-con Systems.

Key features of STURDeCAM31:

• 120 dB HDR – Leveraging sub-pixel HDR technology, it enables HDR performance of up to 120 dB and LFM – eradicating motion blur and the occurrence of underexposed or overexposed images.
• Synchronized Multi-Camera Support – It can connect up to 8 cameras to NVIDIA Jetson AGX Orin platform using GMSL2 interface.
• Designed for Automotive Standards – It withstands harsh environmental conditions – protecting against dust, water, high pressure, high temperature, vibration and shock.
• GMSL Link Monitoring – It ensures the safety and reliability of data transmission over the GMSL, preventing system failures, and enabling timely diagnostics for enhanced security in applications.

Availability

If you are interested in evaluating STURDeCAM31, please visit the online web store and purchase the product.

Customization and integration support

e-con Systems, with its deep expertise in and knowledge of various camera interfaces, provides the necessary customization services and end-to-end integration support for STURDeCAM31. It ensures that unique application requirements can be easily met. If you are looking for any customization or integration support, please write to us at camerasolutions@e-consystems.com.

About e-con Systems

e-con Systems designs, develops, and manufactures OEM cameras. With 20+ years of experience and expertise in embedded vision, it focuses on delivering vision and camera solutions to industries such as retail, medical, industrial, agriculture, smart city, etc. e-con Systems’ wide portfolio of products includes Time of Flight cameras, MIPI camera modules, GMSL cameras, USB 3.1 Gen 1 cameras, stereo cameras, low light cameras, etc. With a team of 450+ extremely skilled core engineers, our products are currently embedded in over 350 customer products. So far, we have shipped over 2 million cameras to the United States, Europe, Japan, South Korea and many more countries.

The post e-con Systems Launches Superior HDR Multi-camera Solution for NVIDIA Jetson Orin to Revolutionize Autonomous Mobility appeared first on Edge AI and Vision Alliance.

The camera module is the most integral part of an AI-based embedded system. With so many camera module choices on the market, the selection process may seem overwhelming. This post breaks down the process to help make the right selection for an embedded application, including the NVIDIA Jetson.

Camera selection considerations

Camera module selection involves consideration of three key aspects: sensor, interface (connector), and optics.

Sensor

The two main types of electronic image sensors are the charge-coupled device (CCD) and the active-pixel sensor (CMOS). For a CCD sensor, pixel values can only be read on a per-row basis. Each row of pixels is shifted, one by one, into a readout register. For a CMOS sensor, each pixel can be read individually and in parallel.

CMOS is less expensive and consumes less energy without sacrificing image quality, in most cases. It can also achieve higher frame rates due to the parallel readout of pixel values. However, there are some specific scenarios in which CCD sensors still prevail—for example, when long exposure is necessary and very low-noise images are required, such as in astronomy.

Electronic shutter

There are two options for the electronic shutter: global or rolling. A global shutter exposes each pixel to incoming light at the same time. A rolling shutter exposes the pixel rows in a certain order (top to bottom, for example) and can cause distortion (Figure 1).

Figure 1. Distortion of rotor blades caused by rolling shutter

The global shutter is not impacted by motion blur and distortion due to object movement. It is much easier to sync multiple cameras with a global shutter because there is a single point in time when exposure starts. However, sensors with a global shutter are much more expensive than those with a rolling shutter.

Color or monochrome

In most cases, a monochrome image sensor is sufficient for typical machine vision tasks like fault detection, presence monitoring, and recording measurements.

With a monochrome sensor, each pixel is usually described by eight bits. With a color sensor, each pixel has eight bits for the red channel, eight bits for the green channel, and eight bits for the blue channel. The color sensor requires processing three times the amount of data, resulting in a higher processing time and, consequently, a slower frame rate.

Dynamic range

Dynamic range is the ratio between the maximum and minimum signal that is acquired by the sensor. At the upper limit, pixels appear white for higher values of intensity (saturation), while pixels appear black at the lower limit and below. An HDR of at least 80db is needed for indoor application and up to 140db is needed for outdoor application.

Resolution

Resolution is a sensor’s ability to reproduce object details. It can be influenced by factors such as the type of lighting used, the sensor pixel size, and the capabilities of the optics. The smaller the object detail, the higher the required resolution.

Pixel resolution translates to how many millimeters each pixel is equal to on the image. The higher the resolution, the sharper your image will be. The camera or sensor’s resolution should enable coverage of a feature’s area of at least two pixels.

CMOS sensors with high resolutions tend to have low frame rates. While a sensor may achieve the resolution you need, it will not capture the quality images you need without achieving enough frames per second. It is important to evaluate the speed of the sensor.

A general rule of thumb to determine the resolution needed for the use case is shown below and in Figure 2.  The multiplier (2) represents the typical desire to have a minimum two pixels on an object in order to successfully detect it.

INSERT GRAPHIC
Figure 2. Sensor resolution required is determined by lens field of view and feature of interest size

For example, suppose you have an image of an injury around the eye of a boxer.

• 
• FOV, mm = 2000mm
• Size of feature of interest (the eye), mm = 4mm

Based on the calculation, 1000 x 1000, a one-megapixel camera should be sufficient to detect the eye using a CV or AI algorithm.

Note that a sensor is made up of multiple rows of pixels. These pixels are also called photosites. The number of photons collected by a pixel is directly proportional to the size of the pixel. Selecting a larger pixel may seem tempting but may not be the optimal choice in all the cases.

Table 1.  Pros and cons of small and large pixel size

Back-illuminated sensors maximize the amount of light being captured and converted by each photodiode. In front-illuminated sensors, metal wiring above the photodiodes blocks off some photons, hence reducing the amount of light captured.

Figure 3. Cross-section of a front-illuminated structure (left) and a back-illuminated structure (right)

Frame rate and shutter speed

The frame rate refers to the number of frames (or images captured) per second (FPS). The frame rate should be determined based on the number of inspections required per second. This correlates with the shutter speed (or exposure time), which is the time that the camera sensor is exposed to capture the image.

Theoretically, the maximum frame rate is equal to the inverse of the exposure time. But achievable FPS is lower because of latency introduced by frame readout, sensor resolution, and the data transfer rate of the interface including cabling.

FPS can be increased by reducing the need for large exposure times by adding additional lighting, binning the pixels.

CMOS sensors can achieve higher FPS, as the process of reading out each pixel can be done more quickly than with the charge transfer in a CCD sensor’s shift register.

Interface

There are multiple ways to connect the camera module to an embedded system. Typically, for evaluation purposes, cameras with USB and Ethernet interfaces are used because custom driver development is not needed.

Other important parameters for interface selection are transmission length, data rate, and operating conditions. Table 2 lists the most popular interfaces. Each option has its pros and cons.

Table 2. Comparison of various camera interfaces

Optics

The basic purpose of an optical lens is to collect the light scattered by an object and recreate an image of the object on a light-sensitive image sensor (CCD or CMOS). The following factors should be considered when selecting an optimized lens-focal length, sensor format, field of view, aperture, chief ray angle, resolving power, and distortion.

Lenses are manufactured with a limited number of standard focal lengths. Common lens focal lengths include 6mm, 8mm, 12.5mm, 25mm, and 50mm.

Once you choose a lens with a focal length closest to the focal length required by your imaging system, you need to adjust the working distance to get the object under inspection in focus. Lenses with short focal lengths (less than 12mm) produce images with a significant amount of distortion.

If your application is sensitive to image distortion, try to increase the working distance and use a lens with a higher focal length. If you cannot change the working distance, you are somewhat limited in choosing an optimized lens.

Table 3. Main types of camera lenses

To attach a lens to a camera requires some type of mounting system. Both mechanical stability (a loose lens will deliver an out-of-focus image) and the distance to the sensor must be defined.

To ensure compatibility between different lenses and cameras, the following standard lens mounts are defined.

Table 4. Common lens mounts used in embedded space

NVIDIA camera module partners

NVIDIA maintains a rich ecosystem of partnerships with highly competent camera module makers all over the world. See Jetson Partner Supported Cameras for details. These partners can help you design imaging systems for your application from concept to production for the NVIDIA Jetson.

Figure 4. NVIDIA Jetson in combination with camera modules can be used across industries for various needs

Summary

This post has explained the most important camera characteristics to consider when selecting a camera for an embedded application. Although the selection process may seem daunting, the first step is to understand your key constraints based on design, performance, environment, and cost.

Once you understand the constraints, then focus on the characteristics most relevant to your use case. For example, if the camera will be deployed away from the compute or in a rugged environment, consider using the GMSL interface. If the camera will be used in low-light conditions, consider a camera module with larger pixel and sensor sizes. If the camera will be used in a motion application, consider using a camera with a global shutter.

To learn more, watch Optimize Your Edge Application: Unveiling the Right Combination of Jetson Processors and Cameras. For detailed specs on AI performance, GPU, CPU, and more for both Xavier and Orin-based Jetson modules, visit Jetson Modules.

Related resources

• GTC session: Real-Time Onboard Processing for Autonomous Indoor Drones using NVIDIA Jetson (Spring 2023)
• GTC session: Jetson Edge AI Developer Days: Getting the Most Out of Your Jetson Orin Using NVIDIA Nsight Developer Tools (Spring 2023)
• SDK: JetPack SDK
• Webinar: Implementing Computer Vision and Image Processing Solutions with VPI
• Webinar: ACCELERATE COMPUTER VISION AND IMAGE PROCESSING USING VPI 1.1
• Webinar: JetPack 5.0 Deep Dive: New Kernel, Bootloader, Trusted OS and Support for Jetson AGX Orin

Vikas Sharma
Product Manager, NVIDIA

The post Selecting the Right Camera for the NVIDIA Jetson and Other Embedded Systems appeared first on Edge AI and Vision Alliance.

“Open Standards Unleash Hardware Acceleration for Embedded Vision,” a Presentation from the Khronos Group

Register or sign in to access this content.

Registration is free and takes less than one minute. Click here to register and get full access to the Edge AI and Vision Alliance's valuable content.

The post “Open Standards Unleash Hardware Acceleration for Embedded Vision,” a Presentation from the Khronos Group appeared first on Edge AI and Vision Alliance.

Gobin shows the Seeed Wio Lite AI board based on a STM32H7 microcontroller. The STM32H7 runs a real-time person-detection algorithm created with NVIDIA’s TAO Toolkit and optimized with the STM32Cube.AI Developer Cloud. When the TAO-enabled model on the STM32H7 sees people present, it wakes a NVIDIA Embedded Jetson Orin to do the much more computationally intensive people-recognition task. Find more information at https://stm32ai.st.com.

The post STMicroelectronics Demonstration of the Cube.ai Developer Cloud, from TAO to STM32 appeared first on Edge AI and Vision Alliance.

August 8, 2023—SIGGRAPH—NVIDIA today announced the next-generation NVIDIA GH200 Grace Hopper platform — based on a new Grace Hopper Superchip with the world’s first HBM3e processor — built for the era of accelerated computing and generative AI.

Created to handle the world’s most complex generative AI workloads, spanning large language models, recommender systems and vector databases, the new platform will be available in a wide range of configurations.

The dual configuration — which delivers up to 3.5x more memory capacity and 3x more bandwidth than the current generation offering — comprises a single server with 144 Arm Neoverse cores, eight petaflops of AI performance and 282GB of the latest HBM3e memory technology.

“To meet surging demand for generative AI, data centers require accelerated computing platforms with specialized needs,” said Jensen Huang, founder and CEO of NVIDIA. “The new GH200 Grace Hopper Superchip platform delivers this with exceptional memory technology and bandwidth to improve throughput, the ability to connect GPUs to aggregate performance without compromise, and a server design that can be easily deployed across the entire data center.”

The new platform uses the Grace Hopper Superchip, which can be connected with additional Superchips by NVIDIA NVLink, allowing them to work together to deploy the giant models used for generative AI. This high-speed, coherent technology gives the GPU full access to the CPU memory, providing a combined 1.2TB of fast memory when in dual configuration.

HBM3e memory, which is 50% faster than current HBM3, delivers a total of 10TB/sec of combined bandwidth, allowing the new platform to run models 3.5x larger than the previous version, while improving performance with 3x faster memory bandwidth.

Growing Demand for Grace Hopper

Leading manufacturers are already offering systems based on the previously announced Grace Hopper Superchip. To drive broad adoption of the technology, the next-generation Grace Hopper Superchip platform with HBM3e is fully compatible with the NVIDIA MGX server specification unveiled at COMPUTEX earlier this year. With MGX, any system manufacturer can quickly and cost-effectively add Grace Hopper into over 100 server variations.

Availability

Leading system manufacturers are expected to deliver systems based on the platform in Q2 of calendar year 2024.

Watch Huang’s SIGGRAPH keynote address on demand to learn more about Grace Hopper.

About NVIDIA

Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling industrial digitalization across markets. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry. More information at https://nvidianews.nvidia.com/.

The post NVIDIA Unveils Next-generation GH200 Grace Hopper Superchip Platform for Era of Accelerated Computing and Generative AI appeared first on Edge AI and Vision Alliance.

Vision Transformers (ViTs) are taking computer vision by storm, offering incredible accuracy, robust solutions for challenging real-world scenarios, and improved generalizability. The algorithms are playing a pivotal role in boosting computer vision applications and NVIDIA is making it easy to integrate ViTs into your applications using NVIDIA TAO Toolkit and NVIDIA L4 GPUs.

How ViTs are different

ViTs are machine learning models that apply transformer architectures, originally designed for natural language processing, to visual data. They have several advantages over their CNN-based counterparts and the ability to perform parallelized processing of large-scale inputs. While CNNs use local operations that lack a global understanding of an image, ViTs provide long-range dependencies and global context. They do this effectively by processing images in a parallel and self-attention-based manner, enabling interactions between all image patches.

Figure 1 shows see the processing of an image in a ViT model, where the input image is divided into smaller fixed-size patches, which are flattened and transformed into sequences of tokens. These tokens, along with positional encodings, are then fed into a transformer encoder, which consists of multiple layers of self-attention and feed-forward neural networks.

Figure 1. Processing an image in a ViT model that includes positional embedding and encoder (inspired by the study Transformers for Image Recognition at Scale)

With the self-attention mechanism, each token or patch of an image interacts with other tokens to decide which tokens are important. This helps the model capture relationships and dependencies between tokens and learns which ones are considered important over others.

For example, with an image of a bird, the model pays more attention to important features, such as the eyes, beak, and feathers rather than the background. This translates into increased training efficiency, enhanced robustness against image corruption and noise, and superior generalization on unseen objects.

Why ViTs are critical for computer vision applications

Real-world environments have diverse and complex visual patterns. The scalability and adaptability of ViTs enable them to handle a wide variety of tasks without the need for task-specific architecture adjustments, unlike CNNs.

Figure 2. Different types of imperfect and noisy real-world data create challenges for image analysis.

In the following video, we compare noisy videos running both on a CNN-based model and ViT-based model. In every case, ViTs outperform CNN-based models.

Learn about SegFormer, a ViT model that generates robust semantic segmentation while maintaining high efficiency

Integrating ViTs with TAO Toolkit 5.0

TAO, a low-code AI toolkit to build and accelerate vision AI models, now makes it easy to build and integrate ViTs into your applications and AI workflow. Users can quickly get started with a simple interface and config files to train ViTs, without requiring in-depth knowledge of model architectures.

The TAO Toolkit 5.0 features several advanced ViTs for popular computer vision tasks including the following.

Fully Attentional Network (FAN)

As a transformer-based family of backbones from NVIDIA Research, FAN achieves SOTA robustness against various corruptions as highlighted in Table 1. This family of backbones can easily generalize to new domains, fighting noise and blur. Table 1 shows the accuracy of all FAN models on the ImageNet-1K dataset for both clean and corrupted versions.

Table 1: Size and accuracy for FAN models

Global Context Vision Transformer (GC-ViT)

GC-ViT is a novel architecture from NVIDIA Research that achieves very high accuracy and compute efficiency. It addresses the lack of inductive bias in vision transformers. It also achieves better results on ImageNet with a smaller number of parameters through the use of local self-attention, which combined with global self-attention can give much better local and global spatial interactions.

Table 2: Size and accuracy for GC-ViT models

Detection transformer with improved denoising anchor (DINO)

DINO is the newest generation of detection transformers (DETR) with faster training convergence compared to other ViTs and CNNs. DINO in the TAO Toolkit is flexible and can be combined with various backbones from traditional CNNs, such as ResNets, and transformer-based backbones like FAN and GC-ViT.

Figure 3. – Comparing the accuracy of DINO with other models

Segformer

Segformer is a lightweight and robust transformer-based semantic segmentation. The decoder is made of lightweight multi-head perception layers. It avoids using positional encoding (mostly used by transformers), which makes the inference efficient at different resolutions.

Powering efficient transformers with NVIDIA L4 GPUs

NVIDIA L4 GPUs are built for the next wave of vision AI workloads. They’re powered by the NVIDIA Ada Lovelace architecture, which is designed to accelerate transformative AI technologies.

L4 GPUs are suitable for running ViT workloads with their high compute capability of FP8 485 TFLOPs with sparsity. FP8 reduces memory pressure when compared to larger precisions and dramatically accelerates AI throughput.

The versatility and energy-efficient L4 with single-slot, low-profile form factor makes it ideal for vision AI deployments, including edge locations.

Watch this Metropolis Developer Meetup on-demand to learn more about ViTs, NVIDIA TAO Toolkit 5.0, and L4 GPUs.

Related resources

• GTC session: AI Models Made Simple Using TAO (Spring 2023)
• GTC session: Solving Computer Vision Grand Challenges in One-Click (Spring 2023)
• SDK: TAO Toolkit
• Webinar: Unleash the Power of Vision Transformers, NVIDIA TAO, and the Latest NVIDIA GPUs
• Webinar: Accelerate AI Model Creation Using AutoML in NVIDIA TAO 4.0
• Webinar: Designing Efficient Vision Transformer Networks for Autonomous Vehicles

Debraj Sinha
Product Marketing Manager for Metropolis, NVIDIA

Chintan Shah
Senior Product Manager for AI Products, NVIDIA

Ayesha Asif
Product Marketing Manager for deep learning products, NVIDIA

The post Improve Accuracy and Robustness of Vision AI Apps with Vision Transformers and NVIDIA TAO appeared first on Edge AI and Vision Alliance.

Check out the Edge Impulse documentation for more information on how to get started with NVIDIA TAO in your Edge Impulse project!

Get to market faster

The integration of NVIDIA TAO facilitates the building of efficient models faster by combining the power of transfer learning and the latest NVIDIA TAO models. These can be deployed across the entire Edge Impulse ecosystem of devices, silicon, and sensors. This comprehensive integration simplifies the edge AI development process and reduces time-to-market, giving you a critical competitive edge in the rapidly evolving AI landscape.

From MCUs to GPUs: All-in-one edge AI solution

Our all-in-one solution enables you to collect data, train and validate your models, and optimize libraries to run on any edge device. The platform scales from extremely low-power MCUs to efficient Linux CPU targets, GPUs, and NPUs. The result is a seamless, flexible solution tailored to the specific needs of edge AI development.

Fast track to enterprise-grade production

NVIDIA TAO comes packed with over 100 NVIDIA-optimized model architectures, like transformers and fully attentional networks, allowing developers to get a jump-start on their model development. You can quickly fine-tune these models with your own data, enabling a faster, more streamlined development process and a quicker path to enterprise-grade production.

Do more with less data

The integration also enhances data collection from any edge device, improving efficiency and usability. Coupled with Edge Impulse’s auto-labeling tools, you can boost data quality and streamline the labeling process. This allows for robust model training even with less data, reducing resources required and accelerating development.

Optimized for edge devices

Our joint solution allows you to profile the performance of your model on different hardware configurations. You can easily identify the optimal target given your specific use case and hardware constraints. This added level of customization ensures your edge AI solution is perfectly optimized for peak performance.

Collaborate with ease

The Edge Impulse Studio, bolstered by NVIDIA TAO, provides an edge AI development environment that promotes real-time, enterprise-wide collaboration. With an emphasis on team-based development, our platform enables teams with diverse expertise to collaborate from anywhere in real-time. Collaboration has never been easier or more effective.

In conclusion, the integration of NVIDIA TAO into Edge Impulse amplifies our commitment to providing a cutting-edge development platform for the AI sector. Our mission is to help you build robust, high-performance AI models quickly and efficiently, without compromising on customization and flexibility. We can’t wait to see what our developer community will create with this powerful new toolset.

Be sure to visit the Edge Impulse and NVIDIA TAO Toolkit landing page for more information!

Jenny Plunkett
Senior Developer Relations Engineer, Edge Impulse

The post Supercharge Edge AI with NVIDIA TAO on Edge Impulse appeared first on Edge AI and Vision Alliance.

Click here to buy and evaluate the NeduCAM25_CUOAGX for your application.

So, why choose this FPD Link III camera? Here are some of its remarkable features and benefits:

• Synchronized multi-camera support: NeduCAM25_CUOAGX harnesses the power of FPD-Link III technology – enabling you to effortlessly synchronize 8 camera modules up to a distance of 15m. You can also leverage e-con Systems’ proprietary 180-degree stitching algorithm to ensure an extended field of view.
• Seamless integration: NeduCAM25_CUOAGX has been meticulously designed to seamlessly integrate with the powerful edge AI computing platform NVIDIA AGX Orin platform to ensure maximum performance, no matter the complexity of the application.
• Exceptional imaging quality: NeduCAM25_CUOAGX is equipped with the advanced AR0234 global shutter sensor to deliver exceptional image quality. It captures images of fast-moving objects without rolling shutter artifacts and comes with an optional IP67 enclosure.

Interested in exploring the full potential of NeduCAM25_CUOAGX and its impressive features? Download the datasheet here.

The post FPD-Link III Camera for NVIDIA Jetson AGX Orin Development Kit appeared first on Edge AI and Vision Alliance.