GPU Direct – Better than Zero copy
Zero copy minimizes the CPU and memory bandwidth utilization by writing to memory only once but we can avoid that transfer altogether by writing directly to the GPU – this is called GPU Direct. And it makes sense in many performance applications to send data directly to the GPU for processing and then taking the lower bandwidth results to the CPU and memory for user or system interaction. Emergent has been supporting GPU Direct with Nvidia GPUs on Windows and Linux for over four years in a variety of applications. Nvidia RTXA6000/5000/4000, Orin, and Xavier are used in many applications using Emergent cameras. Unfortunately for RDMA users, Nvidia/Mellanox only allow GPU Direct on Windows to select partners such as Emergent and this OS is where 80% of machine vision applications continue to be deployed. Linux, however, does remain an option for RDMA with GPU Direct for all.
Integrated interface and processing cards
Zero copy is great. GPU Direct improves on this. But it would be the ultimate achievement if we received and processed the data from the cameras all on one card. In this case, CPU, memory and all server resources are not used at all. Emergent is supporting AMD/Xilinx Alveo cards for this very purpose and have multiple performance applications leveraging this technology. Emergent also is working closely with Nvidia to bring Bluefield NIC support. Think of Bluefield as the merging of Nvidia NICs with Nvidia GPUs. In both cases, the computer can be a very low end PC which primarily supplies power to the chosen card.
Multicast
While not in use by all applications, many of Emergent’s largest deployments utilize the Multicast feature of the Ethernet standard. Point-to-point protocols like TCP and RDMA cannot support multicasting by their nature. RDMA does have other modes it can operate in which essentially remove its flow control and packet retransmission feature – this is tantamount to the current GVSP standard. Two primary benefits of multicast exist: redundancy and distributed processing. Redundancy allows critical systems to have the fastest fail over to avoid downtime. In larger systems, switches are present and back-up servers can be setup to take over camera streams when one server has a problem. Distributed processing is especially important as the number and speed of cameras is increased and also very much dependent on the type of processing required. Some applications will simply take multicast camera data to another system for display while the heavy processing is done in other systems. Even on the same server the switch can send virtually Zero delay copies to different GPUs for parallel processing. It is nice to start with a technology that allows for such an architecture even if not immediately required. One representative deployment is with the 240 Emergent Bolt HB-25000SB 25GigE 25MP cameras running at 90fps across six mid-range servers – that is 40x 25GigE cameras per mid-range server which is unparalleled and miles ahead of any solution out there.
The big picture
Many camera manufacturers focus their attention on enabling the transfer of data from camera to receiver. They claim success once sensor data has arrived in system memory, leaving the integrator or customer with responsibility to manage that data into processing nodes. In some applications, system memory and the CPU are sufficient for managing and processing the incoming data stream(s) particularly when post-processing can be employed. In others however, particularly where multiple 10GigE, 25GigE or 100GigE streams are being used, real-time processing requires the use of offload technologies to more adequate processing nodes. In the concepts and proposals for alternate interface or protocol methods, this seldom comes up. We need to see the big picture. Over the past 15 years, Emergent has pioneered and developed 10GigE, 25GigE, and 100GigE area and line scan cameras and created an eco-system to support the most reliable highest speed imaging applications.
