![]() The main config file will callĪll the supporting config files. Resolution, batch size, number of classes and other customization. The inference specific config files are used to specify models, inference The other supporting config files are for each individual This would set input source and resolution, number of The main config file configures all the high level In the installĭirectory, the config files are located in samples/configs/deepstream-app or There are typically 2 or more configuration files that are used with this app. The graphic below shows the architecture of the The referenceĪpplication is installed as deepstream-app. The app requires a primary objectĭetection model, followed by an optional secondary classification model. UsersĬan configure input sources, inference model, and output sinks. The nvinfer element handles everything related to TensorRT optimization and engine creationĭeepStream SDK ships with an end-to-end reference application which is fully configurable. etlt model file and optional calibration cache for INT8 precision.Ī labels.txt file containing the labels for classes in the order in which the networksĪ sample config_infer_*.txt file to configure the nvinfer element in DeepStream. The installation instructions for DeepStream are providedĪn exported. ![]() To integrate the models with DeepStream, you need the following:ĭownload and install DeepStream SDK. In the Prerequisites for YOLOv4-tiny Model section below, and the required code can be found in this The instructions to build a custom bounding-box parser are provided The instructions to build TensorRT open-source plugins are provided in the TensorRT Open Source Software (OSS) section above. DeepStream can also generate TensorRT engine on-the-fly for YOLOv4-tiny if onlyįor YOLOv4-tiny, you will need to build the TensorRT open-source plugins and custom bounding-box parser. The generated TensorRT engine file can thenīe ingested by DeepStream. Optimized TensorRT engine using TAO Deploy. To integrate a model trained by TAO Toolkit with DeepStream, you should generate a device-specific See the Exporting the Model section for more details on how to export a TAO model. TAO Converter, which is deprecated for x86 devices in TAO version 4.0.0 but is still required for etltįile to TensorRT this file is then provided directly to DeepStream. TensorRT engine generation can take some time depending on size of the modelĮngine generation can be done ahead of time with Option 2: TAO Deploy is used to convert the. DeepStream will automatically generate the TensorRT engine file and then run etlt file and calibration cache are directly ![]() Generated with a different version of TensorRT and CUDA is not supported and will cause unknownīehavior that affects inference speed, accuracy, and stability, or it may fail to run altogether. Libraries of the inference environment are updated (including minor version updates), or ifĪ new model is generated, new engines need to be generated. Machine-specific optimizations are done as part of the engine creation process, so a distinctĮngine should be generated for each environment and hardware configuration. Option 3 (Deprecated for x86 devices): Generate a device-specific optimized TensorRT engine using TAO Converter. The generated TensorRT engine file can also be ingested by DeepStream. Option 2: Generate a device-specific optimized TensorRT engine using TAO Deploy. ![]() etlt model directly in the DeepStreamĪpp. To deploy a model trained by TAO Toolkit to DeepStream we have two options: This section will describe how to deploy your trained model to DeepStream SDK. Has been designed to integrate with DeepStream SDK, so models trained with TAO Toolkit will work out of the boxĭeepStream SDK is a streaming analytic toolkit to accelerate building AI-based videoĪnalytic applications. Such as a Jetson Xavier or Jetson Nano, a discrete GPU, or in the cloud with NVIDIA GPUs. The deep learning and computer vision models that you’ve trained can be deployed on edge devices, ![]()
0 Comments
Leave a Reply. |