How To Use SDK (C++)

1. Overview

• This tutorial is an introduction of how to use our SDK (C++ version). This SDK toolkit is designed to assist engineers in AI visual development. To engage in software development, you need to download it from our official website:
  1. Model files trained on our official website
  2. SDK (C++ version) toolkit

2. Environment setup

• Supported OS versions: Windows 10/11 X64 platform

This SDK toolkit only works on Nvidia GPUs. AMD and other brands are not supported!

• Supported GPU types: NVIDIA GPUs with compute capability "6.1", "7.5", and "8.6" and minimum 2GB GPU memory. For specific GPU type, please refer to the "Select Your Hardware" section in the "Model Download" page on our official website.

You can find the compute capability of NVIDIA GPUs.

• Supported Nvidia driver versions: cuda version >= 11.6. (To check your cuda version, please enter nvidia-smi.exe in your command prompt)

You can download Nvidia drivers.

• Environment Dependencies: Visual Studio 2017 or above. You can download Visual Studio  
• After downloading Visual Studio, proceed with the installation and make sure to select "Desktop development with C++" and "Universal Windows Platform development" options.
• If you have already downloaded Visual Studio, please follow these steps:

a. Click on "Tools" and "Get Tools and Features" to navigate to the installing page.

b. In the "Workloads" tab, check "Desktop development with C++" and "Universal Windows Platform development", and click on "Modify."

3. Quick start with SDK

• Here is an example (Visual Studio 2022) to help you quickly understand how to use the SDK (C++ version) software development.

3.1 Create a new project

• Open Visual Studio and click on "Create a new project".

• Select "C++", "Windows", "Console", and "Empty Project". Click "Next."

• Now let's configure our project. Change the project name and location, and click "Create." In our example, our project is called demo_cpp.

3.2 Change solution configurations and platforms

• In "Solution Configurations," select "Release."

• In "Solution Platforms," select "x64."

• Now your solution configuration and platform are "Release" and "x64."

3.3 Unzip the SDK toolkit

• Extract the downloaded SDK file to a secure location (please do not delete it because it will be referenced). Here is an example:

• Go back to Visual Studio. Under "Solution Explorer", right-click on your project name and then click on "Open Folder in File Explorer".

• After opening it, it should appear as shown below (let's refer to this current path as proj_root/ for now, as it will be used later):

• In the SDK folder, copy the demo.cpp file and paste it into the proj_root/ directory.

Paste into the proj_root/ directory.

3.4 Include demo in your project

• In "Solution Explorer," click on "Show All Files" icon.

• Find demo.cpp

• Right-click on it and click on "Include In Project."

• Now you have your demo.cpp in your project

3.5 Modify include directories

• In "Solution Explorer," click on "Show All Files" icon one more time to show following subsections in "Solution Explorer:" "References," "External Dependencies," "Header Files," "Resource Files," and "Source Files."

• Now, in "Solution Explorer," click on "Properties" icon.

• In "Property Pages," under "VC++ Directories," find "Include Directories," click on the downward arrow icon, and click on "Edit."

• Double click on the empty space and click on "..." icon.

• Select the include folder in the SDK directory and click on "Select Folder".

• Now, the empty spaces becomes your include folder path. Click on "OK" and click on "Apply" to save changes.

Click on "OK" and click on "Apply" to save changes.

3.6 Modify additional library directories and additional dependencies

• In "Property Pages," under "Linker" and "General," find "Additional Library Directories," click on the downward arrow icon, and click on "Edit."

• Double click on the empty space and click on "..." icon.

• Select the lib folder in the SDK directory and click on "Select Folder."

• Now, the empty spaces becomes your lib folder path. Click on "Ok."

• In "Property Pages", under "Linker" and "Input", find "Additional Dependencies", click on the downward arrow icon, and click on "Edit."

• Copy opencv_world454.lib and neuro_det_sdk.lib to the empty space and click on "OK."

• Now, the empty spaces becomes your lib files and click on "Apply" to save changes.

3.7 Modify preprocessor definitions

• In "Property Pages," under "C/C++" and "Preprocessor," find "Preprocessor Definitions", click on the downward arrow icon, and click on "Edit."

• Add _SILENCE_EXPERIMENTAL_FILESYSTEM_DEPRECATION_WARNING and click on "OK."

• Now click on "Apply" and "OK" to save changes and close "Property Pages."

3.8 Add environment variables

• Let's proceed with adding system environment variables. Open "This PC", right-click on an empty area, select "Properties", click on "Advanced system settings", and then click on "Environment Variables", Then, the following interface will appear. Select "Path" and click on "Edit."

• Click on "New" and then click on "Browse."

• Choose the bin/ folder in the SDK directory.

• Move this environment variable to the top.

Click "OK."

3.9 Restart Visual Studio

3.10 Setup License

• Place the License.txt file in the proj_root/ folder (Please refer to the current document for the accurate location to place the License.txt file).

For instructions on how to generate the License.txt file, please refer to ActivateLicense.

3.11 Modify demo code

• The demo.cpp file located in the proj_root/ directory provides an example of how to perform model prediction on test images. Users only need to make few simple modifications to the demo.cpp file.
• In the Main function, there are six lines with TODO: comments. They correspond to:
• image folder path (image_folder_path),
• model folder path (model_path),
• model name (model_name),
• device name (device_name),
• device index（device_index）,
• detect threshold (detect_thres).
• Please modify the values of these six variables.
• Note:
• You can freely choose the model_name but make sure it is unique.
• The device_name can only be either "cpu" or "cuda";
• device_index is 0 by default and you can modify it to select your GPU if you have multiple GPUs;
• detect_thres is used to screen detection targets and they will be discarded if their confidence level falls below this threshold.
• The range of detect_thres is [0, 1] and its default value is -1, which is 0.7.
• For example:

Test images folder path

Model folder path

• Apart from six TODO: comments in the Main function, there are also TODO: comments in other functions such as draw_box, draw_info, and display_results. Please carefully read these comments and modify the values or code according to your needs.

3.12 Run the program

• Now, let's run the program. Go back to Visual Studio, and then open the demo.cpp file. Click on "Start" to run the program.

• If the program runs successfully, the command prompt interface will display similar information as follows:

Test image folder path: D:\nb-ai\test_image
Model path: D:\nb-ai\model\seg_gpu
Model name: test
Device name: cuda
Device index: 0
Detection threshold: -1


The total number of test images: 63
input device: cuda
infer device: cuda
[2023-07-28 19:00:11.522] [logging] [info] [model.cpp:35] [DirectoryModel] Load model: "D:\nb-ai\model\seg_gpu"
[2023-07-28 19:00:11.662] [logging] [info] [inference.cpp:44] {
  ···
}
[2023-07-28 19:00:12.399] [logging] [warning] [trt_net.cpp:24] TRTNet: Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[2023-07-28 19:00:13.731] [logging] [info] [inference.cpp:56] ["img"] <- ["image"]
[2023-07-28 19:00:13.731] [logging] [info] [inference.cpp:67] ["post_output"] -> ["dets"]
Batch size: 1
Reading the image: D:\nb-ai\test_image\000B16302318AE4F083DC80A6EA69EF75014582.bmp
Image size (height, width): 2048 2448
results in the 0-th image:
  bbox_count=100
Model prediction takes 2.086 seconds

label:                              螺帽
label_index:                        0
confidential score:                 0.99967
(x0, y0):                           (149.44,10.4562)
(x1, y1):                           (580.739,481.435)
row_index:                          0
col_index:                          0
mask_width:                         434
mask_height:                        474
...

• The prediction results of test images will be displayed in a pop-up window, for example:

• Congratulations! Now, you have successfully completed C++ software development using the SDK.

4. Function Call Explanation

4.1 load_model()

• Definition: This function is used to load the model.
• Parameter Explanation：

• Return value: The function returns an integer variable. 0 indicates a successful execution, while other numbers represent error codes. The specific error codes are as follows:

4.2 get_batch()

• Definition: This function returns batch (the number of samples processed in a single iteration).
• Parameter Explanation：

• Return value: The function returns an integer variable, which is the batch of the model.

4.3 predict_model()

• Definition: This function is used to perform model prediction on test images
• Parameter Explanation：

• Return value: The function returns a class called PredictResult, which is shown below.

// Bounding box
// (x0, y0) coordinates represent the top left corner of the bounding box
// (x1, y1) coordinates represent the bottom right corner of the bounding box
struct Boxes {
    float x0;
    float y0;
    float x1;
    float y1;
};


// Detection results
// For OCR and Object Detection tasks, results are bounding box (box), confidence level (score), 
//                                                 and category (label, label_index)
// For Pixel Segmentation task, results are bounding box (box), confidence level (score), category (label, label_index), 
//                                          and pixel segmentation image (mask, mask_width, mask_height)
struct DetectionResult {
    Boxes box;                   // Bounding box containing the detected object
    float score;                 // Score after predicting, ranging from 0 to 1
                                 // The closer the score is to 1, the better the result is
    std::string label;           // The target's label name after predicting
    int label_index;             // Index of the label
    cv::Mat mask;                // Mask of the the detected object
    int row_index;               // Position of the the detected object, used for OCR sort (not useful right now)
    int col_index;               // Position of the the detected object, used for OCR sort (not useful right now)
    int mask_width;              // Width of the mask
    int mask_height;             // Height of the mask
};

​ The specific status codes are as follows:

4.4 destroy_model()

• Definition: This function is used to clear the model from the GPU.
• Parameter Explanation：

• Return value: None