Why OpenCV and AI tools Used in CVAT?
OpenCV (Open Source Computer Vision Library) is a popular open-source computer vision library that provides a wide range of functionalities for image and video processing.
It is often used in conjunction with other AI tools in CVAT (Computer Vision Annotation Tool) to perform tasks such as object detection, image segmentation, and classification. Here are some ways in which OpenCV and AI tools can be used in CVAT:
Object detection: OpenCV provides several algorithms and techniques for object detection, such as Haar Cascades, HOG (Histogram of Oriented Gradients), and deep learning-based methods like YOLO (You Only Look Once). These algorithms can be integrated into CVAT to help annotate images and videos more efficiently.
Image segmentation: OpenCV also provides a range of algorithms for image segmentation, such as watershed segmentation, graph-based segmentation, and thresholding. These algorithms can be used in CVAT to segment images into regions of interest and help in annotation.
Classification: OpenCV can be used in conjunction with other AI tools to classify images into different categories. For example, a model trained on a deep learning framework such as TensorFlow or PyTorch can be integrated into CVAT to perform image classification tasks.
Data augmentation: OpenCV can also be used in CVAT for data augmentation, which involves applying various transformations to images to increase the size of the dataset. This can help in training AI models by providing more diverse and representative data.
OpenCV and AI tools can be used in CVAT to perform tasks such as object detection, image segmentation, classification, and data augmentation. By integrating these tools, CVAT can become a powerful tool for computer vision annotation and AI model training.
Overview of semi-automatic and automatic annotation tools available in CVAT
Label and annotate your data in semi-automatic and automatic mode with the help of AI and OpenCV tools.
While interpolation is good for annotation of the videos made by the security cameras, AI and OpenCV tools are good for both: videos where the camera is stable and videos, where it moves together with the object, or movements of the object are chaotic.
Interactors
Interactors are a part of AI and OpenCV tools.
Use interactors to label objects in images by creating a polygon semi-automatically.
When creating a polygon, you can use positive points or negative points (for some models):
- Positive points define the area in which the object is located.
- Negative points define the area in which the object is not located.
AI tools: annotate with interactors
To annotate with interactors, do the following:
- Click Magic wand
, and go to the Interactors tab.
- From the Label drop-down, select a label for the polygon.
- From the Interactor drop-down, select a model.
Click the Question mark to see information about each model:
4. (Optional) If the model returns masks, and you need to convert masks to polygons, use the Convert masks to polygons toggle.
5. Click Interact.
6. Use the left click to add positive points and the right click to add negative points.
Number of points you can add depends on the model.
7. On the top menu, click Done (or Shift+N, N)
AI tools: add extra points
Each model has a minimum required number of points for annotation. Once the required number of points is reached, the request is automatically sent to the server. The server processes the request and adds a polygon to the frame.
For a more accurate outline, postpone request to finish adding extra points first:
- Hold down the Ctrl key.
On the top panel, the Block button will turn blue. - Add points to the image.
- Release the Ctrl key, when ready.
In case you used Mask to polygon when the object is finished, you can edit it like a polygon.
You can change the number of points in the polygon with the slider:
AI tools: delete points
To delete a point, do the following:
- With the cursor, hover over the point you want to delete.
- If the point can be deleted, it will enlarge and the cursor will turn into a cross.
- Left-click on the point.
OpenCV: intelligent scissors
1. On the menu toolbar, click OpenCV and wait for the library to load.
2. Go to the Drawing tab, select the label, and click on the Intelligent scissors button.
3. Add the first point on the boundary of the allocated object.
You will see a line repeating the outline of the object.
4. Add the second point, so that the previous point is within the restrictive threshold.
After that a line repeating the object boundary will be automatically created between the points.
5. To finish placing points, on the top menu click Done (or N on the keyboard).
As a result, a polygon will be created.
You can change the number of points in the polygon with the slider:
To increase or lower the action threshold, hold Ctrl and scroll the mouse wheel.
During the drawing process, you can remove the last point by clicking on it with the left mouse button.
Detectors
Detectors are a part of AI tools.
Use detectors to automatically identify and locate objects in images or videos.
Labels matching
Each model is trained on a dataset and supports only the dataset’s labels.
For example:
- DL model has the label
car. - Your task (or project) has the label
vehicle.
To annotate, you need to match these two labels to give DL model a hint, that in this case car = vehicle. If you have a label that is not on the list of DL labels, you will not be able to match them. For this reason, supported DL models are suitable only for certain labels.
To check the list of labels for each model
Annotate with detectors
To annotate with detectors, do the following:
- Click Magic wand
, and go to the Detectors tab.
- From the Model drop-down, select model
- From the left drop-down select the DL model label, from the right drop-down select the matching label of your task.
4. Optional) If the model returns masks, and you need to convert masks to polygons, use the Convert masks to polygons toggle.
5. Click Annotate.
This action will automatically annotate one frame. For automatic annotation of multiple frames.
Trackers
Trackers are part of AI and OpenCV tools.
Use trackers to identify and label objects in a video or image sequence that are moving or changing over time.
AI tools: annotate with trackers
To annotate with trackers, do the following:
- Click Magic wand , and go to the Trackers tab.
2. From the Label drop-down, select the label for the object.
3.From Tracker drop-down, select tracker.
4. Click Track, and annotate the objects with the bounding box in the first frame.
5. Go to the top menu and click Next (or the F on the keyboard) to move to the next frame.
All annotated objects will be automatically tracked.
OpenCV: annotate with trackers
To annotate with trackers, do the following:
- On the menu toolbar, click OpenCV
and wait for the library to load.
2.Go to the Tracker tab, select the label, and click Tracking.
3. From the Label drop-down, select the label for the object.
4. From Tracker drop-down, select tracker.
5. Click Track.
6. To move to the next frame, on the top menu click the Next button (or F on the keyboard).
All annotated objects will be automatically tracked when you move to the next frame.
When tracking
- To enable/disable tracking, use Tracker switcher on the sidebar.
- Trackable objects have an indication on canvas with a model name.
- You can follow the tracking by the messages appearing at the top.
OpenCV: histogram equalization
Histogram equalization improves the contrast by stretching the intensity range.
It increases the global contrast of images when its usable data is represented by close contrast values.
It is useful in images with backgrounds and foregrounds that are bright or dark.
To improve the contrast of the image, do the following:
- In the OpenCV menu, go to the Image tab.
2. Click on Histogram equalization button.
Histogram equalization will improve contrast on current and following frames.
Example of the result:
To disable Histogram equalization, click on the button again.
