Annotation with rectangles

Annotation with rectangles is a type of image annotation in which a rectangular shape is drawn around an object or region of interest in an image.

In computer vision tasks like object detection or image segmentation, where the objective is to recognize and locate particular objects or regions within an image, this is frequently used.

 Using an annotation tool, a rectangle can be drawn around an object to create a rectangular annotation. This can be done manually or automatically using algorithms that identify and categorize objects in images.

 The object class or label, which can be used to train machine learning models for object recognition or segmentation, can be included with the rectangular annotation as additional information. You can crop an image to focus on the area of interest or annotate it with a rectangle to measure the size or shape of an object.

.

Rotation rectangle

To rotate the rectangle, pull on the rotation point. Rotation is done around the center of the rectangle. To rotate at a fixed angle (multiple of 15 degrees), hold shift. In the process of rotation, you can see the angle of rotation.

 

Annotation with rectangle by 4 points

It is an efficient method of bounding box annotation, proposed here. Before starting, you need to make sure that the drawing method by 4 points is selected.

Press Shape or Track for entering drawing mode. Click on four extreme points: the top, bottom, left- and right-most physical points on the object. Drawing will be automatically completed right after clicking the fourth point. Press Esc to cancel editing.

 

Annotation with polygons

Drawing a closed shape with multiple sides, such as a triangle, rectangle, or irregular polygon, around an object or region of interest in an image is known as annotation with polygons. This is frequently used in computer vision tasks where the objective is to recognize and locate particular objects or regions within an image, such as object detection or image segmentation.

Because they can closely mimic an object’s or region’s contours, polygons can be more precise than other annotation shapes. They can also be used to annotate non-rectangular objects, like the outline of a person in a picture.

Polygons can be annotated manually by drawing the shape around the object using an annotation tool, or automatically by using computer algorithms that recognize and categorize objects in an image.

 In order to train machine learning models for object recognition or segmentation, the polygon annotation may be accompanied by additional data, such as the object class or label. A polygonal annotation can be used to crop an image to focus on a specific area of interest, measure the size or shape of an object, or both.

Guide to creating and editing polygons

Manual drawing

It is used for semantic / instance segmentation.

Before starting, you need to select Polygon on the controls sidebar and choose the correct Label.

• Click Shape to enter drawing mode. There are two ways to draw a polygon: either create points by clicking or by dragging the mouse on the screen while holding Shift.

• When Shift isn’t pressed, you can zoom in/out (when scrolling the mouse wheel) and move (when clicking the mouse wheel and moving the mouse), you can also delete the previous point by right-clicking on it.
• You can use the Selected opacity slider in the Objects sidebar to change the opacity of the polygon. You can read more in the Objects sidebar section.
• Press N again or click the Done button on the top panel for completing the shape.
• After creating the polygon, you can move the points or delete them by right-clicking and selecting Delete point or clicking with pressed Alt key in the context menu.

Drawing using automatic borders

 

You can use auto borders when drawing a polygon. Using automatic borders allows you to automatically trace the outline of polygons existing in the annotation.

• To do this, go to settings -> workspace tab and enable Automatic Bordering or press Ctrl while drawing a polygon.

• Start drawing / editing a polygon.
• Points of other shapes will be highlighted, which means that the polygon can be attached to them.
• Define the part of the polygon path that you want to repeat.

• Click on the first point of the contour part.

• Then click on any point located on part of the path. The selected point will be highlighted in purple.

• Click on the last point and the outline to this point will be built automatically.

Besides, you can set a fixed number of points in the Number of points field, then drawing will be stopped automatically. To enable dragging you should right-click inside the polygon and choose Switch pinned property.

Below you can see results with opacity and black stroke:

If you need to annotate small objects, increase Image Quality to 95 in Create task dialog for your convenience.

Edit polygon

To edit a polygon you have to click on it while holding Shift, it will open the polygon editor.

• In the editor you can create new points or delete part of a polygon by closing the line on another point.
• When Intelligent polygon cropping option is activated in the settings, CVAT considers two criteria to decide which part of a polygon should be cut off during automatic editing.

  • The first criteria is a number of cut points.
  • The second criteria is a length of a cut curve.

If both criteria recommend to cut the same part, algorithm works automatically, and if not, a user has to make the decision. If you want to choose manually which part of a polygon should be cut off, disable Intelligent polygon cropping in the settings. In this case after closing the polygon, you can select the part of the polygon you want to leave.

• You can press Esc to cancel editing.

Track mode with polygons

Polygons in the track mode allow you to mark moving objects more accurately other than using a rectangle.

1. To create a polygon in the track mode, click the Track button.

• Create a polygon the same way as in the case of Annotation with polygons. Press N or click the Done button on the top panel to complete the polygon.
• Pay attention to the fact that the created polygon has a starting point and a direction, these elements are important for annotation of the following frames.
• After going a few frames forward press Shift+N, the old polygon will disappear and you can create a new polygon. The new starting point should match the starting point of the previously created polygon (in this example, the top of the left mirror). The direction must also match (in this example, clockwise). After creating the polygon, press N and the intermediate frames will be interpolated automatically.

• If you need to change the starting point, right-click on the desired point and select Set starting point. To change the direction, right-click on the desired point and select switch orientation.

There is no need to redraw the polygon every time using Shift+N, instead you can simply move the points or edit a part of the polygon by pressing Shift+Click.

Creating masks

Cutting holes in polygons.

Currently, CVAT does not support cutting transparent holes in polygons. However, it is poissble to generate holes in exported instance and class masks. To do this, one needs to define a background class in the task and draw holes with it as additional shapes above the shapes needed to have holes:

The editor window:

Remember to use z-axis ordering for shapes by [-] and [+, =] keys.

Exported masks:

Notice that it is currently impossible to have a single instance number for internal shapes (they will be merged into the largest one and then covered by “holes”).

Creating masks

There are several formats in CVAT that can be used to export masks:

• Segmentation Mask (PASCAL VOC masks)
• CamVid
• MOTS
• ICDAR
• COCO (RLE-encoded instance masks, guide)
• TFRecord (over Datumaro, guide):
• Datumaro

An example of exported masks (in the Segmentation Mask format):

Important notices:

• Both boxes and polygons are converted into masks
• Grouped objects are considered as a single instance and exported as a single mask (label and attributes are taken from the largest object in the group)

Class colors

All the labels have associated colors, which are used in the generated masks. These colors can be changed in the task label properties:

Label colors are also displayed in the annotation window on the right panel, where you can show or hide specific labels (only the presented labels are displayed):

A background class can be:

• A default class, which is implicitly-added, of black color (RGB 0, 0, 0)
• background class with any color (has a priority, name is case-insensitive)
• Any class of black color (RGB 0, 0, 0)

To change background color in generated masks (default is black), change background class color to the desired one.