Interface QualitySettings

Check for partially-covered annotations, useful in segmentation tasks

Enables or disables annotation attribute comparison

Enables or disables annotation group checks

Enables or disables polyline orientation comparison

Consider empty frames annotated as "empty". This affects target metrics like accuracy in cases there are no annotations. If disabled, frames without annotations are counted as not matching (accuracy is 0). If enabled, accuracy will be 1 instead. This will also add virtual annotations to empty frames in the comparison results.

Minimal IoU for groups to be considered matching. Only used when the 'compare_groups' parameter is true

Used for distinction between matched / unmatched shapes

The minimal gain in the GT IoU between the given and reversed line directions to consider the line inverted. Only used when the 'compare_line_orientation' parameter is true

Thickness of polylines, relatively to the (image area) ^ 0.5. The distance to the boundary around the GT line, inside of which the checked line points should be

Used for distinction between strong / weak (low_overlap) matches

The maximum number of job validation attempts for the job assignee. The job can be automatically accepted if the job quality is above the required threshold, defined by the target threshold parameter.

Minimal visible area percent of the spatial annotations (polygons, masks) for reporting covered annotations. Only used when the 'object_visibility_threshold' parameter is true

Like IoU threshold, but for points. The percent of the bbox side, used as the radius of the circle around the GT point, where the checked point is expected to be. For boxes with different width and height, the "side" is computed as a geometric mean of the width and height. Read more: https://cocodataset.org/#keypoints-eval

Use only the visible part of the masks and polygons in comparisons

When comparing point annotations (including both separate points and point groups), the OKS sigma parameter defines matching area for each GT point based to the object size. The point size base parameter allows to configure how to determine the object size. If image_size, the image size is used. Useful if each point annotation represents a separate object or boxes grouped with points do not represent object boundaries. If group_bbox_size, the object size is based on the point group bbox size. Useful if each point group represents an object or there is a bbox grouped with points, representing the object size. * image_size - IMAGE_SIZE * group_bbox_size - GROUP_BBOX_SIZE

The primary metric used for quality estimation * accuracy - ACCURACY * precision - PRECISION * recall - RECALL

Defines the minimal quality requirements in terms of the selected target metric.