A list of proposals is generated by the detection algorithm and compared to the ground truth in the list of labels. The following flowchart removes ambiguity based on the order the proposals are submitted.Ĭhart 1: Flow chart for the detection in SpaceNet. If a true positive is found, then the pair - the label and the proposed region - are removed from the sequence and the search continues. This feature is implemented by a sequential search for a true positive sorted by decreasing IoU values. One additional feature that SpaceNet adopts from LSVRC is the notion that each labeled region can have at most one true positive associated with that labeled region. The first instance of SpaceNet works with building footprint labels, for which the chosen threshold gives ample room to differentiate algorithms. SpaceNet defines a threshold of the IoU score of 0.5, above which is considered a detection and below which is not a detection.Įven though IoU is scale-invariant, resolution limits in an image make the 0.5 threshold challenging for small objects. Just using IoU is insufficient to define a detection. With an object detection algorithm, the performance of an algorithm should depend on how many objects the algorithm detects (true positives), how many objects it fails to detects (true negatives), and how many non-objects it detects (false positives). When working in world coordinates, object detection algorithms can be optimized to use the known scale and reduce the search space. One significant advantage that GIS imagery has over other imagery is the known scale. Conversion between the two coordinate systems is not difficult but could be a barrier-of-entry to working with GIS imagery. We often call the GIS (short for Geographic Infomation System) coordinate system “world coordinates” when comparing to image-specific “pixel coordinates”. Using the geospatial coordinate system to label the objects of interest allows for a resolution independent description of the location of an object of interest. Satellite imagery comes in a variety of resolutions depending on the satellite and sensor. Scale invariance is generally desirable but may result in low IoU scores when automating detection of small objects for various reasons including, inaccurate labeled training data, pixelization in imagery, and sensitivity to occlusions. Since IoU is scale invariant, IoU can be computed in world coordinates or in pixel coordinates with the caveat that the conversion may need to accommodate fractional values for pixels. IoU can be converted into a true, mathematical metric but it is often preferable to use IoU directly as opposed to the related metric. IoU(A,B) = area(A intersection B) / area( A union B) The IoU is a measure of how close two regions are two each other on a scale between 0 and 1 - a value of 0 means the regions do not overlap and a scale of 1 means that the regions are exactly the same. As SpaceNet evolves, IoU can scale with expected diversity of object sizes. The SpaceNet competition leverages the familiarity of IoU to attract participation from the machine learning community. The LSVRC competitions associated to ImageNet use IoU as a metric largely because the scale-invariance works well with diverse object sizes. IoU shines at distinguishing regions that overlap but lacks detail on non-overlapping regions. The IoU presents a normalized (scale-invariant) measure that focuses on the areas of the regions. Additionally, the Hausdorff distance is computationally expensive to compute. The Hausdorff distance may overvalue the boundary of a region compared to the interior. For large, isolated regions Euclidean distance may be appropriate, but lacks the capability to value any detail of the regions. There are several candidates for this measure that have viability for certain applications: To train and evaluate computer vision algorithms, one needs a measure that can relate the distance between two regions. The blue regions are non-bounding box labels for building footprints.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |