Segmentation Application

Segmentation types

1. Instance segmentation involves creating a segmentation mask for each distinct object, distinguishing between individual instances of objects within the same class. This level of granularity is valuable for precise object counting, tracking, and interaction analysis. For example, in Robotics - it can help identify and differentiate multiple objects of the same type in a cluttered environment. In Retail - it can improve store layout and customer experience by providing personalized recommendations and targeted discounts.  

2. Panoptic segmentation aims to label all pixels in an image, including "stuff" and "things," creating a segmentation map of the entire image and segmentation masks for each distinct object. It unifies semantic and instance segmentation into a single framework, benefiting applications like scene understanding and urban planning. Deep learning models, such as convolutional neural networks (CNNs) and Panoptic FPN, have demonstrated remarkable performance in achieving these segmentation tasks.
   
   

Segmentation application

Segmentation applications divide images into multiple regions or segments, making it easier to analyze specific objects, boundaries, or areas. The segmentation application is created to facilitate instance and panoptic segmentation by generating masks for each instance within every class.

*Currently, the SAM Feature is beta-ready and available upon request.

Cogniac uses RLE to store the masks.

RLE, or Run-Length Encoding, is a simple form of data compression that replaces sequences of the same data value (runs) with a single value and a count. For example, instead of storing "AAAABBBCCDAA" as-is, RLE would encode it as "4A3B2C1D2A." We compress RLE masks into LEB128 format.

Effectiveness of RLE:

1. Simplicity: RLE is easy to implement and understand, making it a good choice for basic compression tasks.

2. Efficiency with Repetitive Data: RLE works particularly well with data that has long runs of repeated values, such as simple graphics or certain types of text files. It can significantly reduce file size in these cases.

3. Low Computational Overhead: Since RLE only requires counting and storing runs, it has low computational demands, making it efficient for both encoding and decoding.

4. Lossless Compression: RLE is a lossless compression technique, meaning that the original data can be perfectly reconstructed from the compressed data.

How to provide feedback

Once users open the Segmentation application to provide feedback, they should click the 'Add Instance' button needs to be selected to add an instance. They will then be able to mark the area of interest. By leveraging an advanced Segmentation application, we can accurately label the dice area for this case. The segmentation application currently supports only 'Best F1' Release Metrics. Users must select the 'Submit Feedback' when instances are added and ready to submit.

Currently, there are two available methods for providing feedback. Instances can be created using the SAM annotate tool which allows for easily marking the instance masks, thereby saving time, or using the Brush tool (manually).

1. Using the SAM annotate tool

   The SAM annotate tool uses the Segment Anything Model in the backend to assist users in providing feedback. The user can click (Point-Prompt) anywhere on the image, and SAM provides candidate masks. SAM tries to capture the object under the point using a mask.
   

2. Using the Brush tool

Users can create a mask entirely using a paintbrush or edit the masks suggested by SAM.

Denoising and how it works

The denoising is a frontend feature that allows users to easily remove all pixel groups that are 7 pixels or smaller from an instance mask, eliminating the need for manual cleanup with the eraser tool. By automating the removal of these small pixel clusters, the feature also speeds up the process of fine-tuning and polishing the mask.

Once the user has finished marking the Instance Mask, they should select the Submit Feedback button at the bottom of the page. Before that, the Instance Mask won’t be saved.

Once the user submits the feedback, we fine-tune SAM to the user inputs. We have 2 activities associated with it under the Summary tab of the application:

1. Label Model Trained: This activity updates the new label model (SAM) that was trained to improve the labeling experience.

2. Label Model Released: This activity provides an update on whether we have some new best-winning label models. From here on, we use this model to assist the user while labeling.

Currently supported pipelines

1. Segmentation application downstream to Classification application

2. Segmentation application downstream to Detection Point application

3. Segmentation application downstream to Static Count application

4. Box Detection application downstream to Segmentation application