Best Practices for Camera Selection and Positioning

The purpose of this document is to provide an understanding of how to design an effective optical system that maximizes Cogniac’s platform performance and best achieves reliable and accurate detection of defects and objects of interest.

The placement and configuration of cameras can greatly impact the system's accuracy, reliability, and effectiveness. Proper design ensures that the system can capture the necessary information from the scene and process it effectively. Effective camera placement and configuration can also improve system efficiency and reduce hardware, processing, and maintenance costs.

Camera Selection

Target Size and Distance

The two essential factors that need to be considered when choosing a camera for a computer vision application are the target size and the target’s distance from the camera. These two factors are determined by the lens, the camera, and the camera settings that will be necessary for consistent and reliable detection by the Cogniac platform.

Camera Lens

A camera lens is an element (or elements) that enables light to come into focus in the camera. The focal length of a lens is a measure of its magnification. The higher the focal length, the higher the magnification, which increases the camera's ability to see smaller objects and objects far away. However, the tradeoff is the reduction in how much of the scene is captured. Conversely, smaller focal lengths have a larger field-of-view (FOV) with less detail than lenses with long focal lengths. Some lenses have variable focal lengths, which make the camera more versatile.

Target Resolution

Even though a small focal length sees more of a scene, it does not mean that it is gathering more data than a camera with a large focal length. No matter the lens, the amount of data gathered in a scene is determined by the camera sensor. The sensor is made up of pixels that digitize the light coming onto it. The total number of pixels that a camera sensor is made up of is called resolution.

The resolution of a camera is an approximation of how many pixels the camera sensor has. With higher resolution, targets have more pixels and, in turn, more detail.

The minimum height and width of a target in pixels needs to be at least 20 pixels in order to be reliably detected by the Cogniac system. This can be calculated using the height of the target, the distance to the target, the focal length of the lens, and the size of the pixels on the sensor. See Appendix A for how to calculate the height of a target in pixels.

Field of View

The Field of View is the horizontal and vertical scene that is being captured by the camera. Every camera and lens combination has a viewing angle that limits the amount of field of view that can be seen in a single image.

The camera sensor and lens need to work together to create a field of view that sees the whole scene while also making sure the targets are made up of enough pixels to detect. In cases where the scene is too large to be captured by a single camera’s field of view, multiple cameras will be required. If the images from the two cameras are going to be stitched together, the cameras need to be identical and have a minimum of 25% overlap in the field of view.

Distortion

The lower the focal distance of a camera is, the more severe the image distortion becomes. For lenses with a focal length less than 5 mm, distortion becomes noticeable enough to impact the performance of the Cogniac platform. This warping makes objects in the center of the frame look very different from objects at the edges.

There are some applications where ultra-wide angle cameras can maximize the field of view without impacting performance. One example is tracking the state of something not moving (e.g. how much grain is in a hopper). In this case, the distortion between the edges and the center will still be there, but the distortion of the target will be the same between the training data and inference data.

If the target (or targets) are all contained in the center of the frame, the field of view can be cropped and the ultra-wide lenses can still be utilized. Additional system training can also mitigate the decreased performance caused by distortion.

Camera Configuration

Depth of Field

All cameras have limitations around their minimum and maximum in-focus distance, also known as the Depth Of Field. Objects inside of the depth of field will be in focus and objects outside of it will be blurry.

A key element of a lens is the aperture. The aperture, also known as the f-stop, is a circular opening that opens and closes to allow different amounts of light into the sensor. The aperture has the most control over the depth of field of an image.

The smaller the opening, the larger the depth of field becomes. When the aperture is smaller, there is less light getting to the sensor, so more light in the scene is required. In some use cases, a shallow depth of field that blurs the background can help the Cogniac platform focus on the subject matter in focus.

Target Speed

The shutter of a camera is the device that is responsible for freezing the motion of a target. The slow-mo guys made a great video of a rolling shutter in action here. A higher shutter speed can capture higher speed targets and keep them in good focus, but requires more light on the target.

Cameras that utilize auto-exposure typically use the shutter speed (also called exposure time) to get good exposure. To compensate for a dark scene, the shutter speed will be reduced, possibly to a degree that moving targets become blurred. For reference, to take a good picture from the sidewalk of a car going ~30mph, a shutter speed of 1/60 is needed. Supplemental lighting is recommended in applications where ambient light will not be sufficient for proper exposure and shutter speed.

In a use case where the target will be moving at a constant and repeatable speed, the shutter should be fixed so the images are repeatable as well.

Lighting

Exposure

Sufficient lighting is essential to getting quality images of the target for the Cogniac platform to train and perform well. Illumination plays a critical role in determining the contrast, depth of field, and sharpness of the image. The target needs to be adequately lit so that the features of the target can be distinguished, also known as good exposure. The camera configuration that can be used is highly dependent on if there is adequate lighting available in a scene with low light conditions putting the most constraints around them.

Most cameras can auto-compensate for variation of lighting conditions, but if the use case will need to be active in a wide range of lighting conditions, like at night and during the day, considerations around configuration capabilities, supplementary lighting and additional training need to be addressed.

Auxiliary Lighting

If additional lighting is needed in an application, there are additional hardware considerations and requirements. Fluorescent, quartz halogen, and LED lighting are the most common lighting used in machine vision applications. In settings where the application is lacking in ambient light, simple diffuse lighting in the form of street lamps or overhead lighting is commonly used. In more specific applications like manufacturing inspection, lights are arranged in specific ways to highlight the target in a way that best differentiates it from its surroundings.

When integrating the lighting with a camera (or cameras), additional hardware is required. These can include light controllers (like a PLC), power supplies, cables and connectors, mounting hardware and a system triggering device. Vision system integrators can help in the configuration and install of these devices to ensure the LEDs operate in sync with the camera along with the rest of the system.

Target Contrast

If the subject and the background have similar colors or textures, it can be difficult for the system to separate them and identify the subject accurately. In such cases, the system may make errors in its analysis, leading to incorrect decisions or classifications.

The contrast between the subject and the background is necessary for a machine vision application because it helps the machine to distinguish between the subject and the background. Unique light and camera mounting angles, different light wavelengths and even colored filtering are all techniques that can be used to increase contrast.

Camera Positioning

Objects Covering

Objects like foliage and furniture can impede the camera’s line of sight of a target. In order to consistently detect a target, the target needs to be at least 60% to 70% visible to the camera. The smaller the object is, the greater the need is for complete visibility.

Optimized Target Capture

It is important to place the camera in a position with comprehensive scene coverage. A camera looking at types of cars will more efficiently collect information if it is placed high up and can see many cars at once compared to if it is placed at street level and can only see one car at a time.

Targets should also be viewed at an angle that highlights their unique characteristics. An application looking at people riding a bike or a scooter will not perform well if it is mounted high up and looking straight down. A camera should be set up to see the side profile of the bikes and scooters to be able to detect and differentiate them accurately.

Poor Weather / Looking into the Sun

In outdoor applications, poor weather conditions can cover targets in ways that won’t allow detection. Some examples include fog blurring a target, rain distorting car headlights and taillights, and snow decreasing overall visibility.

It is also important to not position a camera looking into the sun at any time of the day or year. The sun’s direct brightness will overtake a camera’s auto-exposure settings and make the foreground objects extremely dark and background objects extremely bright.

Cleanliness

The cleanliness of a camera lens is essential to its consistent and effective performance. System performance will decline over time if the camera is placed in an environment where particulates can build up on the lens. Dust and particulates can also get into the camera components and reduce their ability to cool, reducing the hardware's performance and lifespan.

To prevent this, regular cleaning of the cameras should occur. Additional hardware like camera enclosures, camera placement in a covered area, or air blowoff systems all work to reduce the maintenance required to keep the camera clean.

Scenarios

The following scenarios bring vision system design considerations together to set up the hardware and mounting requirements for different use cases.

Supermarket

Use Case:

The number of people leaving a supermarket with a bag or cart (meaning they purchased something) and the number of people leaving without purchasing anything.

Camera Location:

They are mounted above the exit door(s) looking into the store approximately 3 meters from people leaving the store at an angle of 45 degrees.

Camera Lens:

A 10mm lens will result in a shopping bag 3 meters away having a pixel height of 595 pixels on an average image sensor. The field of view is wide enough to be able to see large shopping carts in the frame fully, but it will also have enough resolution to see shopping bags well.

Shutter Speed:

Greater than 1/60 to freeze the motion of the people walking out the door.

Aperture:

f/4 for a large depth of field, but it can still work off of the artificial illumination of the store.

Sidewalk Usage

Use Case:

Determine the usage of sidewalks. Identify if people are using the sidewalk for walking, biking, or scootering at night and during the day.

Camera Location:

It was mounted on a light pole ~15 feet high and pointed towards the opposite side of an intersection. The camera should be mounted above or on the opposite side of the pole as the light to prevent light interference when it turns on.

Camera Lens:

A 6mm Lens will provide a large field of view allowing you to see a lot of the sidewalk in the scene. People are relatively large targets to detect and even more significant when combined with a bike or scooter, so moderate magnification is sufficient.

Shutter Speed:

The lowest shutter speed the camera should go to is 1/60th second. The slower 1/60th shutter speed will be used at night when there is only artificial illumination from the street light available and a faster shutter speed will be used during the day when there is ample sunlight.

Aperture:

f/2.8 during the day to get more depth of field and f/1.4 at night to allow more light in.

Cleaning Solution Bottle

Use Case:

Looking at the fullness level of a clear bottle of cleaning solution coming off from the manufacturing line.

Camera Location:

Placed beside the manufacturing line ~1 meter away, looking at the necks of the bottles where the fullness of the liquid contents is expected to be.

Camera Lens:

A 65mm lens will frame the top of the bottle to see where the fill line should be while leaving a safe distance from the machinery for the camera system.

Shutter Speed:

The camera should be triggered based on the line speed of the bottles using a device like an encoder. The shutter speed (or exposure time) will need to be fast to keep up with the speed of the manufacturing line and should be fixed.

Aperture:

f/1.4. The front and the back of the bottle should be in full focus, but blurring contents in the background will simplify the scene.

Lighting:

A diffuse light should be placed behind the bottles so that the light is shining through the bottles and to the camera.

Appendix A

The height of a target in pixels can be calculated using this formula:

h: Image Height on Sensor (pixels)

H: Height of Target (mm)

F: Focal Length of the Lens (mm)

D: Distance to the Target from the End of the Lens (mm)

Px: Sensor Pixel Size (μm)