|
V+ 12.4 Release Notes |
 |
 |
 |
 |
ObjectFinder Enhancements
This section describes the enhancements to the ObjectFinder for AdeptVision VXL that were made after version 12.1 was released.
Overview
This section provides a summary of the ObjectFinder enhancements. Several of these topics are detailed in later sections.
Feature-based Refinement
Feature-based refinement was added in version 12.2 to increase significantly the accuracy of alignment for instances found with the ObjectFinder. In version 12.3, the calculation for pose refinement has been improved to provide a better estimate of the orientation of elongated parts. See Details of Pose Refinement for details.
Automatic Learning
Automatic learning is now implemented in two stages. The first stage uses the found instances to generate intelligent weights for the features. This is called auto-multi-instance mode, since it is essentially an automated way of performing the manual multi-instance training procedure, which was available in AdeptVision VXL version 12.1 and VisionWare version 3.2F.
The second stage of automatic learning is designed to work with the final model and feature weights. It dynamically, but progressively, adjusts the ordering on the feature classes, which determines the order of proposals, to optimize performance (objects/sec.). Although it is designed to be a runtime replacement for the finder, it can be a bit slower (up to 10%). Therefore, you should adjust the speed of the part-feeding mechanism accordingly.
In VisionWare, the two stages are controlled by the Auto-learn and Auto-plan checkboxes. (See the AIM 3.x Release Notes for details.) Learning requires that you have Auto-plan enabled (checked) so that you do not have to select Plan, which would cause the learning results to be replaced (see the CAUTION above). When Auto-learn is enabled, the finder begins automatic multi-instance training. When the system detects that the model has converged, it performs a quick plan and then switches to learning mode-gradually reordering feature classes.
The recognition speed has also been improved by enhancing the recognition strategy so that it is now more effective for general viewing conditions.
Object Disambiguation
Models may be rotationally ambiguous, meaning that matches may be found in more than one orientation. There can also be ambiguities between models: A model may match another model with a verify percent above the threshold.
The ObjectFinder has been enhanced to detect and compensate for rotationally ambiguous models and similarities between models. Intramodel disambiguation is enabled by default and can be disabled using the symmetry parameter in the VTRAIN.FINDER instruction. Intermodel disambiguation is enabled by default and can be disabled using a new parameter in the VPLAN.FINDER instruction.
ObjectFinder Model File Format
Starting with version 12.3, the ObjectFinder file format changed to be compatible with new releases of the ObjectFinder. AdeptVision versions 12.0 through 12.2 can read only the model files created by each respective version. (AdeptVision versions 12.3 and 12.4 can read model files created by version 12.2.)
For version 12.3 and later, the structure of model files is a superset of the structure used in previous versions. When older model files are read, the fields from later versions are filled with appropriate default values to maintain compatibility. When models are saved, the latest file format is always used.
Concurrency With Other Vision Tools
Starting with version 12.4, the ObjectFinder can run concurrently with other vision tools. This allows other tools to be applied to an ObjectFinder image based on information obtained about that part model instance, such as the position and orientation of the model instance, while the ObjectFinder continues to search for other model instances. For example, an object can now be recognized by the finder, checked with other vision tools such as rulers, then picked by the robot while other object instances are being recognized.
Position and Orientation Offsets
Starting with version 12.4, the ObjectFinder allows position and orientation offsets to be applied to recognized objects. The offsets do not affect the functionality of the finder. They simply provide a way for application programmers to specify the reference point and orientation for object instances.
Position and Orientation Limits
Starting with version 12.4, the user can specify limits on the range of positions and orientations of model instances. For example, the user can specify that a model instance can rotate only plus or minus 10 degrees about the nominal orientation at which the model was trained. This improves ObjectFinder performance and reliability in applications where the objects do not occur in the scene with the full range of positions and orientations.
Binary and Edge Images
Starting with version 12.4, the ObjectFinder can work with a binary image like the prototype finder. Since the ObjectFinder is much easier to train, this should eliminate any need to use the prototype finder.
Additionally, the ObjectFinder can work with edge images that are computed using vision instructions such as VEDGE, VCONVOLVE, or VMORPH, instead of using the edge detection algorithms that are internal to the ObjectFinder. This allows application programmers to specify the image processing algorithms used by the ObjectFinder, and allows the ObjectFinder to work on images that resemble edge images.
Memory Allocation for Vision
For systems using V+ version 12.4 or earlier and a vision processor board with 8MB of memory, Adept recommends that you increase the vision memory allocation from 1.5MB to 2.0MB (the default for V+ 13.0 and later). This increase provides sufficient memory when using the ObjectFinder with complex scenes. The vision memory allocation can be changed using the CONFIG_C utility. See the manual Instructions for Adept Utility Programs for details.
Details of Automatic Learning
Setting the "type" argument in the VFINDER instruction to 3 or 4 invokes the automatic learning feature. Whenever VFINDER is executed in learning mode (type value 4), if the add instances ok flag in the model is FALSE, or if the maximum number of instances to accumulate has been reached, then any new instances are ignored. The maximum number of instances is 17.
To set the add instances ok flag in the model, use the VTRAIN.FINDER instruction. See VTRAIN.FINDER (cam, mode, dmode, arg, arg2, arg3) $model_name, ibr = value, value2, value3 for details. To read the value from the model, use VSHOW/VFEATURE. See VFEATURE (index) for details.
Stage One (VFINDER mode 4)
During "stage one" learning, the feature weights are adjusted. Once the feature weights have converged, learning proceeds to "stage two" and the feature classes are reordered. In V+, stage-one learning does not automatically change to stage-two learning. The programmer must write the program so that it detects when stage-one learning is completed and then activates stage two by changing the VFINDER mode to 3.
Stage Two (VFINDER mode 3)
During "stage two" learning, statistics are gathered about the successes and failures of all the proposals made during the recognition process. After each call to VFINDER (cam, mode, dmode, how_many_total {, times[]}) ibr in this mode, you have the option of using that data to reorder the feature classes. Theoretically, certain classes will have a better likelihood of generating a successful proposal. These will get bubbled to the front of the list if the reordering is done.
You can re-sort after each recognition call or wait until more statistics are accumulated and then re-sort in one pass. A bubble sort is performed because the list should be close to the correct order prior to each re-sort. Therefore, incremental reordering will not require much processing time.
When applying the new statistics, each is applied in a fractional manner to the existing weight. Therefore, the effect is gradual.
If the new weights after a learning-mode VFINDER do not result in a new ordering of the feature classes, the "learning convergence count" is incremented. If there is a new order, this count is set to 0. Therefore, if there is no change in the order of the feature classes for several scenes in a row, this count will rise and can be used to indicate that you can exit learning mode.
Execute a VGETCAL for the virtual camera that has planned models to see the learning convergence count. See VGETCAL (cam) scalers[i], pmm.to.pix[j,k], pix.to.pmm[l,m], to.cam for details.
Details of Pose Refinement
The ObjectFinder part alignment accuracy in version 12.2 and later has been improved over the previous version (12.1). The new pose refinement is based on correcting a confirmed pose hypothesis using selected model features (the refine features). During planning, the refine features are automatically determined based on their relative importance in the model. Selection criteria are based on the lengths and weights of the features as produced by multi-instance training.
During recognition, before carrying out the final verification of a proposed instance pose, the new pose refinement attempts to match the refine features against corresponding image features. If a sufficient match has been determined, then the pose is refined based on these matches and the whole model (i.e., the whole feature set) is verified to determine the verification percent and decide whether it is above the user-specified threshold. In comparison to version 12.1, where refinement relied solely on aligning points (pairs), the improved refinement is extended to align portions of the model (the set of refine features) with their image counterparts before performing the final verification. This new pose refinement significantly improves the alignment of the recognized instances, especially when the image is at one-half resolution (hierarchical level 1).
Additionally, the pose refinement enhancements in version 12.3 result in more accurate poses for parts that are somewhat elongated (i.e., whose length is relatively large compared to the width). The feature-based pose refinement techniques have been improved to better handle these parts by minimizing the orientation error along a part's elongation direction.
Details of Object Disambiguation
The disambiguation functionality allows the ObjectFinder to resolve ambiguities due to self-similarities within models and due to similarities between different models planned for recognition. Self-similar models are "almost" symmetric except for a few features, and similar models look "almost" alike except for a few features, or look like a subset of another model.
When disambiguation is enabled, the ObjectFinder automatically determines potential ambiguities within a model, and between models, and remembers them. At runtime, whenever a model is recognized, before reporting a successful match, the ObjectFinder checks whether the model was found to have ambiguities when it was planned. If this is the case, it attempts to detect and resolve potential ambiguities by performing verification with the alternative model correspondences. This operation is relatively fast because the transformations are precomputed during planning. After alternative model correspondences have been verified, the match with the highest verify percent is reported.
NOTE: Disambiguation is not performed for objects that are circularly symmetric.
Details of Concurrent Use
ObjectFinder concurrency works automatically, without any new instructions or changes to existing instructions, so existing V+ programs do not have to be changed to take advantage of it. The vision system automatically detects situations where concurrency should be used and executes the sequence of vision instructions in a separate system task while the ObjectFinder VFINDER instruction is executed in its own task. After the ObjectFinder places an object instance in the queue, it continues to process the image and attempts to recognize additional model instances. A V+ program could then, for example, execute a VLOCATE instruction followed by VFEATURE instructions to obtain information about the recognized instance. This information could be used to move the robot to pick up an object while the ObjectFinder is attempting to recognize additional objects.
The following vision instructions can be executed concurrently with the ObjectFinder VFINDER instruction: VRULERI, VFIND.LINE, VFIND.ARC, VFIND.POINT, VWINDOWI, VDEF.AOI, VGET.AOI, VDEF.TRANS, and VGET.TRANS.
Before ObjectFinder concurrency was implemented (up to and including AdeptVision version 12.3), none of the vision tools listed above could be executed until the VFINDER instruction was completely finished and all objects had been found in the image. For complex, cluttered images and/or complex parts, this could cause a significant slowdown in the part-consumption cycle time. For example, in an image with several objects, the first model instance could not be inspected until the entire image had been processed and all the objects were found. With automatic concurrency, a part can be immediately inspected with any combination of the vision tools listed above as soon as the object has been recognized and placed in the queue. Inspection of the object can proceed concurrently while the ObjectFinder recognizes other parts.
The following V+ code provides an example of concurrent execution.
VPICTURE(cam) 2 VFINDER (cam, 1, -1, howmany) aoi_num VLOCATE (cam, 0, 0) $name, loc DO IF VFEATURE(1) THEN DECOMPOSE xform[] = loc x = xform[0] y = xform[1] ang = xform[5] VRULERI (cam, 2) ruler[] = 1, x, y, r, ang ; Code to test the edge points VFIND.LINE (cam) line[] = 1, x, y, r, r/4, ang+90 ; Code to test the line VFIND.ARC (cam, 4,,, 2) arc[] = 1, x, y, r, r/2, ang, ang+270 ; Code to test the arc VWINDOWI (cam, 0) data[] = 1, x, y, r, r, ang ; Code to test the window statistics END VLOCATE (cam, 0, 0) $name, loc UNTIL NOT VFEATURE(1)The vision tools VRULERI, FIND.LINE, VFIND.LINE, VFIND.ARC, and VWINDOWI are executed for each part found by the VLOCATE while the ObjectFinder VFINDER instruction is executing to recognize and queue other parts. The first VLOCATE instruction (before the DO loop) causes the instruction sequence to wait until the first object is recognized and placed in the queue, the second VLOCATE instruction (at the bottom of the loop) causes the instruction sequence to wait until at least one additional object is recognized and placed in the queue. The ObjectFinder is always allowed to run while the instruction sequence waits for an object.
Details of Position and Orientation Offsets
With AdeptVision version 12.4, the application programmer can specify position and orientation offsets for an ObjectFinder model. This is done using mode 5 of the VTRAIN.FINDER instruction. The offsets are added to the position and orientation of recognized instances before they are placed in the VLOCATE queue. This enhancement allows V+ programmers to use other vision tools, such as rulers, line finders, or blob finders, to determine a desired reference point and orientation for the model (overriding the values chosen by the ObjectFinder when the model is trained).
For example, let's consider how this would apply to a rectangular part. The ObjectFinder may set the reference point at the center of the rectangle. However, the programmer may need to set the reference point to be at a particular corner of the rectangle. The programmer can use line finders or rulers to calculate the location of the corner of the rectangle, use VSHOW to determine the default reference point chosen by the ObjectFinder for the model, calculate the offset required to place the reference point at the desired location, and assign this offset to the model using mode 5 of the VTRAIN.FINDER instruction. The offset will be added to each recognized instance of the rectangle so that the reported position and orientation will be relative to the reference point and orientation determined by the programmer.
The reason that the reference frame must be specified indirectly as offsets is that there is no way to specify the reference frame for a model before the model has been trained. The offsets allow the programmer to specify the desired reference frame as offsets in position and orientation from the reference frame computed automatically by the ObjectFinder.
See the description of VTRAIN.FINDER (cam, mode, dmode, arg, arg2, arg3) $model_name, ibr = value, value2, value3 on This Topic for more details.
Details of Position and Orientation Limits
In some applications, the object will not translate or rotate significantly from the position or orientation in which it was trained. Effective with version 12.4, the ObjectFinder allows the application programmer to specify the range of positions and orientations in which the object will occur in an image. This constraint can be specified using mode 14 of the VTRAIN.FINDER instruction. For example, an object may occur with an orientation that is plus or minus 30 degrees from the orientation in which it was trained. If the programmer specifies these limits with the VTRAIN.FINDER instruction, the ObjectFinder will not attempt to recognize objects with orientations outside this range. This can improve recognition time and reduce false matches. See the description of VTRAIN.FINDER (cam, mode, dmode, arg, arg2, arg3) $model_name, ibr = value, value2, value3 on This Topic for more details.
Details of Binary Images
The ObjectFinder was originally designed to work with gray value images, applying its own image processing algorithms to the image to compute features for training and recognition. In contrast, the prototype finder was designed to work with binary images and compute features from the boundary points of the binary regions and holes. There are situations where binary images are better for recognition than edges computed from gray-value images (e.g., binary images may be better when parts are very small). For that reason, the ObjectFinder in version 12.4 is enhanced to work with binary images using the same image processing algorithms that were used in the prototype finder. Since the ObjectFinder is easier to train, this enhancement should allow the ObjectFinder to replace the prototype finder in binary applications.
In binary-image processing mode, the ObjectFinder uses the edges of the boundary of the binary blobs extracted from the binary image specified in the AOI. The switches and parameters that define binary processing in the general vision system are also used by the ObjectFinder in binary mode, including V.BINARY, V.BACKLIGHT, and V.THRESHOLD. The following examples demonstrate the use of the switches and parameters for the binary-mode ObjectFinder. The program bintrain( ), shown below, can be used to train a model using binary edge processing.
PROGRAM bintrain(cam, threshold, back, $modn) aoi = 2000 cx = 307 cy = 256 width = 400 height = 230 VDEF.AOI aoi = 1, cx, cy, width, height, 0 acq_ibr = aoi+11 ;Set switches and parameters for binary image processing ENABLE V.BINARY[cam] PARAMETER V.THRESHOLD = threshold PARAMETER V.MAX.AREA = width * height ;Specify light (dark) object on dark (light) background IF (back == 1) THEN ENABLE V.BACKLIGHT[cam] ELSE DISABLE V.BACKLIGHT[cam] END VPICTURE (cam,, acq_ibr) 2 VWAIT ;Tell finder to use binary data as the input VTRAIN.FINDER (cam, 15,, 0) ;Invoke finder with the usual instruction syntax VTRAIN.FINDER (cam, 1, 2, 0, 75, 0) $modn, acq_ibr ;Reset the finder to default input mode (if desired) VTRAIN.FINDER (cam, 15,, 1)The V.BINARY switch must be enabled so that the vision system performs binary image processing on the image acquired by the VPICTURE instruction. The VTRAIN.FINDER mode-15 instruction with argument 0 tells the finder to use the binary information. It is necessary to set the V.BINARY switch and tell the finder to use the binary information.
After training, the VPLAN.FINDER instruction must be used to generate a recognition plan before recognition can be done. The program binfind( ), shown below, can be used to recognize part instances using binary image processing.
PROGRAM binfind(cam, threshold, back) AUTO times[2] aoi = 2000 cx = 324 cy = 244 width = 612 height = 440 VDEF.AOI aoi = 1, cx, cy, width, height, 0 acq_ibr = aoi+11 ENABLE V.BINARY[cam] PARAMETER V.THRESHOLD = threshold PARAMETER V.MAX.AREA = width * height IF (back == 1) THEN ENABLE V.BACKLIGHT[cam] ELSE DISABLE V.BACKLIGHT[cam] END VPICTURE (cam,, acq_ibr) 2 VTRAIN.FINDER (cam, 15,, 0) times[0] = 2 times[1] = 9000 times[2] = 9000 VFINDER (cam, 1, -1, -1, times[]) acq_ibr VLOCATE (cam, 0, 0) $name, loc DO IF VFEATURE(1) THEN DECOMPOSE xform[] = loc x = xform[0] y = xform[1] ang = xform[5] TYPE "Found model ", $name, " at ", x, ", ", y, ", ", ang TYPE "Verification = ", VFEATURE(9), " percent" END VLOCATE (cam, 0, 0) $name, loc UNTIL NOT VFEATURE(1)Note the setup instructions used to specify certain options for binary processing such as:
These instructions operate in the same manner with the ObjectFinder as with other binary image-processing applications, since the ObjectFinder uses the same algorithms. For example, the V.BACKLIGHT switch, when enabled, specifies that the scene contains dark objects on a light background. If disabled (the default), the scene contains light objects on a dark background.
See the description of VTRAIN.FINDER (cam, mode, dmode, arg, arg2, arg3) $model_name, ibr = value, value2, value3 on This Topic for more details.
Details of Edge Images
The ObjectFinder includes its own edge detection algorithms for extracting features from gray value images. These algorithms are more sophisticated than the edge-detection algorithms available to the application programmer and should be used in most cases. However, there are situations where the programmer may need to use a custom edge-detection algorithm. Some images resemble edge images and may not require any edge processing. In version 12.4, the ObjectFinder has been enhanced to accept an edge image as its input and to use the edge image without further image processing.
In the edge mode, the ObjectFinder bypasses its own default edge-detection process and directly uses the edge image supplied in the instruction. All instructions, parameters, and switches that are relevant to an edge-detection instruction are also relevant to this mode.
The program edgtrain( ), shown below, can be used to train a model using an edge image computed using the VEDGE instruction.
PROGRAM edgtrain(cam, etype, edgestr, $mod) AUTO $fmods[10] VDEF.AOI 2000 = 1, 308, 240, 500, 400, 0 ; Specify the edge strength parameter PARAMETER V.EDGE.STRENGTH[cam] = edgestr ; Extract edges from the image (either Sobel or Cross-gradient) ; depending on the value of the supplied edge type argument VEDGE (cam, etype) 2000 ; Specify edge image option for ObjectFinder VTRAIN.FINDER (cam, 15,, 2) ; Train model VTRAIN.FINDER (cam, 1, 2, 0, 75, 0) $mod, 2000 ; Unset mode and restore default (gray image processing) mode VTRAIN.FINDER (cam, 15,, 1)After training, the VPLAN.FINDER instruction must be used to generate a recognition plan before attempting recognition. The program edgefind( ), shown below, can be used to recognize part instances in an edge image supplied by the user.
PROGRAM edgefind(cam, etype, strength) VDEF.AOI 2000 = 1, 308, 240, 500, 400, 0 ; Set edge strength parameter. PARAMETER V.EDGE.STRENGTH[cam] = strength ; Execute edge detection (either Sobel or Cross-gradient, ; depending on the value of the edge type argument) VEDGE (cam, etype) 2000 ; Set edge image processing option for ObjectFinder VTRAIN.FINDER (cam, 15,, 2) ; Apply object recognition VFINDER (cam, 1, -1, -1) 2000 VLOCATE (cam, 0, 0) $name, loc DO IF VFEATURE(1) THEN DECOMPOSE xform[] = loc x = xform[0] y = xform[1] ang = xform[5] TYPE "Found model ", $name, " at ", x, ", ", y, ", ", ang TYPE "Verification = ", VFEATURE(9), " percent" END VLOCATE (cam, 0, 0) $name, loc UNTIL NOT VFEATURE(1) ; Unset edge image mode. VTRAIN.FINDER (cam, 15,, 1)See the description of VTRAIN.FINDER (cam, mode, dmode, arg, arg2, arg3) $model_name, ibr = value, value2, value3 on This Topic for more details.
|
Please submit comments to: techpubs@adept.com |
|
|
CAUTION: You must already have planned with models you are going to use. Any replanning will undo the learned data.