Robot System Design

The Challenge

Industrial robots must operate in cluttered environments with obstacles while maintaining precise positioning and avoiding collisions. This project required creating a simulation environment and implementing path planning for a robot to guide it in accurately tracing a model design within an assigned workspace.

1. Simulation Environment Setup

Built a virtual workcell which is a table in SolidWorks and:

• ABB IRB 120 Robot: 6-DOF industrial manipulator with 3kg payload capacity
• Workspace Layout: 1 table (picking and placing)
• Obstacles: CAD model of flag with traceable text "RSD"

ABB IRB 1600-6/1.2 industrial robot: 6kg payload, 1.2m reach, 0.02mm repeatability

ABB IRB 1600 manipulator axes configuration showing 6 degrees of freedom

Custom "RSD" flag design created in SolidWorks, exported as .SAT format for RobotStudio import

RobotStudio simulation environment

2. RAPID Code

• The four initial paths were merged into a single path named **COMPLETE PATH**. The **RAPID code** was structured to call this path in the main procedure function (`PROC main() COMPLETE_PATH; MOVE_HOME; ENDPROC`), ensuring the entire tracing task executed as a single function call.
• Simulation Control: A `MOVE_HOME` function was implemented, along with a **WaitTime 20** function, allowing the simulation to pause before restarting the task. This enabled the robot to run in **continuous run mode** for recording.
• Deliverables: The project successfully achieved the simulation and tracing goal, with deliverables including a video demonstration and the **Pack and Go file** from RobotStudio, which allows the simulation programme to be accessed on other computers.

PROC main()
      COMPLETE_PATH;
      MOVE_HOME;
      ENDPROC

      PROC COMPLETE_PATH()
      ! Trace flag pole and letters R, S, D
      MoveL Target_Flag_Start, v600, fine, MyTool\WObj:=Table_Workobject;
      ! ... (path targets for flag)
      MoveL Target_R_Start, v600, fine, MyTool\WObj:=Table_Workobject;
      ! ... (path targets for letter R)
      MoveL Target_S_Start, v600, fine, MyTool\WObj:=Table_Workobject;
      ! ... (path targets for letter S)
      MoveL Target_D_Start, v600, fine, MyTool\WObj:=Table_Workobject;
      ! ... (path targets for letter D)
      ENDPROC

      PROC MOVE_HOME()
      MoveL Target_930_4, v600, fine, MyTool\WObj:=Table_Workobject;
      WaitTime 20;  ! Pause before restarting cycle
      ENDPROC

3. Robot Specifications & Selection Rationale

Selected ABB IRB 1600-6/1.2 for its balanced specifications and workspace coverage:

The Challenge

Modern industrial robots require vision systems to handle unstructured environments where object positions are not predetermined. This project involved creating a hand-eye coordination system capable of detecting, localizing, and grasping objects of varying sizes and orientations using computer vision and coordinate transformation.

Project Planning & Team Collaboration

Executed as a 3-person team project with structured task distribution and timeline management:

Project timeline and work distribution showing coordinated effort across image processing, robot programming, and kinematics analysis modules

My System Architecture

System Components

Webcam

Image Acquisition

MATLAB

Image Processing

TCP/IP

Communication

ABB IRB 120

Robot Execution

1. Camera Calibration & Setup

Performed comprehensive camera calibration to ensure accurate coordinate mapping:

• Calibration Method: Zhang's checkerboard-based camera calibration
• Parameters Determined: Intrinsic matrix (focal length, principal point), distortion coefficients (radial & tangential)
• Calibration Accuracy: Mean reprojection error <0.3 pixels
• Camera Mounting: Fixed overhead position at 800mm height, 45° viewing angle

Multiple checkerboard views captured for camera calibration. Grid: 9×6 corners, square size: 25mm

Physical setup showing ABB IRB 120 robot, overhead camera, workspace, and target objects

2. Image Processing Pipeline

Developed a robust MATLAB-based vision processing pipeline:

Image Acquisition & Preprocessing

• Undistortion: Applied calibration parameters to remove lens distortion
• Color Space Conversion: RGB → Grayscale for intensity-based processing
• Noise Reduction: Gaussian filtering (5×5 kernel, σ=1.0)
• Contrast Enhancement: Adaptive histogram equalization (CLAHE)

Object Detection & Segmentation

• Thresholding: Adaptive binary thresholding (Otsu's method)
• Morphological Operations: Opening (erosion + dilation) to remove noise
• Connected Components: Blob analysis for individual object identification
• Contour Analysis: Boundary extraction using Canny edge detection

Feature Extraction

• Centroid Calculation: Geometric center using image moments
• Bounding Box: Minimum area rectangle for object dimensions
• Orientation: Principal axis angle using second-order moments
• Area & Perimeter: For object classification

Processing Pipeline Visualization

Step-by-step transformation from raw capture to feature extraction:

1. Original capture

2. Grayscale conversion

3. Binary thresholding

4. Object masking

Final Result: Detected objects with centroids, bounding boxes, and orientation vectors overlaid. Object properties (position, rotation, area) extracted for robot control.

3. Coordinate Transformation

Implemented precise transformation from image coordinates to robot base coordinates:

Transformation Chain

1. Image → Camera Frame: Using intrinsic calibration matrix and known Z-height (workspace plane)
2. Camera → Robot Base: Homogeneous transformation matrix from hand-eye calibration
3. Validation: Physical measurement verification (target grid points)

    [R₁₁  R₁₂  R₁₃  Tx]     [Xrobot]
	  T = [R₂₁  R₂₂  R₂₃  Ty]  →  [Yrobot]
	  [R₃₁  R₃₂  R₃₃  Tz]     [Zrobot]
	  [ 0    0    0    1]     [  1  ]

	  Rotation Component (R): Camera → Robot orientation
	  Translation (T): Camera → Robot position offset

	  Example: Camera at X=450mm, Y=300mm, Z=800mm above robot base

4. Robot Control Integration

Developed seamless communication between MATLAB vision system and ABB robot controller:

Communication Protocol (TCP/IP)

• Socket Connection: MATLAB client → ABB IRC5 controller server
• IP Address: 192.168.125.1 (robot controller)
• Port: 1025 (custom RAPID socket server)
• Data Format: ASCII string "X,Y,Z,RX,RY,RZ" (comma-delimited coordinates)
• Acknowledgment: Robot returns "OK" upon successful move completion

RAPID Programming (Robot-Side)

• Socket Server: Listens for incoming coordinate data
• Motion Planning: Generates safe trajectory to target pose
• Execution: MoveL (linear) or MoveJ (joint) motion commands
• Error Handling: Timeout detection, unreachable position checks

Denavit-Hartenberg coordinate frames for ABB IRB 120 kinematic model showing link assignments

ABB IRB 120 dimensional specifications for kinematic calculations and workspace analysis

Denavit-Hartenberg Parameters

DH table for ABB IRB 120 forward/inverse kinematics implementation:

3D robot model in MATLAB Robotics Toolbox using DH parameters for forward/inverse kinematics validation

RobotStudio joint jog interface showing end-effector position verification for kinematics accuracy (error <1%)

5. System Testing & Performance

Conducted extensive testing to evaluate positioning accuracy and system reliability:

Key Results & Insights

Key Learnings & Improvements

• Lighting Sensitivity: System accuracy degraded by ~40% under variable lighting. Implemented controlled LED ring light to maintain consistent illumination.
• Calibration Drift: Periodic recalibration (every 50 cycles) necessary to compensate for mechanical vibrations and thermal expansion.
• Z-Height Assumption: Current system assumes planar workspace. Future enhancement: integrate depth sensor for 3D object localization.
• Multi-Object Handling: Successfully handled up to 5 objects simultaneously. Priority-based selection implemented (nearest-first strategy).