Intellectual Merit

The Real-Time Robot Control System™ was designed to revolutionize local and remote robot operations much like the computer mouse did with personal computers. It provides a simplified human machine interface (HMI) that drastically reduces operator data input requirements to orchestrate robot movements in real-time. It provides an intuitive feel to what was hopelessly complex and laborious in the past. This patented technology has allowed for the world’s first laser-guided robotic arm. Similar to that of a GPS, the Real-Time Robot Control System™ uses coordinates from a 2D or 3D pointer (3D Laser Distance Meter) to dictate the articulated movement of a robotic device. Think of it like hand/eye coordination. The laser finds its coordinates, similar to the human eye, and directs the robotic hand accordingly. The Real-Time Robot Control System™ can be applied to many precision control industries such as medical and industrial robotics, giving the ability for a unique, real-time control.

The Da Vinci S medical robot Surgical System uses joysticks, levers, and wheels to guide a medical instrument's location. The Da Vinci robot system mimics joints of the human arm, wrists, and fingers. The Da Vinci robot system acts as an exo-skeleton in amplifying human motions while removing the tremors. In the Da Vinci system, instrument position control is determined by trial and error movements of the surgeon's hands. The robot end effector moves relative to the operator’s hands, not relative to the surgery patient. The surgeon gauges relative distances between the patient and surgical tool using a 3D camera. These trial and error motions sometimes result in collateral damage of neighboring tissue. The learning curve for the controls of the Da Vinci system is steep because of the complexities of the joystick, levers, and wheel motions needed for instrument operation. Transferring the human arm, hand, and finger motions requires intense focus and skill. The demand on the surgeon is severe and thus limits the time a surgeon can perform competently. For example, surgeons spend an appreciable amount of time stitching thin-walled membranes with a microscopic curved needle. Countless hours of training are needed to master the sensory and dexterity skills needed for completion of an operation.

The Real-Time Robot Control System™ (RTRCS) uses a 3D laser pointer to guide a surgical instrument’s location instead of gauging the instrument’s distance relative to the surgeon’s hands and from the surgeon’s 3D camera. The 3D laser pointer gives an absolute instrument position based on the 3D pointer’s reference frame. This will reduce the risk of collateral damage to neighboring tissue. The work of finding the appropriate robot joint angles for a given task is done by a computer based on the surgery patient’s position, not relative to the surgeon’s hands or relative to the 3D camera. A virtual reality version of the robot control system will be used as a skills simulator with virtual reality images before actual motion occurs. This virtual reality simulator would allow a surgeon to practice before a critical step is taken. The laser-guided system would be able to map the membrane location and use an end effector specifically designed for stitching thin-walled membranes. The end effector and real-time surface mapping could reduce the countless hours of specialized skills needed for surgical training and would reduce the sensory and dexterity demands required to complete a surgery.

Real-Time Robot Control Proposal for NSF

General Plan of Work

The general plan of work is to develop both a subscale and a full-scale mixed-reality robot control system. The benefits of this approach are similar to the Wright brothers’ developmental path in designing the first airplane. Subscale air tunnel work with less expensive prototypes allowed refinement of the critical features of the original aircraft’s design with less capital expenditure. Likewise, one can leverage the software and hardware components developed in the subscale mixed reality robot control system to be used in the full-scale version. The penalty for changing directions with a design approach is minimized when the decisions are based on subscale parts. One societal benefit from this approach is the subscale version of the virtual reality system can be used in education and research environments to train others to use the technology in other applications. A second societal benefit is the subscale system can be modularized in the research environment to provide benefits to other industries.

  1. The insights offered by the NSF panel of experts allow more refinement in the next presentation whether it is to NSF or to other investors
  2. The NSF review panel agreed that the concept can be applied to multiple fields
    a. "Overall, while the laser-guided robotics seems to be applicable in multiple fields, these are not well articulated in the proposal." (Quote from one panel member)
    b. "The product has a good potential in the targeted medical market." (Quote from second panel member)
    c. "The mixed-reality control systems of robots envisioned in the proposal are interesting." (Quote from third panel member)
  3. One panel member agreed with the particular potential benefits for medical robotics:

  • In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to intellectual merit:
    o + Enables robotic actuators to be positioned with user guidance from a mouse or laser pointer.
    o + Reduces error and fatigue associated with human operation of robotic systems by conventional means.
    o + Accommodates local and remote robotic control.
    o + Control is more intuitive and user-friendly, with motion trajectories projected in a GUI.

  • In the context of the five review elements, please evaluate the strengths and weaknesses of the proposal with respect to broader impacts:
    o + Surgeons could learn the use of robotic tools more rapidly and easily.
    Note: The "+" sign means it is a benefit to medical robotics.