Publications

A portable robotic system, such as the HRX-1 robot designed for neuromechanics and rehabilitation research, enables motor assessment and training across diverse settings including clinics, laboratories, and home environments. This study presents the findings of wrist muscle activation measurements conducted using the HRX-1 robot in conjunction with surface electromyography (sEMG) electrodes on a cohort of fifteen healthy participants. Participants were seated and instructed to resist and maintain wrist flexion and extension against torques applied to their hand by the robot. Muscle activation data were recorded using two 32-channel high-density surface EMG electrodes placed on the forearm to capture activity from the flexor and extensor muscle groups. The analysis focuses on identifying the number of active regions within the recorded muscle activations under each experimental condition. As expected, flexor muscles were most active during wrist flexion and extensor muscles were most active during wrist extension. There was also no statistically significant change in the number of muscle regions active when torque was increased in each configuration.

Colour Vision Deficiency (CVD) also known as colour blindness affects many people in their day-to-day lives, reducing their ability to distinguish certain colours. The pre-dominant types of CVD are red-green CVD and blue-yellow CVD. We introduce colour spectrum shifting to aid CVD users, enhancing their colour differentiation by making better use of the existing range of colours they can perceive. Spectrum shifting is demonstrated within ASTER, an iOS phone app applying real-time image processing to the live camera feed. Using ASTER, users with CVD can now correctly identify all the numbers in a stand-ardised Ishihara test and they also report it effective in everyday use.

The remote operation of underwater vehicles at depth is complicated by the presence of invisible and unpredictable environmental disturbances such as cross-currents. Communicating the presence of these disturbances to an operator on the surface is made more difficult by the nature of the disturbance and the lack of visible features to highlight in the visual display presented to the operator. Here we explore the use of a novel interactive soft haptic touchpad that utilizes vibration and particle jamming to provide information about the presence and direction of cross-currents to the operator of an ROV (remotely operated vehicle). An in water experiment using a thruster-based ROV and artificially generated cross-current was performed with non-expert ROV operators to evaluate the effectiveness of multimodal haptic feedback to communicate complex environmental information during high-risk operations. Advanced haptic displays can signal both the presence of external factors as well as their direction, information that can enhance operational performance as well as reduce operator cognitive load. Using haptic feedback resulted in a statistically significant reduction in cognitive load of 24.3% and increase in positioning accuracy of 28.3% for novice operators. Deviation from an ideal path was also reduced by 29.5% for experienced operators when using haptic feedback compared to without. While this experiment took place in controlled conditions with a fixed direction cross-current and haptic interface, this approach could be extended to communicate real-time environmental information in real-world unstructured environments.

Physical interaction with patients, for example conducted as part of a diagnostic examination or surgical procedure, provides clinicians with a wealth of information about their condition. Simulating this interaction is of great interest to researchers in both haptics and medical education, and the development of softness changing tactile interfaces is important in recreating the feel of different soft tissues. This paper presents designs for a variety of novel electromechanical and electromagnetic mechanisms for controlling particle jamming-based, hardness changing tactile displays, intended to allow medical trainees to experience these physical interactions in a range of simulation settings such as clinical skills teaching laboratories. Each design is then subjected to a battery of mechanical tests to evaluate its effectiveness compared to the state of the art, as well as their suitability for simulating the physical hardness of different types of soft tissues, previously characterised in established literature. These results demonstrate that all of the technologies presented are able to exhibit a measurable hardness change, with Shore hardness values between 3A and 57A achieved by the most effective constriction-based device. The electromechanical devices based on constriction and compression, and the state-of-the-art pneumatic device, were able to achieve hardness changes within a range that is useful for replicating the softness of organic tissue. The electromechanical and electromagnetic devices were also found to effect their full range of hardness change in less than a second, compared to several seconds for the state-of-the-art. These results show that the performance of softness changing tactile displays can be improved with the electromechanical actuation techniques proposed in this paper, and that such displays are able to replicate the physical characteristics of soft tissues and may therefore be of benefit in medical training and simulation scenarios.

The Dermaphone is an educational tool for medical students learning dermatology. Within the application, consultants can use photogrammetry to create 3D models of skin lesions. These are presented to students with texture rendered through vibration by the iPhone’s taptic engine. The application is designed to give medical students exposure to dermatology and act as a study resource to fill gaps from their limited opportunities for practical experience. A framework for crowdsourcing a skin lesion library of 3D models and associated texture data is also provided, built as more educators contribute scans from real life cases.

Particle jamming is an emergent technology widely used to create haptic devices that can change their physical stiffness to render hard or soft surfaces. Conventional implementations of particle jamming-based interfaces have relied on bulky and expensive vacuum systems to force the particles together. This paper presents designs for two alternative, mechatronic approaches to activating a particle jamming-based haptic interface. Each design is subjected to a battery of mechanical tests to evaluate the range and uniformity of the achievable hardness change and response time. Results are presented and the effectiveness of these designs is considered against established pneumatic approaches.

Haptic technology is no longer limited to rendering simple effects in game controllers and smartphones. Recent advances in the design of haptic devices have made important advances in both the rendering of high-definition haptic sensations and integrating haptic feedback into an ever expanding range of digital technologies. However, true high information density haptic interfaces still remain out of reach. This PhD project aims to make progress toward higher information-density haptic interfaces by layering multiple haptic effects on top of each other, such that they can be actuated independently. The project has both technical and scientific goals. The technical objective is to design, validate and utilize a novel soft technology based on particle jamming for rendering tactile hardness, vibration and shape change. The scientific goal is to use this technology to conduct empirical studies into how multiple haptic cues can be layered on top of one another, perceived and understood by a user. This thesis begins with an overview of the state of the art in haptic technology, followed by in depth reviews into aspects of soft haptics and applications of haptic feedback in telerobotics. A method for combining multiple haptic cues is then described and prototypes demonstrated and characterised. The technology is integrated into two interactive haptic interfaces and used in two scientific studies, one addressing the psychophysics of multi-modal haptic feedback, the other investigating applications of haptic feedback in a difficult robot teleoperation task.

The principle of particle jamming, a physical effect where fluids can be made to change their hardness at will, has many applications in engineering. Previous research has investigated combining this change of hardness with other haptic effects, resulting in a technology that can render vibration, hardness/softness and shape. This paper proceeds to describe the application of this technology to a soft haptic joystick handle for use in interactive games and telerobotics scenarios. Dynamically generated sound waveforms are used to drive vibrations inside the handle, and a microphone records these as they reach the tip of the handle under different jamming conditions. Audio frequency analysis is then used to analyse the behaviour of the resulting vibrations. This analysis shows that vibration is lowest under a strong vacuum, confirming previous observations that increasing the hardness of the particle fluid has the effect of restricting the displacement of the source vibrations. Moreover, frequency of vibration remained broadly stable in both hard and soft states again confirming previous observations. These results, obtained with a fundamentally different haptic device and sound-based instrumentation, necessitate the conclusion that the behaviour of particle jamming controlled vibration is repeatable and controllable regardless of the physical configuration in which it is used.

This paper presents the design and experimental characterisation of a wrist haptic interface based on a twisted string actuator. The interface is designed for controlled actuation of wrist flexion/extension and is capable of rendering torque feedback through a rotary handle driven by the twisted string actuator and spring-loaded cable mechanisms. The interface was characterised to obtain its static and dynamic haptic feedback rendering capabilities. Compliance in the spring and actuation mechanism makes the interface suitable for smooth rendering of haptic feedback of large magnitudes due to the high motion transmission ratio of the twisted strings. Haptic virtual wall rendering capabilities are demonstrated.

Dealing safely with nuclear waste is an imperative for the nuclear industry. Increasingly, robots are being developed to carry out complex tasks such as perceiving, grasping, cutting, and manipulating waste. Radioactive material can be sorted, and either stored safely or disposed of appropriately, entirely through the actions of remotely controlled robots. Radiological characterisation is also critical during the decommissioning of nuclear facilities. It involves the detection and labelling of radiation levels, waste materials, and contaminants, as well as determining other related parameters (e.g., thermal and chemical), with the data visualised as 3D scene models. This paper overviews work by researchers at the QMUL Centre for Advanced Robotics (ARQ), a partner in the UK EPSRC National Centre for Nuclear Robotics (NCNR), a consortium working on the development of radiation-hardened robots fit to handle nuclear waste. Three areas of nuclear-related research are covered here: human–robot interfaces for remote operations, sensor delivery, and intelligent robotic manipulation.

During the pandemic healthcare faced great pressure on the availability of protective equipment. This paper describes the entire novel innovative process of design optimisation, production and deployment of face-visors to NHS frontline workers during SARS-CoV-2 pandemic. The described innovative journey spans collaboration between clinicians and academic colleagues for design to the implementation with industry partners of a face-visor for use in a healthcare setting. It identifies the enablers and barriers to development along with the strategies employed to produce a certified reusable, adjustable, high volume and locally produced face-visor. The article also explores aspects of value, scalability, spread and sustainability all of which are essential features of innovation.

Studies of social behaviors in animals are faced with various methodological difficulties, which can be addressed by using controlled artificial social agents. Previous studies have shown that various animal species interact with passive replicas or interactive robots that mimic their conspecifics. In the case of chickens, filial attachment (imprinting) to robots is observed in young chicks. However, the features and functions of the robots that maximize the efficiency of chicken-robot attachment have not yet been identified. Therefore, we designed RoboChick, a simple robot that can be easily customized with different features. Further, we developed a protocol for assessing the attractiveness of each feature. In the current study, we tested the attractiveness of two RoboChick features during robot-chick interactions: the presence of flashing lights and vocalizations in response to chick interactions. Our proposed protocol proved suitable for assessing the efficacy of the features. RoboChick, which is open and modular, can be easily reproduced by other research groups and adapted to test different features in different experimental conditions.

This extended abstract describes the design of a haptic interface based on a joystick to provide vibrotactile, shape-changing and hardness/softness based feedback to the operators of remote mobile robots.

This extended abstract describes the design of a haptic interface based on particle jamming and a pair of vibration actuators for simultaneous rendering of vibration, shape and softness/hardness sensations.

This work-in-progress project describes the evolution of a design for a small form factor, affordable tactile display using simple, easily obtainable motors and mechanical linkages. Performance characteristics of each display are given, as are plans for experimental validation of their potential to support visually impaired smartphone users.

Whilst common in devices ranging from smart-phones to game controllers, vibrotactile feedback has generally been limited to providing a uniform sensation across a tactile surface. We propose a haptic interface based on the emerging physical effect of particle jamming with both vibrotactile and shape changing outputs, which can be extended in space to create haptic surfaces and devices with shape and vibrotactile responses localised to one part of the device. This paper gives an overview of the physical principles behind this technology and presents detailed performance metrics obtained from a working prototype. These include experimental characterization of the relationships between air pressure and electric motor power and vibration amplitude and frequency which show that it is possible to control vibrotactile amplitude and frequency independently.

Riffling the pages of a book, perhaps in the search for a specific image, is an example of Rapid Serial Visual Presentation (RSVP). Even at a pace of 10 images per second, successful search is often possible. Interest in RSVP arises because a digital embodiment of RSVP has many applications. There are many possible ’modes’ of RSVP. However, a mode can be especially helpful if, after the appearance of an image, and without delaying the arrival of other images, it can remain in view for a second or two to allow a user to confirm that a desired image has been found. Moreover, if a collection of images is presented in such a way as to be perceived as moving in 3D space, it is thought that the search for an individual image can thereby be enhanced by comparison with a 2D presentation. To test this conjecture we devise and use the "Deep-Flat" visual illusion whereby a column of moving images magnifying in size is perceived as approaching the viewer as in a 3D space. When the images are presented in an equivalent way horizontally as a row, the viewer tends to see this as images growing in size, but now on a flat (2D) plane. We tested comparable RSVP designs in these two illusions to ascertain the relative effects of 2D and 3D style presentation under precisely controlled conditions. Elicited data included both performance measures (e.g., recognition success), and user preferences and opinions. We established the effectiveness of RSVP using the illusion. When tested under directly comparable conditions, we concluded that performance is not significantly affected by the illusion of depth, but that the inclusion of certain background cues can have a significantly detrimental effect on performance.