This project aims to develop an immersive telediagnostic platform for remote sessions in situations where there are sensitive relationships, such as counsellor-patient, which require rich communication cues including non-verbal behaviour and physiological signals.

The information provider (e.g., patient) will wear a virtual reality (VR) headset and see a real counsellor (e.g., doctor) in a virtual space as if meeting in the real world. Verbal and non-verbal behaviours, along with physiological signals, will be collected and analysed using a deep learning approach in real time.

The research team aims to develop the telediagnostic platform, analyse non-verbal cues for rich communication information, and to evaluate the system as an effective consultation tool for complex dyadic conversations.

This project is funded by a grant from the Science for Technological Innovation National Science Challenge (SfTI) – Kia Kotahi Mai: Te Ao Pūtaiao me Te Ao Hangarau programme hosted by Callaghan Innovation and the Ministry of Business, Innovation and Employment.

People