Clinical-friendly Drug Psychotherapy with Virtual Reality and Biofeedback Technologies

This study explores how to incorporate virtual reality and biofeedback technologies to assist psychotherapists in raising patients' awareness and identifying their cues in psychotherapy.

Website: TBA

Publication: CHI 2021 (Late-Breaking Work)

An example screenshot of cues appearing in the virtual reality video.

Enabling Social Sensing for Gig Workers

we propose that social sensing could address bottlenecks in worker's awareness, interpretation of the informatics, and subsequent behavioral change. Through a four-week technology probe study, we found the "social sensing" design was able to further shape such awareness or intention into action, highlighting the potential of using the sociotechnical approach in promoting work-life balance among long-hour workers.

Website

Publication: CHI 2021 (full paper), UbiComp 2019 (poster)

DriveSense is a phone application to enable social sensing to share personal data with intimate partner for the health and wellbeing of long-hour workers.

SoberComm

A phone-based support system aimed at facilitating the sharing of alcohol-use data between patients and their family members, while providing treatment teams with the data necessary to make constructive suggestions concerning inter-family communications.

Website

Publication: UbiComp 2019 (full paper), UbiComp 2018 (poster)

SoberComm is a phone application to enhance problem-solving skills and facilitate communication between alcohol-dependent patients and their family members.

SoberMotion

A phone-based support system to assist DUI offenders on probation in recording alcohol use by using customized breathalyzers connected to a phone app installed on their phones via Bluetooth. The phone app also identifies high-risk commuting events to remind offenders checking if they are sober before operating vehicles.

Website

Publication: UbiComp 2018 (full paper), UbiComp 2017 (poster)

SoberMotion is a phone application to support DUI offenders on probation to reduce the risk of re-committing DUI.

RehabDiary

A phone-based support system for helping ketamine-dependent patients maintaining abstinence. We design and implement this system to help patients self-monitor their ketamine use after withdrawal treatment with a customized saliva screening device, which is wirelessly connected to the patient’s smartphone. 

Website

Publication: CHI 2016 (Note paper), UbiComp 2015 (poster)

RehabDiary is a phone application to help ketamine dependent patients to maintain abstinence.

SoberDiary

A phone-based support system to assist alcohol recovery. We design and implement this phone-based support system to enable continuous patient monitoring and provide feedback support to the patients. This study explores the use of mobile phones as a normal life support system that connects alcohol addict patients to their addiction physiatrists after they leave a rehabilitation center.

Website

Video (English version, 49.5MB), Video (Chinese version, 49.5MB)

Publication: Addictive Behaviors , CHI 2015 (full paper), UbiComp 2014 (video), CHI 2013 (poster)

SoberDiary is a phone application to help alcohol dependent patients to maintain sobriety.

BioScope

A sensor-embedded smart bandage system. The system has a LEGO-like extensible design that enables health professionals to customize different sensor patches and monitor different health signals from patients. Arduino prototyping and programming, Matlab programming, embedded system programming.

Publication: UbiComp 2014 (short paper), UbiComp 2017 (poster)

BioScope is a sensor-embedded smart bandage system for Facilitating Data Collection in Nursing Assessments.

Outdoor Smart Parking

The goal is to build an interactive outdoor smart parking system that can facilitate users to search available parking slots and ease facility management. Currently conducting data collection in a parking zone and designing detection algorithms which recognize learned patterns associated with car arrivals or departures, and filter out unreliable events based on spatial and temporal characteristics.

Website

Publication: PerCom 2017 (full paper), SenSys 2015 (poster)

An IoT-based outdoor smart parking sensor node.

CarSafe

A driver safety app that detects dangerous driving behavior using dual-cameras on smartphones. We developed the CafeSafe app (the first dual-camera application) for Android phones, which fuses information from both front and back cameras and others embedded sensors on the phone to detect and alert drivers to dangerous driving conditions in and outside of the car. CarSafe uses computer vision and machine learning algorithms on the phone to monitor and detect whether the driver is tired or distracted using the front camera while at the same time tracking road conditions using the back camera. The system implementation involved extensive Android and OpenCV programming and machine learning toolkits. (Collaborated with Prof. Andrew Campbell at Dartmouth College)

Video (47.3MB)

Slide (44.2MB)

Publication: Mobisys 2013 (full paper), UbiComp 2012 (poster, demo, video)

CarSafe is the first dual-camera app which detects dangerous driving behavior using smartphones.

Visage

A face interpretation engine that senses users’ visual feedback and utilizes their face responses on smartphones. Visage fuses data streams from the phone’s front-facing camera and built-in motion sensors to infer, in an energy-efficient manner, the user’s 3D head poses (i.e., the pitch, roll and yaw of user’s heads with respect to the phone) and facial expressions (e.g., happy, sad, angry, etc.). Visage supports a set of novel sensing, tracking, and machine learning algorithms on the phone, which are specifically designed to deal with challenges presented by user mobility, varying phone contexts, and resource limitations. The system implementation involved extensive iPhone and OpenCV programming. (Collaborated with Prof. Andrew Campbell at Dartmouth College)

Publication: MobiCase 2012 (full paper)

Visage a face interpretation engine that senses users’ visual feedback and utilizes their face responses on smartphones.

ConvenienceProbe

ConvenienceProbe is a phone-based data collection system for Consumer Behavior Research, which gathers “offline” consumer data from everyday users and their phones. ConvenienceProbe specifically targets local residents shopping at neighborhood convenience stores. We collaborated with a consumer behavior scientist (Prof. Lien-Ti Bei, Vice Dean, College of Commerce, NCCU) and deployed and tested the system by collecting real customer flow data to neighborhood convenience stores. Results show that the consumer flow data collected from the ConvenienceProbe system is comparable to that from a traditional face-to-face interview method. The system implementation involved extensive Android programming and Google Web mapping toolkits (Google Earth & Google Map).

Publication: UbiComp 2010 (poster), IEEE pervasive magazine

ConvenieceProbe is a phone-based data collection system for consumer behavior research.

ShoppingTracker

ShoppingTracker is a phone-based system that uses sensors on a phone to recognize user shopping activities at physical stores. The goal here is to bring awareness to users on the amount of time they spent shopping and the amount of money they spent. The system first reconstructs a shopper’s movement trajectory by analyzing accelerometer and digital compass readings and then extracts features in the movement trajectory that matches layouts at physical stores. If the temporal-spatial features of the shopper’s movement trajectory and those induced by a store layout match, this movement trajectory likely corresponds to shopping. By recognizing time intervals with those shopping trajectories, total shopping time can be determined by adding up all time intervals which are labeled as shopping.  The system implementation involved extensive Android programming and SVM toolkits.

Publication: IEEE pervasive magazine

ShoppingTracker is a phone-based system which uses sensors on a phone to recognize user shopping activities at physical stores.

SocialCRC

A mobile rendezvous coordination application that enables socially aware rendezvous coordination and to simplify the process of coordinating an impromptu rendezvous. SocialCRC identifies a more satisfactory rendezvous point by considering both contextual information and the social relationships between rendezvous attendees. The system implementation involved considerable Android programming and Web programming.

Publication: CHI 2010 (poster), Pervasive Mobile Computing

SocialCRC is a phone-based rendezvous coordination tool.

Recognition of On-Premise Signs (OPSs)

A probabilistic framework for recognition of real-world OPSs. We first proposed an OPS dataset, namely OPS-62, in which totally 4,649 OPS images of 62 different businesses are collected from Google’s Street View. Further, we developed a probabilistic framework based on the distributional clustering, in which we proposed to exploit the distributional information of each visual feature (the distribution of its associated OPS labels) as a reliable selection criterion for building discriminative OPS models. Experiments on the OPS-62 dataset demonstrated the outperformance of our approach over the state-of-the-art pLSA models with a significant 151.28% relative improvement in the average recognition rate. The system implementation involved extensive Matlab programming.

Publication: IEEE Transactions on Image Processing (TIP)

A probabilistic framework for recognition of real-world objects.

AttachedShock

A novel target selecting technique on augmented reality devices using goal-crossing actions. The changing target movement pattern creates difficulty for users in selecting the targets on time before targets escape from the screen. AttachedShock eases target selection tasks on augmented reality devices by crossing a naturally expanding wave pattern that is attached to targets. We evaluated the effectiveness of the proposed technique by conducting comparative studies on measuring the performance of four techniques under various mobile navigation scenarios in an emulated Adobe Flash program. The system implementation involved extensive Adobe flash programming.

Video (6.61MB)

Publication: MM 2012 (short paper), International Journal of Human-Computer Studies

AttechedShock is a Moving Targets Acquisition on Augmented Reality Devices using Goal-crossing Actions.

HeatProbe

HeatProbe is a per-appliance power meter system that uses thermal imaging to track the power consumption of individual appliances. Given the strong link between energy and behavior, tracking an individual energy footprint is critical for designing personalized feedback to reflect and promote individual energy-saving behavior. We have designed, prototyped, and tested the HeatProbe system. Results show that HeatProbe successfully tracks per-user power consumption within an average error of 27 %. The system implementation involved considerable OpenCV programming.

Video (48MB)

Publication: Pervasive 2011 (poster), UbiComp 2011 (full paper)

HeatProbe is a thermal-based personal electric energy consumption system.

ThermalProbe

ThermalProbe is a per-user power meter system that uses thermal imaging to track the power consumption of individual users in a shared space. Given the strong link between energy and behavior, sensing and metering per-user energy consumption is critical for understanding individual energy behavior and for customizing personalized feedback to promote energy-saving behavior. This research explores the feasibility of per-user energy metering by proposing a per-user energy metering system that uses thermal-imaging and thermal-identification to track and associate energy usage among individual occupants in a shared working/living space. Each occupant wears a thermal tag that emits a unique temperature signature for user identification. The system introduces location-based per-user energy disaggregation that accounts for per-appliance energy usage to individual energy consumer(s), i.e., occupant(s) nearby activated appliances. We have designed, prototyped, and tested the ThermalProbe system. Results show that the system meters per-user energy consumption with an average error of 12.66%. The system implementation involved considerable embedded hardware prototyping with the Arduino platform and OpenCV programming.

Publication: GreenCom 2014 (full paper)

ThermalProbe is a thermal-based personal electric energy consumption system.

Sensor network localization system

Numerous ubiquitous computing applications require the deployment of a sensor network infrastructure to collect a variety of data sensed from the physical world. These sensor data are then processed to implement different digital services that can exhibit intelligent context-aware behaviors by automatically adapting their services to changing environments. To make correct inferences on these sensor data, these systems require reliable and accurate location information in the observed sensor data, necessitating accurate location tracking in sensor networks. To address this need, I designed, implemented, and evaluated two localization systems: a Zigbee-based fingerprinting system and a centimeter-level interferometric system named “adaptive RIP system”. Furthermore, an 80-node sensor network was deployed in the CSIE building in NTU to conduct real-time tracking. The system implementation involved considerable embedded system programming such as TinyOS nesC and scripting languages.

Energy-efficient localization algorithm

Energy efficiency and positional accuracy are frequently contradictive goals. To decrease power consumption without sacrificing significant accuracy, I designed, implemented, and evaluated an energy-aware adaptive localization system using the deployed signal strength fingerprinting. The energy-efficient localization system saved energy by adapting its sampling rate based on the tracked object’s mobility level. The system implementation involved considerable TinyOS nesC programming and PowerTOSSIM simulation.

Location-aware services

As indoor localization technologies become more affordable and sophisticated, location-aware services for indoor environments increase in commercialization potential. Since many of the indoor location-aware applications have been prototyped and proven useful, similar location-aware applications will become increasingly popular in Taiwan. The first adopter, the first real daily use, will arise from the office environment where the need is the most critical. Therefore, I implemented and deployed a localization system in collaboration with IBM’s location-based service (LBS) to track personnel in the IBM NanKang office. The system implementation involved considerable Web programming. (Collaborated with IBM China Software Development Lab (CSDL))

Zigbee-based fingerprinting system in the CSIE building in National Taiwan University.

Adaptive RIP system on a square near the sport stadium of National Taiwan University.

Location-based service in the IBM NanKang office.