Soul Sketch is an intelligent, cross-platform mobile application designed to digitize and streamline psychological assessments. It addresses the limitations of traditional paper-based methods by integrating AI-assisted testing and real-time physiological monitoring. The app centralizes patient management, drawing-based tests (HTP, HFD), and questionnaires (BAI), utilizing AI models for analysis and generating detailed reports. It also features a GSR (Galvanic Skin Response) hardware module to monitor patient stress levels, bridging the gap between cognitive evaluations and physiological data to enhance diagnostic accuracy.
Objectives
? To provide a centralized mobile platform for conducting psychological assessments such as HTP, HFD, BAI, and the Predict Career Test.
? To assist therapists in efficiently managing patient data, history, and test results.
? To generate psychological reports based on test inputs and physiological data using AI analysis.
? To enable real-time monitoring of patients' physiological signals (GSR) using hardware sensors integrated with a Python Server and Firebase.
? To support therapists with an AI chatbot for guidance and allow the management of mental health blogs for awareness.
? To reduce manual paperwork and improve the accessibility and accuracy of psychological evaluations.
Socio-Economic Benefit
? Clinical Efficiency: The system automates repetitive tasks like record-keeping and report generation, allowing therapists to focus more on patient interaction and treatment planning.
? Enhanced Diagnostics: By fusing cognitive-emotional assessments with physiological data, it supports evidence-based mental health care and more precise identification of disorders.
? Societal Accessibility: The platform is suitable for clinics, rehabilitation centers, and educational institutions. Its drawing-based assessments are particularly beneficial for children, trauma survivors, and patients who have difficulty expressing emotions verbally.
Methodologies
The development of Soul Sketch follows a composite, multi-layered architectural approach that integrates mobile application development, cloud computing, artificial intelligence, and hardware-based physiological monitoring. The methodology is divided into three distinct but interconnected modules:
1. Mobile Application and Backend Architecture:
The frontend is built using the Flutter framework with the Dart programming language, chosen for its cross-platform capabilities to ensure a consistent user experience on Android devices. The application utilizes a provider-based state management system to handle complex data flows between the Admin, Therapist, and Patient dashboards. For the backend, the system operates on a serverless architecture using Firebase. Firebase Authentication manages secure role-based access control, while Cloud Firestore serves as the real-time NoSQL database to synchronize patient records, test history, and chat logs instantly across devices. Firebase Cloud Storage is utilized to securely store unstructured data, such as the digital drawings generated during HTP and HFD tests.
2. Artificial Intelligence and Server Integration:
The core intelligence of the system is hosted on a dedicated Python Server that acts as an API gateway. This server hosts the Machine Learning models, specifically Convolutional Neural Networks (CNNs) and Fully Connected Dense Layers, which are trained to analyze visual data from psychological drawing tests. Additionally, the system integrates the Gemma 3B to power the AI chatbot and generate comprehensive textual summaries for psychological reports. The mobile application communicates with this Python server via RESTful APIs to send test data and receive diagnostic insights, which are then stored back into Firestore for retrieval.
3. Hardware and IoT Implementation (GSR Module):
The physiological monitoring component employs a specific hardware pipeline to ensure safe and accurate data transmission. A GSR (Galvanic Skin Response) sensor collects raw analog skin conductance data, which is first processed by an Arduino Nano. To ensure voltage compatibility and hardware safety, a Bi-directional Logic Level Shifter bridges the 5V Arduino signal to the 3.3V ESP32 Microcontroller. The ESP32, utilizing its built-in Wi-Fi capabilities, transmits the processed physiological data packets to the Python Server. This data is then relayed to the Firebase Realtime Database, allowing the therapist to view live stress metrics (graphs and numerical values) on their mobile dashboard with minimal latency.
Outcome
The primary outcome is the deployment of a fully functional, cross-platform mobile application that digitizes the entire psychological assessment workflow. Built with Flutter, the application successfully establishes a centralized environment where Administrators, Therapists, and Patients can interact securely. This digital platform replaces traditional manual methods, effectively managing user roles, authentication, and data privacy through a robust Firebase backend.
The project successfully implemented a comprehensive suite of digital psychological tests. This includes interactive drawing modules for the House-Tree-Person (HTP) and Human Figure Drawing (HFD) tests, where patients can sketch directly on the device. Additionally, the system includes digitized versions of the Beck Anxiety Inventory (BAI) and a Career Prediction test. These modules are fully operational, allowing for the seamless collection of patient responses without the need for physical paper or manual scoring.
A significant technical achievement is the successful integration of Artificial Intelligence for automated analysis. The system employs Convolutional Neural Networks to process and interpret the drawing-based tests, identifying patterns that aid in diagnosis. Furthermore, the integration of the Gemma 3B Large Language Model powers an intelligent chatbot and assists in generating textual diagnostic reports. This outcome provides therapists with immediate, AI-driven insights that support their clinical judgment.
The project delivered a working hardware prototype for real-time physiological monitoring. By engineering a circuit involving a Galvanic Skin Response (GSR) sensor, an Arduino Nano, and an ESP32 microcontroller, the team successfully created a live data stream. This system captures the patient's skin conductance levels and transmits them wirelessly to the mobile application via a Python server. This allows therapists to observe objective stress markers in real-time alongside cognitive assessments.
The final system achieved a unified reporting and history management capability. The application automatically synthesizes data from the digital tests, AI interpretations, and physiological sensor readings into detailed, exportable reports. This feature streamlines the documentation process, allowing therapists to easily save, retrieve, and compare patient history over time, thereby reducing administrative overhead and ensuring data consistency.
