Exploring the Temporal Lag
Between Gaze & Finger as a
Reading Difficulty Biomarker

In this project, eye-tracking data is captured using specialized equipment that records where the reader's gaze lands on each word. The key metrics extracted per word-token are:

• FFD — First Fixation Duration: how long the eye first rests on the word (ms)
• FPD — First Pass Duration: total time spent on the word during the first reading pass (ms)
• TRT — Total Reading Time: the cumulative time the eye spends on a word across all visits (ms). Used in the target variable.
• RPD — Regression Path Duration: time before the eye exits the word to the right, including re-reading (ms)
• fixNum — number of fixations landing on the word
• isReg — whether a regression (backward eye movement) started after this word

These metrics are collected from the Greek ΤΑΧΙΤΑΡΙ reading corpus, with 48 participants reading 4 texts in both aloud and silent modes.

Finger tracking captures the kinetic behavior of readers who use their finger to guide attention along the text. In the ΤΑΧΙΤΑΡΙ dataset, FT sessions are recorded separately from ET sessions for the same participants and texts. The key metrics are:

• coverage — the percentage (0.0–1.0) of the word-token that was covered by the finger during reading. This is the core FT signal used to compute lag_proxy.
• dt — finger-tracking time (ms): the duration the finger spent on each word position.

A critical challenge of this project is that ET and FT data cannot be directly paired per-word by timestamp — they come from separate sessions. This is why we engineered lag_proxy as an indirect temporal lag measure.

The temporal lag is the core novel contribution of this thesis. Since the dataset lacks direct timestamp alignment between eye-tracking and finger-tracking sessions, we cannot compute a direct time difference. Instead, the engineered feature is:

• When lag_proxy is high → the eye spends relatively more time than the finger covers → higher cognitive difficulty
• When lag_proxy is low/negative → the finger tracks comfortably alongside the eye → easier processing

Cross-correlation analysis is also performed to examine the broader temporal alignment patterns between the two modalities across all word positions.

Two machine learning models are trained on the same preprocessed dataset and compared:

• XGBoost — Gradient-boosted decision trees (300 estimators). Excels on structured/tabular data, trains fast, and offers native feature importance. SHAP TreeExplainer is used to quantify each feature's contribution.
• MLP — Multi-Layer Perceptron neural network (128→64→32 neurons). Acts as the baseline deep learning model for comparison. SMOTE oversampling is applied to handle class imbalance.

The binary target variable is defined as: TRT > mean AND isReg = 1, indicating words that required both extended reading time and triggered regressions — a strong signal of comprehension difficulty. Models are evaluated using Accuracy, F1-score, classification reports, and confusion matrices.