(1) Abstract: 

This study develops an ophthalmic video foundation model, OVFM, for microscopic ophthalmic surgical step recognition and intraoperative navigation. Based on multi-center surgical videos, the model uses a self-supervised video transformer to learn spatiotemporal motion features and is compressed by staged knowledge distillation for deployment on a surgical microscope processing unit. In real surgical videos and wet-lab porcine eye experiments, OVFM supports step recognition, structural segmentation, and navigation cues, and is used to evaluate its effect on surgical performance.

(2) Background: 

Medical foundation models are mostly focused on preoperative diagnosis, while intraoperative applications are limited by scarce surgical video data, real-time deployment challenges, and insufficient integration with surgical workflows. Ophthalmic microsurgery requires high precision and involves highly dynamic video, making it suitable for intraoperative video foundation models. Existing surgical video datasets have limited scale and procedure coverage, and there is a conflict between model parameters and intraoperative computing resources.

(3) Main Content: 

The study integrates multi-center, multi-procedure ophthalmic microsurgical videos to build a large-scale pretraining dataset. A self-supervised video transformer is used to learn cross-frame spatiotemporal representations. Multiple downstream tasks are evaluated, including surgical step recognition, tool recognition, complication detection, skill assessment, surgical scene segmentation, limbus boundary segmentation, and nucleus block localization. A generic-to-task-specific knowledge distillation strategy is designed, and the model is integrated into a microscope-based navigation system.

(4) Study Design: 

A multi-center ophthalmic surgical video dataset was constructed for self-supervised pretraining. The model was fine-tuned on multiple downstream tasks and compared with baselines. Staged knowledge distillation was used to obtain a lightweight model. The distilled model was deployed on a surgical microscope to develop a navigation system for step recognition and structural segmentation. A porcine eye cataract surgery user study was conducted with a crossover design to compare surgical performance with and without navigation and to analyze differences across experience levels.

(5) Results: 

OVFM outperformed baselines in multiple surgical video recognition and segmentation tasks, showing effective spatiotemporal feature learning. The distilled model retained major task performance while reducing model size and supported real-time processing on the microscope. Cross-center evaluation showed that step recognition and limbus segmentation capabilities were largely retained. In the porcine eye user study, navigation cues improved incision- and capsulorhexis-related metrics, increased operational consistency, and narrowed the performance gap between experts and novices.

(6) Future Directions: 

Future work may expand posterior segment and additional procedure data, use higher-resolution full-length videos to preserve fine-grained temporal information, and strengthen cross-center and cross-device generalization. Attention should be paid to model transparency, decision responsibility, and human-machine collaboration boundaries. Combining video foundation models with augmented reality, robotic assistance, and multimodal sensing may extend surgical navigation to more complex procedures.

We provide full-process services for various material designs (including highly innovative scheme design, preparation, characterization, in vitro and in vivo validation (cell/animal experiments), etc.): a postdoctoral team (with multiple high-impact publications in Zone 1 journals) can meet all kinds of tricky and challenging requirements, including hydrogels / drug delivery / nanomaterials / polymers / liposomes / microneedle drug delivery, etc. We can assist with project design / NSFC grant applications / publication of high-impact Zone 1 and Zone 2 papers in journals such as AFM, SCI, AM, etc. Welcome to consult 👇️
17768014569 (Senior Engineer Gao)
17761925168 (Engineer Zhang)
18013729810 (Engineer Yang)