A verification-aware framework for representing, diagnosing, and repairing reusable web-agent skills.
Long-horizon web agents frequently fail without knowing where or why execution broke down. This issue is particularly pronounced in skill-based agentic web systems, where failures arise within composite skills whose internal decision processes are not directly traceable, making precise diagnosis and repair especially difficult over long horizons. We introduce SkillTracer, a framework that represents skills as attributed plan graphs structured by hierarchical nodes and verifiable edge transitions, enabling programmatic verification of execution progress.
By decomposing skills into inspectable hierarchies, SkillTracer converts raw interaction traces into structural evidence, making execution breakdowns localizable to specific node-level decision points and attributable to failing components. This attribution signal facilitates targeted structural repair, allowing the agent to selectively revise failing components while preserving the integrity of valid substructures for partial reuse and adaptive recovery. Furthermore, SkillTracer synthesizes short-term traces with long-term historical evidence to construct a persistent skill graph, enabling failure patterns to drive continual refinement across episodes.
Evaluated on challenging long-horizon benchmarks, SkillTracer achieves a 17.7% average improvement in success rate over strong baselines, with gains of up to 56.3% in cross-domain settings, demonstrating that structural attribution and skill repair are critical for reliable long-horizon web interaction.
SkillTracer converts reusable web-agent skills into structured, verifiable plan graphs. Instead of treating a skill as an opaque action sequence, it exposes the internal decision structure, localizes execution failures to concrete graph components, and performs targeted repair while preserving valid progress.
Skills are represented as hierarchical plan graphs with explicit node attributes, executable actions, and state-dependent transition conditions.
Execution failures are traced to specific intents, components, actions, or semantic descriptions using verification evidence from page observations.
SkillTracer repairs only the erroneous structure and resumes execution from the preserved valid prefix, avoiding unnecessary global re-planning.
This case study shows an address-editing skill induced from successful form-filling trajectories. The skill is represented as a six-node plan graph, where each node specifies its intent, target component type, executable action, and semantic description.
The transition from postal-code filling to state selection is guarded by a verifiable condition:
the page should expose a combobox component. If the target website instead implements
this field as a textbox, SkillTracer localizes the mismatch to the state-selection node
rather than discarding the entire skill.
The valid prefix is preserved, the failed node is repaired, and execution resumes after re-verification. This illustrates how SkillTracer enables partial skill reuse and robust cross-site generalization.
We evaluate SkillTracer on challenging long-horizon web tasks based on the WebArena benchmark. Our experiments cover multiple domains with diverse interaction patterns, including shopping, administration, discussion forums, software repositories, map-based search, and cross-domain tasks. We compare SkillTracer against strong skill-based and prompting-based baselines, and examine whether structural failure attribution and localized repair improve robustness in long-horizon execution.
The results show that SkillTracer consistently improves task success across challenging web domains. Compared with strong baselines, SkillTracer achieves a 17.7% average improvement in success rate, with gains of up to 56.3% in cross-domain settings. These improvements indicate that representing skills as attributed plan graphs, together with node-level failure attribution and localized repair, substantially enhances execution stability, adaptive recovery, and cross-domain generalization in long-horizon web interaction.
@inproceedings{li2026skilltracer,
title = {SkillTracer: Structural Failure Attribution and Refinement of Agentic Skills in Long-Horizon Web Tasks},
author = {Li, Yuyang and Dou, Yiran and Shao, Jie-Jing and Lyu, Yueming and Tsang, Ivor and Yin, Haiyan},
booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining},
year = {2026}
}
Paper, supplementary materials, and implementation updates will be added to this page.