MemGUI-Agent improves both zero-shot 235B and trained 8B settings on long-horizon mobile GUI benchmarks. On MemGUI-Bench, MemGUI-Agent-235B reaches 62.5% Pass@3, and MemGUI-8B-SFT reaches 23.4% Pass@1 ...
OmniParser is a comprehensive method for parsing user interface screenshots into structured and easy-to-understand elements, which significantly enhances the ability of GPT-4V to generate actions that ...