Automate Routine Desktop Work with Gemini's Computer Use Capability
Automate repetitive UI tasks using Gemini 3.5 Flash's computer interaction.
Use the 'computer use' capability in Gemini 3.5 Flash to delegate mundane navigational tasks and data entry across different applications.
The Scenario
You have a repetitive administrative task involving transferring data between two legacy web applications that don't have an API.
Before & after
A worker manually navigates through multiple browser tabs, copy-pasting customer names and addresses into a spreadsheet, taking roughly 45–60 minutes for a large batch.
You can describe the task using the Gemini 3.5 Flash 'computer use' capability, allowing the AI to navigate interfaces and input data for you in 2–3 minutes.
The Prompt
I need to extract data from [WEBSITE_OR_APPLICATION] and move it to [DESTINATION_FILE]. Please look at my screen, identify the fields for [DATA_POINTS], and automate the transfer for the next [NUMBER] entries.
This feature leverages Gemini 3.5 Flash's ability to interact with computer interfaces directly, automating repetitive UI-based tasks that previously required manual clicks.
Source
Official Gemini news and updates | Google Blog"Introducing computer use in Gemini 3.5 Flash... allow the AI to navigate interfaces and input data for you."