Microsoft is kicking off the testing phase for the innovative Copilot Vision feature today and it was first revealed in October that this tool will allow the AI companion of Microsoft to visually engage with the content on an Edge webpage they are exploring.
The users can pose questions about the images, texts and other elements that they encounter or they can seek assistance as they navigate further. The Copilot team shared in a blog post that the users can select to activate the Copilot vision and it observes the page they are viewing as well as reading alongside them and they can also collaboratively tackle any challenges they face.
They further added that browsing transforms from a solitary task with just the users and their tabs into a shared experience. This feature is completely optional and it needs the users to explicitly allow Copilot Vision to access and read webpages within the Edge browser of Microsoft.
During the initial demonstration in October, Microsoft also showcased how the AI assistant could interpret images from a OneDrive collection and even red handwritten recipes. It provides helpful cooking suggestions and in addition to this, it can also assist the users while shopping online by providing product recommendations. Copilot Vision in the testing phase is only available to subscribers of Copilot Pro who are taking part in the Copilot Labs program of Microsoft.
Microsoft is moving ahead cautiously with this feature and it has also given the implications of allowing AI to read what users see on their screens and this has also raised a valid privacy concern.
The recent security issues are surrounding the new Recall feature and testing began testing last month which likely influenced the careful approach of Microsoft. The company intends to broaden its access to more Pro subscribers and websites in the future.
