well of course it does. There is no way for it to know what it is capturing. Best it can do is capture it, and maybe discard it if it manages to detect any sensitive info. Which won’t work every time
Demanding perfection for a system as dangerous as recall is not silly.
It’s like keeping an armed nuclear bomb in the center of a city at all times and being like “hey, it’s ok that it’s activation sequence isn’t perfect, it probably won’t go off”.
The solution to make it perfect is to not install the nuke/recall at all.
It’s like keeping an armed nuclear bomb in the center of a city at all times and being like “hey, it’s ok that it’s activation sequence isn’t perfect, it probably won’t go off”.
Obligatory mention that for 20 years the launch code for nukes in the US was 00000000.
Because malware is already using such methods to steal credentials, so by having something “legitimate” work towards preventing such situations, a countermeasure will eventually be born.
Right now, all kind of applications take screenshots and send data without user’s knowledge. If something like blurring can trigger automatically and modify what is being sent, then the user will have some protection available instead of none.
Depends. Can anyone with the know-how create a custom way of taking a screenshot/capture independent of all others or do all methods have to use an immutable function as the base?
If the former, i agree with you. If the latter, you’re kinda wrong to believe so.
Let’s take a step back. If we allow Microsoft Recall to take snapshots of everything, now there are two places to protect your information, and one has historical information.
Why would you want that? Even if we somehow prevent malicious software from taking screenshots, we now have to worry about malicious software breaking Recall or any servers that have Recall info. That’s a much bigger attack surface, especially if there’s a server involved.
Many problems in the past were solved by inventions that were meant for other things.
Seeing something for what it is and ignoring its untapped potential is a narrow view of life.
This will 100% be hacked and steal peoples identity. The first version was an unencrypted database that reaseaechers had to note for MS to change it. This is how poorly this feature is being developed.
It will also be a parental lazy tool for spying on your kids rather than teaching them good habits and achieving autonomy. Same with employers.
The only thing this will serve is MS. They are now selling their own MS windows $400 thin-clients that have no onboard storage, everything is cloud access. This will be MS way of giving you access to things you would probably save local, except now it is in the cloud for government or bad actor theft. There was just a giant data breach stealing meta data like this from government officials.
We don’t need AI for AI sake. Put that processing power into AI protein folding for drug and gene research.
Programs can already be refused from being able to view screen contents. If malware is able to circumvent this, why do you believe it would abide by the filtering rules? Further, if you really do believe this is useful, Microsoft could implement this technology without also randomly screenshotting your computer.
If a malware bypasses a function directly, then closing that loophole would force future versions to find more complicated ways of achieving the same thing, which makes them more visible in the long run.
Edit: Also, Microsoft sucks. But now that the ugly crap is out there, you’ll come to face it eventually. Why not be more prepared?
What are you even talking about? Where have the goal posts gone? You wanna know what else sounds like malware? A program randomly and persistently taking screenshots of my computer and sending it to someone.
This isn’t even a matter of perfection, this is Recall barely managing to censor the most blatantly sensitive information (see: the article saying “I also created my own HTML page with a web form that said, explicitly, “enter your credit card number below.” The form had fields for Credit card type, number, CVC and expiration date.”)
Demanding a system protect user data is not silly, it is necessary. And if a given system can’t do that, then it should never be used. Especially considering the fact this is likely going to make its way onto PCs handling extra sensitive data with strict privacy requirements, such as medical data protected by HIPAA.
Context can be trained.
Maybe Microsoft shouldn’t have released a tool until it had that context then?
If a company releases a half-baked tool that doesn’t do what it advertises, easily fails in simple attempts at identifying sensitive data, and is almost impossible to guarantee data security with, then it should never be used or advertised for any context in which any sensitive data could ever be present.
no, it cannot. It implies you having samples of every form possible so the llm can interpolate. And even then, something sensitive to me might be harmless to you. The llm cannot know your intent.
I doubt that OCR (optical character recognition) is done on device so it likely being sent to some server for processing.
As a software engineer, in any of our corporate applications when a user hits delete we toggle an archived flag, but the data is still there. So I wouldn’t trust any application to do what it actually says.
There are so many technical barriers for recall to ever be able to not snipe your private data that I wouldn’t go anywhere near the thing.
Edit: Furthermore, what happens when MS inevitably gets hacked again and someone steals all the data it has and then starts using that to commit fraud.
I don’t understand your meaning. Screenshots of a photo are still screenshots and manipulating text on a photo is already a thing (you can use phone camera to translate text directly from a fixed surface).
well of course it does. There is no way for it to know what it is capturing. Best it can do is capture it, and maybe discard it if it manages to detect any sensitive info. Which won’t work every time
Technically, it could be coded to recognize the various formats of strings and blur everything indiscriminately.
In this case perfection is very easy. It could avoid capturing 100% of credit card info by not taking screenshots of everything.
Reminds me of my favourite quote:
“You miss 100% of the screenshots you don’t take.”
Demanding perfection for a system as dangerous as recall is not silly.
It’s like keeping an armed nuclear bomb in the center of a city at all times and being like “hey, it’s ok that it’s activation sequence isn’t perfect, it probably won’t go off”.
The solution to make it perfect is to not install the nuke/recall at all.
Obligatory mention that for 20 years the launch code for nukes in the US was 00000000.
Even nuclear technology isn’t perfect, yet people are pushing for it in spite of the dangers.
Is the solution to give it up completely?
Please.
You’ve successfully missed the point, congratulations.
But you never caught my initial point either.
If you agree that it will never be perfect at filtering out sensitive information, why support it?
Because malware is already using such methods to steal credentials, so by having something “legitimate” work towards preventing such situations, a countermeasure will eventually be born.
Right now, all kind of applications take screenshots and send data without user’s knowledge. If something like blurring can trigger automatically and modify what is being sent, then the user will have some protection available instead of none.
how will capturing screenshots prevent other software from capturing screenshots?
And we all know countermeasures don’t exist. They can be used to train the ais out of their own existence.
Depends. Can anyone with the know-how create a custom way of taking a screenshot/capture independent of all others or do all methods have to use an immutable function as the base?
If the former, i agree with you. If the latter, you’re kinda wrong to believe so.
Let’s take a step back. If we allow Microsoft Recall to take snapshots of everything, now there are two places to protect your information, and one has historical information.
Why would you want that? Even if we somehow prevent malicious software from taking screenshots, we now have to worry about malicious software breaking Recall or any servers that have Recall info. That’s a much bigger attack surface, especially if there’s a server involved.
This is just a terrible idea all around.
Recall is not anti-malware though, the mal-ware can still do its own data gleaning. This is just an AI feature solving a problem that nobody had.
Many problems in the past were solved by inventions that were meant for other things. Seeing something for what it is and ignoring its untapped potential is a narrow view of life.
This will 100% be hacked and steal peoples identity. The first version was an unencrypted database that reaseaechers had to note for MS to change it. This is how poorly this feature is being developed.
It will also be a parental lazy tool for spying on your kids rather than teaching them good habits and achieving autonomy. Same with employers.
The only thing this will serve is MS. They are now selling their own MS windows $400 thin-clients that have no onboard storage, everything is cloud access. This will be MS way of giving you access to things you would probably save local, except now it is in the cloud for government or bad actor theft. There was just a giant data breach stealing meta data like this from government officials.
We don’t need AI for AI sake. Put that processing power into AI protein folding for drug and gene research.
Programs can already be refused from being able to view screen contents. If malware is able to circumvent this, why do you believe it would abide by the filtering rules? Further, if you really do believe this is useful, Microsoft could implement this technology without also randomly screenshotting your computer.
If a malware bypasses a function directly, then closing that loophole would force future versions to find more complicated ways of achieving the same thing, which makes them more visible in the long run.
Edit: Also, Microsoft sucks. But now that the ugly crap is out there, you’ll come to face it eventually. Why not be more prepared?
What are you even talking about? Where have the goal posts gone? You wanna know what else sounds like malware? A program randomly and persistently taking screenshots of my computer and sending it to someone.
If a company releases a half-baked tool that doesn’t do what it advertises, easily fails in simple attempts at identifying sensitive data, and is almost impossible to guarantee data security with, then it should never be used or advertised for any context in which any sensitive data could ever be present.
Agreed.
no, it cannot. It implies you having samples of every form possible so the llm can interpolate. And even then, something sensitive to me might be harmless to you. The llm cannot know your intent.
that would require knowing the formats of strings. And it requires the text to be text.
What if you had a photo of a handwritten piece of sensitive information?
I doubt that OCR (optical character recognition) is done on device so it likely being sent to some server for processing.
As a software engineer, in any of our corporate applications when a user hits delete we toggle an archived flag, but the data is still there. So I wouldn’t trust any application to do what it actually says.
There are so many technical barriers for recall to ever be able to not snipe your private data that I wouldn’t go anywhere near the thing.
Edit: Furthermore, what happens when MS inevitably gets hacked again and someone steals all the data it has and then starts using that to commit fraud.
What many people don’t realize is that this is how some low level data stores work as well. Even regular ol’ file systems do this (basically).
deletes are difficult
For a site called boring sql I found that very interesting and informative. Thank you for sharing!
Yeah I was shocked when I first started and was questing this behaviour with the owner and lead engineer.
They assured me it complies with GDPR as after a period of time we will anonymise the data.
I don’t understand your meaning. Screenshots of a photo are still screenshots and manipulating text on a photo is already a thing (you can use phone camera to translate text directly from a fixed surface).
handwritten. ocr isn’t perfect, especially with handwritten content.
Good point.
Blurring isn’t destructive.
In that case, instead of blurring, let’s have it turn the device into an I.E.D.