Blog

Big Candy Is Watching You: Facial Recognition In Vending Machines Upsets University | Hackaday

Most people don’t think too much of vending machines. They’re just those hulking machines that lurk around on train stations, airports and in the bowels of school and office buildings, where you can exchange far too much money for a drink or a snack. What few people are aware of is just how these vending machines have changed over the decades, to the point where they’re now collecting any shred of information on who interacts with them, down to their age and gender.

How do we know this? We have a few enterprising students at the University of Waterloo to thank. After [SquidKid47] posted a troubling error message displayed by a campus M&M vending machine on Reddit, [River Stanley] decided to investigate the situation. The resulting article was published in the February 16th edition of the university’s digital newspaper, mathNEWS. chocolate machine

Big Candy Is Watching You: Facial Recognition In Vending Machines Upsets University | Hackaday

In a bout of what the publication refers to as “Actual Journalism”, [Stanley] found that the machine in question was produced by Invenda, who in their brochure (PDF) excitedly note the many ways in which statistics like age, gender, foot traffic, session time and product demographics can be collected. This data, which includes the feed from an always-on camera, is then processed and ‘anonymized statistics’ are sent to central servers for perusal by the vending machine owner.

The good news is that this probably doesn’t mean that facial recognition and similar personalized information is stored (or sent to the big vaporous mainframe) as this would violate the GDPR  and similar data privacy laws, but there is precedence of information kiosks at a mall operator taking more liberties. Although the University of Waterloo has said that these particular vending machines will be removed, there’s something uncomfortable about knowing that those previously benign vending machines are now increasingly more like the telescreens in Orwell’s Nineteen Eighty-Four. Perhaps we’re already at the point in this timeline were it’s best to assume that even vending machines are always watching and listening, to learn our most intimate snacking and drinking habits.

Thanks to [Albert Hall] for the tip.

Could lobotomize the camera with a drill bit.

Apparently they used sticky post it notes. Message to machine: “Please stop taking photos of us humans or we will unplug you from the main.”

Similar methods work great for gas pumps which play commercials at you while you’re paying through the nose for gas

Don’t be monetized, buy an EV and charge at home with solar.

Tell me where to get an EV that can have it’s battery replaced for less than $50,000.

Tesla’s response: “Clean energy charging can be enabled with a monthly payment of only $59.99 per month!”

Protip: The physical button one down from the top right is *usually* a mute button.

Telescreens in a private space. Vending machines usually aren’t.

A few years ago there was a case of store franchise in Poland called “Stokrotka” (polish for “Daisy”) used their cash register computers with hidden cameras to collect data on customers. They used face recognition to match age and gender of the clients with the stuff they bought. Company claimed they collected anonymous data for preference tracking and stock adjustments for each store. This was discovered accidentally when one of the customers saw the app running on the screen on the sales person side.

I’m pretty sure this is done by many other franchises, too. But they hide it better.

Many self service checkouts in UK supermarkets now have cameras, so it’s now going to be easier for them to tie your store loyalty card to a picture. They already have a complete list of your purchases…

To the vending machine operator, who walks past and *doesn’t* use the machine would be of great interest. And if there’s already a camera…

As with unwanted cameras on laptops, it seems to me that if you know where the camera is a strategically applied bit of black tape would be an easy answer. Creep up on the machine from the side and it need never know who did it. And it isn’t really vandalism, as it can easily be undone.

I wonder how a tiny projector, or a screen at a reasonable/focusable distance might be employed to give the vending machine a more “interesting” view of its surroundings. “Our statistics show that primarly 180 cm tall, purple kangaroos buy M&M’s from university based vending machines”.

You could also use a bodged HMD lens with an off the shelf screen

I want to see this project on HackaDay next! Apply an instagram filter to every customer!

/s It’s alright, it’s GDPR compliant and running windows so it can’t possibly be misused or hacked.

Yeah what could possibly ho wrong?

Above where you pay there is a blue M&M, and just to the bottom left of that a tiny bit away is a pinhole, the camera is behind there. So it is perfectly placed to capture your image (for local processing, may as well have the local provider pay the power bill for the processing. They will add the additional power use to the price of the items being sold) when you pay.

Two solutions in this case:

(1) A small piece of electrical tape , or (2) contact M&M and tell them because of this you won’t be eating their garbage anymore.

Option 2 is really the best one. Collecting customer interaction data has far less value to them if they no longer have customers. Ask Bud Light how pissing off their consumer base worked out.

Unfortunately, too many people will cry about the invasive nature of vending machine cameras, only to calm their nerves later with a sugary treat from the same machine.

https://m.youtube.com/watch?v=CISFw4o0j4U&pp=ygUbQ2FydG1hbiBjcnlpbmcgZWF0aW5nIGNhbmR5

Option 2 doesn’t work because the people who care about this are statistically insignificant. Option 1 solves the problem for everyone.

Unfortunately you are totally right. I was just pointing out the location, because I’m sure that this is not the only hardware with cameras in them, best to have at least one example of where it is to know where to stick the tape.

Anyhow m&m (and their parent company “Mars, Incorporated”) have made it onto my never buy again list. Which is a shame, I really did like some of their products. But far better for my longterm health.

“Above where you pay there is a blue M&M…” but cut off in the HaD cover pic, you have to follow the link in the article.

I think the company just wanted to know who is rocking the machine trying to get their bag of chips to fall down.

My toothbrush listens to me poop

I don’t think you’re supposed to use it like that.

New policy for Invenda being written as we speak…

Going forward all filenames, any plain text messages, and any error reporting will be renamed/reworded to something cryptic and unreadable so as not to expose or give any hint about actual capabilities. All troubleshooting and repairs will be done by Invenda employees who have access to the translated information.

All technology providers will be making this their policy as well after learning their lesson from Invenda.

Well I’m sure I’ll get some dirty looks for mooning the vending machine but it’s the only way the company will learn.

It’s all downhill from here. They will reposition the cameras away from the machine and still get the metadata they need (esp with GDPR considerations precluding direct facial recognition). Add some Bluetooth and cell tower data and the height/weight/gender/hair color data you get from a camera is just cream on top.

Juggalo masks are unfortunately one of the best answers for camera driven metadata /facial recognition. More mainstream juggalos means less effective tracking, and maybe we can get a religious exemption from ‘you must show your face’ type laws if juggalos are prevelant enough.

My God what a world the 20’s is turning out to be when ICP is a hope for data security. Save us all, maybe a big solar flare?

We should be concerned about this sort of thing, but I have the nagging suspicion that a company could produce similar ‘market statistics’ by purchasing location and other data on collected by the mobile devices we walk around with. You don’t always need a face to deduce gender, approximate age, loiter times, etc. As for telescreens, we carry that mobile device with us in our public and private lives (government spooks excepted).

Please be kind and respectful to help make the comments section excellent. (Comment Policy)

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Big Candy Is Watching You: Facial Recognition In Vending Machines Upsets University | Hackaday

jelly candy machine By using our website and services, you expressly agree to the placement of our performance, functionality and advertising cookies. Learn more