All in a day's work

It all started as a typical workday for Chris Vickery. The director of cyber risk research for UpGuard was performing a typical port scan, which is a computer process similar to knocking on doors to see who is home. It's a routine he does nearly every day in an attempt to see which computers are susceptible to break-ins from hackers.

UpGuard's mission as a cybersecurity startup is to raise awareness of data leaks. Part of that effort included hiring Vickery, who has made news in the last few years for finding a lot of sensitive information available via the Internet. He has found account details for 13 million users of Apple's MacKeeper online, information on nearly every U.S. voter left accessible online by a Republican consultancy and evidence that a New York airport had left highly sensitive files unsecured online for nearly a year.

On the morning of July 1, Vickery was scanning random Internet Protocol addresses through Port 873 when he noticed a pair of hard drives that were accepting connections from the public Internet.

Port numbers are kind of like addresses for those house doors Vickery was knocking on. Most Internet traffic goes through Port 80 or Port 443, which are used for traffic to and from http and https addresses. Port 873 is set aside for an open-source remote file synchronization tool known as Rsync, which is typically used to back up files. Port 873 can be restricted to trusted computers and users by putting up a simple door.

But what Vickery found that day was a pair of backup hard drives that were completely exposed to public Internet traffic. The door was wide open.

Security professionals lament that industries tend to underappreciate the risks they face until a major hack or breach jolts them awake. The auto industry's wake-up call for connected-vehicle cybersecurity came three years ago, when security researchers Chris Valasek and Charlie Miller publicized a vulnerability that allowed them to hack a Jeep Cherokee.

Many security experts try to wake up industries by finding vulnerabilities and making the companies aware of the data breaches.

Finding an open door like the one Vickery discovered that July morning is not particularly unusual. Vickery says his routine scans turn up unprotected data about once a week, and UpGuard's automated systems find thousands of smaller data breaches every day.

But as soon as he downloaded the data, he could tell there might be sensitive information. One of the directories was named "Client Files."

"That's always a juicy directory," he said.

Even juicier: Inside that directory was a folder named "Tesla." That's when Vickery knew this was likely an important data breach.

"I have the general impression that data related to Tesla is generally pretty heavily protected, and they seek to enforce that protection," Vickery said.

Vickery quickly Googled the name of the company the hard drives belonged to, Level One. Once he saw the Windsor, Ontario, company provided automation and robotics to the auto industry, he realized he had just found an extremely sensitive set of data. The files exposed the entire relationship between the robotics company and its secretive electric automaker client.

"There were nondisclosure agreements, pictures of Tesla's manufacturing floor, computer-aided drafting schematics of their factories," he said. "It's not 100 per cent of the files needed to build a Tesla factory, but there was a surprising amount there. It struck me as generally something that a Tesla attorney would have a heart attack about if they knew it was available to the open Internet."

The data wasn't just about Tesla. General Motors, Ford, Fiat Chrysler, Volkswagen and Toyota all had projects with Level One, and their sensitive data was revealed in the breach. Information that could have helped malicious hackers wreak further havoc was exposed, including bank account details, virtual private network access request forms and ID badge request forms, as well as personal details about Level One employees, such as scans of licenses and passports.

All told, 157 gigabytes of sensitive data were exposed. For perspective, an hour of streaming video uses about 1 GB of data. So the data breach involved about six and a half days of nonstop TV show binging.

Because of the amount of data and the level of detail it described, Vickery was quickly able to determine that this was not a "false flag" intended to fool him or make Level One look bad. So he moved to the next step: sending the CEO of Level One, Milan Gasko, a standard email informing him of the breach that had been found.

It took a few days to pull the email together. And then, nothing. No one responded.

Vickery said it's not unusual for his initial outreach to go unheeded because companies often think he is a scammer preying on a company's worst fears.

Several days later, still with no response, Vickery called Level One and spoke to a receptionist who told him that the CEO didn't regularly monitor the email address he had tried. She took a message and said she would bring the matter to his attention the next morning.

"I got a call back about 45 minutes later," Vickery said. As soon as he was on the phone with the company, it was clear that it took the situation seriously.

Vickery described the situation and told Level One what kind of device he thought was being accessed, based on how many bits of metadata were exposed.

The company found the storage devices while Vickery was on the line and fixed the problem in an instant with one simple move: It unplugged them.

Sure enough, Vickery scanned the ports again, and they were gone. The breach had been secured, at least temporarily.

Asked whether most breaches are closed that easily, Vickery laughed.

"It's not usually the case that they are able to just unplug the device," he said. "Usually it takes a little bit more than that."

Since there were no signs of criminal activity, or evidence that the exposed data had been seen by anyone besides Vickery, there wasn't much left to do but write UpGuard's official report.

It's still not completely clear to Vickery how these storage devices were left open to public traffic, let alone publicly writable, and Level One isn't saying. The company declined to comment for this report.

"Level One has been very humble and nice," he said, "but they have engaged outside counsel, and I am signing a declaration that I have purged all the information that I had."

Vickery says most data breaches are caused by straightforward mistakes, such as typos in the coding or plugging a database into the wrong port in a server room. Especially as manufacturing companies reinvent themselves for the hyperconnected, data-driven paradigm, he says, there will always be some risk of an inadvertent error leaving sensitive data exposed.

In the manufacturing world, suppliers are often trying to compete with one another on price, and many companies see security as an additional expense, Vickery said. But the axiom "If it ain't broke, don't fix it" doesn't work in the connected world, he said.

"Well, a lot of times, the fundamental security problem is that your eyes are shut, so you don't realize that something is broken," Vickery said. "You never know the bad guys are getting in."

If a company can't afford to hire a company such as UpGuard, Vickery said there is an easy way to monitor database security.

"Just send one of your IT people home early one afternoon with a list of your IP addresses, and have them try to connect to your systems from home without any privileged access," Vickery said. "If every company did that just once a month, I probably wouldn't have a job."