Solving Ransomware attacks with Project Management

Ransomware attack leads to shutdown of major U.S. pipeline

That was the headline that caught my attention this morning.

As a Project Manager, certain thoughts  crossed my mind - using subconscious techniques like risk management, crisis control and borderline-use-cases:

1. All ransomware attackers want to be paid in cryptocurrency. 

The whole world is suffering  from a crackdown on grey & black-market currency. In Israel - for example - there's a limit as to the amount of cash you can pay when buying items. How come you have so much cash?

Every time I transfer money to one of my kids, I worry that some algorithm will flag it for some bored paper-pusher to investigate, and waste hours & days (and possibly lawyer fees) to prove my innocence. 

Yet, you can squirrel away your criminal earnings into crypto and voila! life is good. No way to trace what you do with it. No way to trace you are you.

Clearly, it's time to change the anonymous aspect of crypto. (I'm a firm believer that only people with things to hide refuse to divulge information that would help solve international identity issues. From biometric identity documents to cryptocurrency, and more. But that's not the point I'm dealing with here.)

I'm not sure how to go about it, and I sure hope that law-enforcement agencies are working on this...

2. It's not the mouse, it's the hole that steals.

A somewhat amusing Talmudic statement - shifting the blame from the mouse (or hacker, in our case) to the hole.

So, how do we fix the hole? That's the cat & mouse game (pun intended) that half the world is playing against the hackers. 

But, if one can mitigate the affects, to the point where one doesn't even consider paying the ransom, one has shrunk the hole tremendously.

Here are some ideas for mitigating the pain of a ransomware attack, based on being responsible, inter alia, for IT and DevOps for the past 9 years.

A. Backups

Duh. Don't you have anything to write about? Well, let's reword that to backups and fast restores. 

I refuse to believe that something as crucial as a U.S. gas pipeline supplying the entire East Coast does not have backups in place. But... if they cannot restore in a timely fashion, then they will have to work with a crippled system for a while.

This is unacceptable. The smallest Mom & Pop shop will suffer from a few days down-time. The more critical the computer system, the less it can afford down-time. So it's crucial that the backups are:

• Actually happening  - and not suffering from out-of-space or limited bandwidth issues, or other excuses.
• Up to date - you don't want to lose more than a few hours (or minutes) of work. 
• Fast restore - the closer to instant restore, the better.
• Ransomware-safe. If the backups are hacked, or you are backing up a system with the ransomware attack installed, you essentially don't have a useable backup.

Let's discuss these last 2 points in more detail.

Fast restore

As already mentioned, if you cannot restore in a timely fashion, you are going to be without the ability to function; from being unable to sell lollipops to having patients dying.

One technique I came across was to virtualize everything - then the machines can be backed up periodically by taking a snapshot. This technique is used on VMWare servers, for example, and is easy to automate, doesn't require downtime (you can backup while working) and the restore is almost instant.

I've also  used offline-continuous-backups like Carbonite; the restore can take a while, depending on your bandwidth. Your apps won't be restored - you'll have to reinstall them -  but your data will be almost up-to-date.

Ransomware safe

You have to ensure that you have a backup from before the system became infected with ransomware. Otherwise, the ransomware will attack again once everything is fully installed.

A rotation of 7-daily and a few weekly backups is one way to ensure you will have a clean restore point. Yes, it will take up a LOT of disk space; but, hey, disk space is almost free nowadays. Compare $800 for a 20TB disk to 1 BTC (1 Bitcoin is North of US$57,000 - yes, fifty-seven-thousand dollars).

That's 70 external disks of 20TB each.

Make sure you have on-site copies - for quick restores - but make sure they are off-line, or else ransomware will destroy them too. Yes, this requires frequent manual intervention, but is well worth it.

Do yourself a favour and keep off-site copies (like in AWS S3) for off-site security. (Good in case if fires, thefts or ransomware to note a few possibilities.)

Mirror, mirror on the wall

After many decades in Hi-Tech, few work-relate incidents stand out in my memory as starkly as those days that I got to work and my computer was dead.

Walk over to Tech Support (since you cannot email without a computer - and in the olden days we didn't have smartphones; they hadn't been invented yet) and after hours of walking around aimlessly you can start working again. 

Well, not really, you have to reconstruct your work environment, reinstall your favorite apps, map your favorite drives, etc.

All this changed about 9 years ago when I became responsible for IT. Who does IT go to when they need IT? Who cuts the barber's hair? Who does a dentist go to if he has a toothache?

What I did was to take an older machine and - one that was too old & slow for normal work - and install everything I needed on it. Every time I installed a new program I would install it on my mirror-machine too.

The rest of the time the mirror-machine was powered off. (This created an interesting question when we were audited for licenses.)

When the dreaded day arrived, I had to deal with reviving my laptop . But I could continue working - I just powered up my mirror-machine and everything was there, as I liked it.

This got me thinking. Why don't hospitals, government offices and everybody else  who has mission critical  computers have a mirror system in place?

 Then when the ransomware attack happens, or flood, or fire or theft, everybody uses their mirror-machines. Of course, you would need to be careful not to infect the mirrors - possibly having them on their own network - and maybe in a separate room/building.

The devil in the in the details, but I want to raise the awareness of these possibilities.

Mix & Match

Now imagine if you would virtualize all your machines and restore them frequently to mirror-machines.

Then in an emergency you would have an almost up-to-date useable system, with minor down time.

Eureka

Thinking though the details and what I should include to keep this a manageable size, I realized that I may have a possibility for a business.

Anybody want to join me? Creating a system implement and automate the mirror-backup system? Or maybe off-site locations to instantly relocate to in times of system failure?

I no longer want to start the day with headlines about impeding catastrophe because of ransomware.