Global Airline Outage – One Vendor Chaos


Vendor and supplier management is amongst the most boring topics to explain to your spouse, in-laws, and friends when they ask “What exactly is it that you DO for a living?”

It is always great to have a real life example to help. Today’s apparent failure causing chaos nationwide is PERFECT. Many birds, one stone.

A company, Amadeus, has successfully innovated and created a platform for airline operations that many airlines have adopted. Not surprising actually since user friendly and airlines have rarely been expressed in the same sentence.

Apparently this morning there was a problem. A large one. One that affected most/all airlines using the system. Reservations, kiosks, gate agent operations, et al.

In June 2016 I snapped a photo of Amadeus in action at AUS. It was in stark contrast to the text screens that agents used in decades past. Note the ability for multiple airlines to share one gate. Awesome. If it works. ALL the time.


  • Disaster Recovery? – Amadeus? Airlines? Why are passengers affected when a single vendor fails?
  • Regulators – Is it acceptable for a single vendor to be allowed to substantively disrupt airport operations globally?
  • Attackers – Want to inflict max chaos and disruption to critical infrastructure? A single vendor may all you need to target.
  • Protectors – Do you trust your vendors? Based on what evidence?

Forecast: Mostly Cloudy @ISC West 2017

I’ll be speaking on Cloud Security at ISC West on April 6, 2016.

Mostly Cloudy with a Chance of Security

ISC West bills itself as the “largest security industry trade show in the U.S.”

I attended last year and blogged about my experience.

This year I’ll be wearing two hats and advocating on behalf of both enterprises and cloud service providers with a goal to improve the security of both.



Fiber Cut? No Internet, Mobile or SMS? Are You SOL?

I woke up this morning thinking about the increasing impact that any disruption between us and the Internet has in our daily personal and professional lives. I love the security profession but sometimes it drives your brain to an offroad or two (or more) that many people do not oft travel.

Think. What use is your smartphone, computer or tablet if it loses all ability to communicate with anyone or anything else?

When fiber is cut and Internet and phone service are down it often affects an entire community or region. If this disruption occurs inline with a disaster, either natural or terror, lives can be at risk.

Fortunately most fiber cuts to date are accidental the result of an errant backhoe or other less than nefarious cause.

This is changing. Intentional cuts in the Bay Area in 2015 and allegations from Verizon that striking workers intentionally cut fiber are troublesome indicators that our fiber optic networks will increasingly be a target for those with a desire to disrupt.

But what is old is new again. The first recorded telecommunication sabotage took place during the second battle of Bull Run in 1862.  I would argue the stakes are no higher today just different.


The business impact of a fiber cut can be measured quantitatively and qualitatively. I dare you to go brick and mortar shopping in a region experiencing a fiber cut. You will quickly learn which retailers have the most resilient and effective disaster recovery and business continuity efforts.

Most retailers rely on fiber for every connection they make at the point of sale or otherwise. The fallback is normally satellite and works much more slowly, if it works at all. And you thought the lady ahead of you in line at Wal-Mart buying 500 cat key chains insisting on 50 separate receipts was slow.

The inability for teenagers to reach their friends via Facebook, Snapchat, SMS, phone or any other means other than face to face may seem in the moment catastrophic but is in reality only a symptom pointing to a future where the stakes are much higher.

Today Alexa’s inability to respond from the mothership to turn off lights and tell dad jokes arguably worse than my own is but a minor annoyance.

Future Alexa controlling my in home medical devices, fire suppression systems, and life safety equipment sets the stage for a future where being always connected is as critical as having water, power and oxygen to breathe.


Ensure you have multiple Internet connections over disparate paths. Businesses in mission critical industries do this as a normal course of business. I recommend small businesses and families do as well.

My small business maintains two Internet connections and a satellite backup. Keep in mind fiber cuts often render all land based communication useless. Maintaining satellite Internet as a backup is a relatively cheap insurance policy. We use Exede.

Invest in a SOHO router that manages multiple Internet connections and provides for automatic failover/failback. My preference is CradlePoint.

Invest in an out-of-band communication technology to ensure that fiber cuts or other outages do not prevent you from reaching your family or business associates.

Not fully baked but amongst the most promising and exciting innovation for communications not reliant on Internet or even mobile coverage are these two companies. Beartooth  and GoTenna.

Both systems utilize a combination of your smartphone and a built in 900 mhz unlicensed radio frequencies to allow communication over several miles with no dependency other than a similar device on the other end.

Although GoTenna appears more consumer friendly and geared towards the social, crowdsourced model they do purport to have a mission critical “professional” line in the works. I’ve ordered a pair of GoTenna devices and will be posting a review after some time assessing their merits and limitations.

Technology solutions aside the most important action you can take as a business, family or individual is to have a plan and TEST the plan regularly.  Many great resources to assist with this over at

As always feel free to reach out to me directly via LinkedIn if you would like more information about this topic or any other.

Stuart Clark

AWS Security Is Better Than Yours

Amazon will not tell you this but they think AWS security and compliance practices are way better than yours. And they would be spot on.

AWS re:Invent 2012 Leading with Security

How would I know? I’ve been working with service providers running on AWS since 2010. I helped a fintech startup born on AWS infrastructure win deals with mammoth financial services firms to proxy traffic between AWS and their internal networks back in the stone ages of the cloud, 2012. In that same year we completed a SOC 2 Type 2 Audit, one of the first cloud service providers running on AWS to do so.

Alas this is not a story about me. This is a story about continuous improvement about what can be accomplished over time and at scale. This is a story about a company that understood very early the importance of security, invested appropriately and now stands to reap the rewards of a tipping point tipped as a deluge of cloud migrations and associated revenue fills their coffers.

My perspective is simply as an AWS customer and partner that cares about security and has chosen to go deep to better understand what it takes to create a company that used infrastructure to change the world. I have no magic beans only my compounded experience of many years in the cloud to guide me.

But we warned. If you drink the Kool-Aid and decide to host infrastructure on AWS do NOT think you are off the hook for your own security and compliance efforts. I have a special dark place in my heart for organizations that HIDE behind the security and compliance of their cloud provider.

Security logos copied and pasted from a cloud provider to a marketing website are a poor defense against poorly secured applications and data.

What evidence do I have to make my claim that AWS security is better than yours?

Attestations, Standards and Frameworks Galore

AWS does not have the luxury of serving one particular industry or vertical. They provide infrastructure services to everyone from startups working on the latest useless social media app to three-letter government agencies which may or may not be spying on us. This is an incredibly high bar and requires a massive investment in security and compliance.

Yes compliance is not equal to security… blah blah blah… If you actually do your diligence and READ these reports you get a sense as to the true investment in security that goes way beyond check boxes.


You can gain insight as much by what does not happen as what does. Anyone that tells me their availability is as good as AWS (or Azure) for that matter gets my respect. And my skepticism. I immediately wonder if they are trying to fool me, themselves or both.

In late 2016 do we not have better things to do with our time than hug iron and troubleshoot blinky lights? Do you really think you can build resiliency to remotely equal the capabilities of multiple geographically disparate AWS Regions available at the push of a button?

DDoS mitigation anyone? Short of being an infrastructure provider why would you want to hassle with having to manage fighting this beast?  Amazon has your back.  Relax. Sort of. Obviously any applications or infrastructure you manage must be architected in such a way as to be resilient against DDoS attacks.  DDoS Best Practices Guide here.

Tools and Extensive Partner Network

It just makes sense that the longest tenured cloud company would have the most robust partner solutions and tools. AWS CloudTrail, Trusted Advisor, IAM, Inspector, WAF, HSM, KMS, Directory Service, etc.  More here.

Layer on partners that have offered cloud security solutions for many years. Companies like; Alertlogic, Sumologic, OneLogin, Ping, CloudPassage, Cavirin, and more.

The End

A key takeaway is that it took AWS years to get to this position. In 2011 AWS compliance efforts were nascent in comparison. Most of the tools and partners mentioned above either did not exist or if they did functionality was “limited”. (read sucked)

The significance of AWS’ strong leadership position in security cannot be understated. A public cloud provider is now the security reference from which all others can aspire. We have come a long way baby.

I’m excited to be speaking at Innotech Austin on Thursday November 16, 2106. Innotech Austin is the region’s largest business technology conference.  I will be speaking about the role of cloud in digital evidence and the challenges law enforcement face when implementing new technologies.


Date: November 17, 2016
Time: 03:20 pm
Event: Innotech Austin 2016
Topic: Evidence Management and the Role of the Cloud
Venue: Austin Convention Center
Location: Austin, Texas 78701
Public: Public

Beginning of the EndPoint – Challengers


I’ve been a Cloud CISO for a little more than 5 years now. One consequence is that enterprise endpoint security products and I have rarely passed paths. Agile orgs running Linux / OSX with users perpetually outside the perimeter is not easily solved for with legacy endpoint products.

But I was curious what has changed…

This afternoon I read with interest the “Forrester Wave Endpoint Security Suites Q4, 2016” report.

The report could have been written ten years ago, with the notable exceptions:

  1. Companies like Carbon Black, Cylance, CrowdStrike and Bromium have emerged to challenge perennial industry giants. Any innovation in endpoint security is noteworthy. No longer is it acceptable for the incumbents to ride the cash cow of enterprise renewals without significant development efforts to keep pace.
  2. Quarantine = Remediation

You can find the Forrester report over on the Carbon Black website.  (Gated, Sigh…)


Modern Evidence Management: Challenges and Solutions

AUSTIN TX — As I write this in October of 2016, a constant of American life is the inescapable media coverage of critical incidents involving law enforcement. The Media, in a free and open society, plays a critical role reporting and providing to the public a degree of transparency about how our government is policing us. We should be cautioned, however, to form our own individual opinions when digesting these events and to avoid being led blindly by a media narrative woven with information that is often, at best, incomplete, and at worst, completely wrong. Its easy to forget that even the best media coverage lacks the context that comes from having all of the information available to investigators.

The most important part of that information is, of course, the evidence. Evidence is and always has been the impartial witness that enables the facts to be known and justice to be served. Evidence collection and processing has evolved over several hundred years of policing into a mature discipline. A critical component of this discipline is “chain of custody”, a process that seeks to ensure the integrity of the evidence from the time of collection to final disposition of the case. Until recently, the traditional processes and technology used for evidence management had been sufficient. Unfortunately that is no longer the case. A disruptive force threatens even the most mature evidentiary processes. This force, in a word: data.

Every day we create 2.5 quintillion bytes of data, and a staggering 90% of the world’s data was created in the last two years. While the digital evidence associated with critical law enforcement incidents represents a tiny percentage of that data, it is undoubtedly one of the most important parts of it. While no data points exist to speak to the amount of digital evidence being created daily, empirical observations are telling. In addition to the video evidence generated by Body Worn Cameras (BWC’s) used by police officers, other sources such as video surveillance systems and the proliferation of video recording by the public are all contributing to an ever growing mountain of digital evidence that must be managed. This huge trove of data combined with public scrutiny of critical incidents is exerting new pressures on both law enforcement administrators and the technology they use to manage it.

Evidence management systems have evolved significantly from the paper systems of the 1960’s and the mainframes of the 70’s that were only tasked with tracking physical evidence. The personal computer revolution at the end of the 80’s finally enabled digital evidence management, even if only in the most rudimentary fashion. Today, the rise of cloud has enabled the creation of a new class of cloud-enabled Digital Evidence Management Software (DEMS) products, purpose built to manage the enormous amounts of data we must maintain with integrity. While its admittedly not a panacea, in today’s world, cloud based DEMS may represent the best method we have for ensuring that all evidence eventually facilitates justice.


This article is a repost of an article originally guest authored for the DoubleHorn blog.

Early Adopter Pain and Hyperconverged Excellence

Austin’s @pivot3 recent tweet of an @ITWire article about the projected growth in the HCIS (Hyperconverged Integrated Systems) market contain a couple of nuggets of information that demand expanding upon.  

First:  Hyperconvergence has yet to enter the mainstream.  

This is true.  The article states that while HCIS is the fastest growing segment it will be less than 1/4 of the integrated systems market by 2019. 

Innovators and early adopters are clearly benefiting from the fruits of hyperconvergence, namely software centric infrastructure built upon commodity hardware to deliver transparent solutions to internal and external stakeholders quickly and efficiently. That said early adopters are also on point to absorb the inevitable pain of industry upstarts and established giants finding and exploiting a particular market niche and redefining their solutions on a path to revenue nirvana. 

Pivots, acquisitions and chaos lay ahead for some. As evidence some friction has arisen between Dell and EMC as they struggle to understand how Nutanix fits into their new world order. 

Secondly:  Hyperconvergence use cases have been limited and silos have emerged.  

No kidding.  If you treat hyperconvergence like a traditional enterprise point solution, roll with a limited POC, and get frustrated when the touted benefits are not realized, then you deserve your fate.

Hyperconverged excellence requires a philosophical shift to a software first/software only mentality from the help desk to the CIO.  Hyperconverged excellence requires that you are willing to shrewdly assess your organization and act to make the necessary changes to be successful.  Blockers must be eliminated be they technological, cultural or personnel. Hyperconvergence holds great reward only for those organizations that understand the revolution at hand and have the courage to execute.  

The irony is that those that languish in perpetual POC hell will eventually abandon their infrastructure entirely for a public/private cloud solution managed by someone that achieved hyperconverged excellence long ago.

ISC West 2016 Conference Wrap – Connected Death

Last week I had the opportunity to attend ISC West billed at the “largest security industry trade show in the U.S.” Held annually at the Sands Expo in Las Vegas the show features over 1000 exhibitors and is attended by over 28000 security professionals.

IoT Vendors at Connected Security 2016

IoT Vendors at Connected Security 2016

Make no mistake ISC West is largely a physical security conference.  The sheer number of cameras, access control solutions, retractable electronic bollards, electrified fencing and oddly skin care vendors was staggering.

For the first time the show included the Connected Security Expo a cybersecurity conference within the larger conference.  A recognition that many of the aforementioned vendors now manufacture internet connected devices that need to be secured (with the probable exception of the skin care vendors).

I was pleasantly surprised at both the execution and content of the expo.  A successful conference for me is largely defined by what I learned.  My thoughts about the conference based on my multiple personalities, err personas…

As a Technologist – Very Cool. As a corporation you can actually buy a drone fleet and patrol your perimeter, then deploy a robot to investigate and intercept violators.  All remotely controlled and monitored.

As a CISO – Abject Fear.  Very few of these IoT device manufacturers appear to have any expertise in cybersecurity.  I’ve probably spent more time thinking about their supply chain and secure software practices than they have.

As a Security Entrepreneur  – Unlimited business opportunities abound.  See CISO thoughts above.

We can expect 200 billion new devices to come online by 2020 according to Matthew Rosenquist of Intel.  Many of these devices will impact life and safety.  Vehicle control systems and medical devices are examples.  The bad news is that a significant amount of blood will be spilled in the next few years.  Innovation has always outpaced cybersecurity but now the consequences of failure include the likelihood that people will die.

Are we ready for this connected future?  We had better be.  There is no other choice.