GhostNomad.com

The Recap

A male was caught urinating behind a vehicle. He told police he could not find a restroom. Police advised against repeating the behavior.

The Lesson

I just couldn’t pass this story up after I read it while looking for a police blotter to comment on. The funny thing is the correlation hit me immediately. Despite knowing they shouldn’t do it, end users install things on their work computers that aren’t always approved. The problem usually stems from a need to complete a task for work that can not be easily achieved with the tools they have. We all know why this is bad. We have to protect our assets (phsycial and intellectual) not only from malicious software, but also software that causes unexpected behavior on the computers.

The question you need to ask becomes “Is the process I have in place to get items approved to complex”. Sure you will have those people who will install anything if you let them, but many times there is a perceived benefit to the persons work performance. So do we make the process as simple and straightforward as possible when the need arises. Do end users have a clear sense of who they need to contact and what information to provide to make the case? Are end users advised of the steps needed to validate software for use, and are they kept notified during the process? When the request is denied is an effort made to work with end users to get them the tools they need?

If you answered all or some of the above questions no you may see where the gap has been created. Taking in to consideration the reason your organization exists (usually to make a profit), you need to ensure policies don’t make you a hinderance. Well defined policies that support both employees job funtions and meet security are the best. Take a look at your policies and procedures and make sure they are clear. Educate employees on not only the hazards of unauthorized software, but the benefits of working to get software authorized. If they see the process as friendly to their needs they will begin to use it. Poorly written policies and poorly executed procedures are like putting up portable bathrooms and not maintaining them. If they are disgusting people will just go some place they are not supposed to, like in a bush or behind a car.

This doesn’t mean you will always approve every request. However, if you take the time and educate end users why you couldn’t approve their request they may better understand what other tools they can use and be more willing to follow procedures going forward. It’s worth a try.

The Recap

A resident reported an elderly female dressed in a bathrobe standing in the tree lawn. She was waiting for a visitor.

The Lesson

Growing up, there were at least three nursing homes in my town. It seemed there was always a report of a resident of the facilities wandering off. In my college years I worked as a painter in a building next to one of those nursing homes. On a weekly basis we would have a visitor who had wandered off, which we would kindly escort back. The street where the above incident took place is across the street from three elder care facilities. The first thing that struck me about the story was that I had seen this event unfold. That night as I passed by, there were already two cars and a patrol car attending to the matter.

As professionals who deal in information security, do we practice responsible disclosure. I know there is a lot of debate around this topic, but I am not focused on new vulnerabilities. I a more focused on our normal interactions with websites that appear to be in their “pajamas”. We have all come across something that just doesn’t seem right, but do we then inform the site owner of our discovery. On one hand there is the fear we open ourselves up to scrutiny as to how we found it, and on the other there is the potential to be ignored. Fear and frustration then lead us to feel it is not our problem so we take our business elsewhere, or we refuse to use the site until someone notices it and finally fixes it.

In the end the answer is difficult. Do you mind your own business and leave the woman in her pajamas on the side of the road late at night? Do you stop your car and try to get a resolution to the perceived problem. In the end there is risk and reward both ways.

It is very typical as an auditor to review your clients disaster recovery procedures. As a government auditor this was always a challenge because typically speaking most organizations I audited didn’t have the resources to test their recovery plan. A typical test  was to ensure there were backups being performed, that a written plan existed, and that backups were stored offsite in a secure location. Often clients would give us a hard time about the probability of a real disaster occurring, that the most likely scenario occurring was a loss of their server with a simple restore from backups.

Disaster

Sometime in March I was sitting at a client site when I got a phone call from another client I had been at 9 months prior. The client was rather flustered and made what I thought was an odd request. When auditing the client on the phone we get a file that contains the entire security settings for all the users, with the exception of the password. The client on the phone asked if I still had the file they had given me the previous June. When I told him it was included in the work papers from the audit he seemed to sound a little less stressed. He then asked if I could send him the file from our last audit. After validating the client identity and getting permission from my supervisor, I sent the file over.

After a few days I followed up with the client and inquired as to why they needed the file. He indicated there were some issues with their server and they needed the file in the event of a system recovery. I reminded the client the file we had did not have passwords, to which he indicated that shouldn’t be an issue. I was told they had their primary backups and that they only requested the prior audit files in the event something had gone wrong with the tapes. I then updated my manager and made some notes in the audit file for the client I would be back in auditing in the next couple months. Life as an auditor went along as normal as possible for the next few months.

Recovery

A few months later I schedule the entrance meeting with the client that had called me in a panic. Meeting day came and I went through the normal opening formalities with the client until we came to the event. In prior years we had cited the client for not having proper environmental controls or security. They had made great strides in correcting the issues, and even found a good use for the new security cameras. I asked the client why they requested the files, and got a brief moment of silence. “We had an incident that caused our systems to go down, we didn’t lose any data files” was the response. “So you were able to recover without incident then?” was my question back.

Turns out at some point in December the system file backups were spanning over to a second tape. The operations manager in charge of backups had made the decision to temporarily stop backing up the user security files until he could determine what was causing the need for multiple tapes. The change was only supposed to be temporary, but due to year end processing and other priorities the system files were never added back to the backups. A few days prior to the call to me, the server went down and when a restore from the tapes was attempted the user profiles of over 4,000 users were not available. The backup rotation only was a four week period and by the time the mistake was discovered, it had been well over 14 weeks. The new security cameras that had been installed revealed the entire drama of the operations manager trying to find the proper tapes to restore from and ultimately making the call to the director that all user profiles had been lost.

The client ultimately had to use the file we had from the previous June to restore users profiles. They then had to go through change, termination, and new users requests they received through out the year and rebuild all the user profiles. Finally they had to coordinate the reset of passwords for the over 4,000 users within a 48 hour time period. Not only did this situation provide a valuable lesson to the client, it also provided myself and my fellow auditors a valuable lesson in a true recovery process. For example, we had to spend extra time testing user access since we could not trust that controls had been operating effectively during the period under audit. This was also a great illustration that no matter how good system settings are at a point in time, they are only as strong as the controls around changing them and ensuring they are maintained properly. I hope your auditors aren’t part of your recovery plan.