The Risk

We are all aware of the risk of allowing unencrypted traffic to traverse the network.  For those in banking I expect some of you have had that conversation with a penetration tester who has presented you with some various information sent in clear text which was sniffed.  I’ll leave the rest of the implications of this security risk to your imagination.  Additionally we have to deal with the risk of hijacked printers.  IronGeek.com shows us a few things about hacking HP Printers (also see PaulDotCom.com Episode 65 show notes for some more fun with printers).  These are risks that we can mitigate.  Let’s take a look at how we can do that.

Mitigation Processes

Regarding the HP 4000/4200 series printers (both on-board and embedded JetDirect) we need to look at the various threat vectors available to a malicious intruder and some quick and easy steps to minimize our exposure.  (None of the following items are revolutionary, however they should all be considered and reviewed on a printer by printer basis.)

• Administration rights are a good start and, like seemingly every other piece of network equipment, the default administrator account is “admin” with a blank password.  Our first step in reducing our risk is to configure a strong password for this account.  On some models you will be able to modify or delete the default “admin” user-name.  Do it if you can.  Additionally, it is in our best interest to create a secondary administrator account which has a strong user-name and password for regular management of the printer.
• After you are comfortable with your new administrative account look into upgrading the firmware of the device.  Depending on the model this may provide you with a few new security features like ACLs (which we’ll look at later).  Couple this with the most up to date drivers on the print server (HP remote DoS attack for your consideration).
• The next area we want to look at is what protocols are running on the printer.  This must be coupled with a consideration of how your institution is accessing the printer.  I prefer not to use any of the embedded FTP servers or web servers/print servers.  Stick to a separate Linux or Windows print server for that functionality.  When you open the administration section on these HP printers one of the first things you should notice is the many protocols open.  I disable FTP, TFTP, Telnet, all versions of SNMP, and HTTP right out of the gates.  Also enable and force HTTPS.  Under the web management section you can force 3DES encryption for all web communication.
• Now take a look at the print protocols your institution uses.  If you do not use IPX/SPX, AppleTalk, or DLC/LLC disable those.  Also look at the print services and disable any you are not using, for instance FTP and IPP.  There is no good reason to have any protocols or services running on your printers which are not in use.
• At this point it is a good practice to use your certificate server to create a certificate to use for all of your printers.  Allowing your network assets to continue to use default certificates will lead to a bad habit of clicking through certificate error messages in general, let alone when using admin WUI’s.  Printers are no exception and you certainly do not want one of your peers to watch you clicking through SSL Cert errors in IE when accessing the printer web management portal.
• The last technique we’ll use to mitigate the risks our printers brings to our networks is ACL’s.  Within the authorization configuration section there is an option to input an IP ACL.  This list should contain all of the IP’s of workstations/servers which must communicate directly to the printer.  You should place your print server IP here and the IP of any workstation or remote service IP that prints directly to it; for instance via a custom form printing program or check printing software.  If you find yourself using the web management interface often you will want to configure your own workstation IP in the ACL to allow access without the need to log into the print server.  Also within the ACL management section most models have a button which says something like “Allow Web Server (HTTP) access” which should be unchecked to help prevent any possibility of allowing HTTP management access.

Wrap Up

If you’ve ever taken the above steps towards hardening a printer you’ll notice it takes longer to read the steps than to configure them.  These are easy steps to help protect your printers and your network.  Before taking any of these steps it is best to outline all of the routed and print protocols used to interact with your printers and consider what your IP ACL’s need to look like.  This is a daunting task if it must be done on a printer by printer basis; and usually that is the case.  HP does provide central management tools for scanning and updating their networked printers however I have had exceptionally bad luck using it with embedded jet direct cards, especially in the 4000 series printers.  The above mitigation techniques should be considered in tandem with the security approach you take regarding your network.  This is one piece of the puzzle of your defense-in-depth approach at print security which can be coupled with router and switch configurations and print server configurations to reduce the attack surface area of your network.  These techniques have the added bonus of keeping wandering employees away from your print management interfaces.

Monday 1-12-09:

• Read Chapter 2 Spanning Tree Protocol

• Review Chapter 4 IP Routing: Static and Connected Routes – I needed to review the ISL and 802.1Q configuration and the Classful and Classless Routing sections.

Tuesday 1-13-09:

• Read Chapter 5 VLSM and Route Summarization

• Read Chapter 6 IP Access Control Lists

• Read Chapter 8 Routing Protocol Theory

Wednesday 1-14-09:

• Review Chapter 9 OSPF – I am solid on all of this but will review the configuration section, especially Authentication and Load Balancing.

• Read Chapter 10 EIGRP

• Read Chapter 12 Point-to-Point WANs

Thursday 1-15-09:

• Database Fundamentals class from 5-7PM

• Read Chapter 13 Frame Relay Concepts

• Read Chapter 14 Frame Relay Configuration and Troubleshooting – I will stop before the Troubleshooting sections to leave it for Saturday.

Friday 1-16-09:
(no class because my classes are only Mon/Wed and Tues/Thurs)

• Read Chapter 15 Virtual Private networks

• Read Chapter 16 Network Address Translation

• Read Chapter 17 IP Version 6

Saturday 1-17-09:

• Read and review all the troubleshooting chapters:

Chapter 3 Troubleshooting LAN Switching, Chapter 7 Troubleshooting IP Routing, Chapter 11 Troubleshooting Routing Protocols, Chapter 14 Frame Relay Configuration Troubleshooting.

• Review IP Subnetting, Hex conversions, Binary conversions, and Route Summarization

• Review practice questions from the included ICND2 CertSim test in study mode

“The drawback with PAP is that both the username and password are sent in clear text.”

I was studying Point-to-Point Protocol and turned my attention to the authentication methods for the link. I found the above statement while comparing PAP to CHAP. What I can’t seem to understand is why PAP is still supported in newer IOS versions. Given the state of security in IT does it make any sense to use PAP? Sending the password in clear text just doesn’t cut it. For that matter, sending the username in clear text can also be a disaster. If I had to guess, a network administrator that would choose PAP over CHAP is probably the same kind of network administrator that would use the same username and password for the PPP link as he does for his Windows domain administrator account. Additionally, should we trust our PtP providers that much?

I don’t see how the marginal overhead gain of running PAP could be the selling point. I would prefer not using an authentication method over PAP for the simple reason that you could give somebody a hint of your username/password style (on top of the actual username and password on the link). As far as I can tell from what I am studying right now CHAP is the only way to go here. PAP should go the way of IGRP and lose version support.

If I am missing the boat here please let me know. I would like to know if there is a real use for PAP.

As I added items I separated things into categories like Finance, Banking, Windows, CISCO, Networking, etc. I spend a bit of time at work looking through my Reader to see what people are blogging about and then I spend more time at home as well. A lot of the days my Reader was my lunch break companion.

My main goal behind my Reader was self-education. It wasn’t until December that I realized I created something else; something that has proved to be far more useful. I created my own little self-interest based search engine, powered by Google Search.

My discovery came one day when I was trying to find extra resources for my studies into STP, RSTP, and VTP for my CCNA. I started with Google’s main search function and it yielded a bunch of links I really wasn’t interested in. So I moved to my Reader and thought I might find an article by one of the authors I follow. Sure enough, it returned several articles that were immensely useful.

I used it again to troubleshoot a problem I was having on an OSPF lab where a neighbor wasn’t forming and I wasn’t sure why. I tried searching my Reader and it returned the OSPF cheatsheet post from PacketLife.net. I recognized the mistake I made and fixed it. In comparison, Google Search returned several links to CISCO’s main resource site with documents outlining various aspects of OSPF. Using my Reader to search for OSPF was a lot faster than using a regular search engine.

The clear reason the Reader did so well is content recognition. I know what all of the content on my reader is about, and where it come from, and what I want to do with it. I know the people I read regularly are writing about OSPF so I know I can find something there. But, and more importantly, searching my Reader is more useful because once I get the results back I already know what I’m looking at. I know what articles I should look at within the results because I’ve read them already (ok so I don’t get to read everything that comes through my reader, but I read over 90% of it). Additionally, I already know that I can trust the authors because I’ve been using them to education myself for a while now.

All of the blogs I’ve been reading and tying into my Reader are a cornerstone of my self-education. I’ve learned a lot about general topics like marketing, banking, finance, networking; but also I’ve picked up specific things like what should be included in a Windows Share Point Services 3.0 install. Plus I’ve picked up a lot of new CISCO IOS commands I never knew about; some that saved time for me on my CCENT test. There is no doubt that my Reader is useful for self-education. But I never thought I would have such a useful search tool at my fingertips when I needed to look up a protocol, or find out what people are saying about a specific windows vulnerability.

My question to you is, what tools have you created for yourself while utilizing some other service or product?

The “Why”

Government regulation plays a huge part in banking. My estimation is that it will play a bigger role in the near future (putting a pin in it possibly for another post). In the small time that I have been involved in community banking, and banking in general, one thing I have learned is that “Vendor Management Program” is a buzz-word. What I mean by that is auditors and examiners are looking it at as a more and more important piece of the puzzle. A small proof of that is in this PDF from may of 2008 regarding the IT/E-Banking Program now implemented in California. For community banks this can be burdensome to say the least (much like SOX 404). But the programs are not without merit. Those who know me understand that I am very organized. It should be no surprise that I would want an organized system for handling vendors. But it goes beyond organization. Putting a Vendor Management Program in place is a time-saver in the long run and relatively cheap to put together (especially compared to SOX 404 tools!).

Make no mistake; this is a monster of a program for any companies. If you are a start up I highly suggest you implement this immediately. If you have been around for several years and have no Vendor Management Program you can expect it to take you anywhere from a quarter to the whole year to put it in place. A lot of companies -especially those with large IT needs, changing marketing strategies, and diverse retail products- will have hundreds of vendors in their current system. Do not let that scare you away from the project.

The other side of the monster is that this kind of project has to be a team effort. A company simply cannot expect the accounting department to come up with this stuff on their own. All departments must take a role in the project. More importantly, the members of those departments need to take ownership of their area and ensure that their section of the Vendor Management Program is top notch. I keep saying it over and over and here it comes again: this will save to a lot of time and hassle in the future. I mentioned I see three key junctures where the program will be useful and here they are:

Disaster Recovery

DR comes first in by book. I’ll paint the picture this way: your main building was just crashed into by a school bus, a crop duster just landed on your brand new ATM you just placed at the outskirts of town (you know, so you can quietly grow your geographical footprint), and your entire IT department is at a conference in Hawaii (I WISH!!). Would you rather have a systematic, governed, managed, updated, central repository of vendors as the disaster unfolds or would you rather have the “ask so and so, he knows who to call about fixing that” vendor program? I think it is a no brain-er. If you have not looked at Disaster Recovery you should, and it should include some information about vendors. In truth the kind of Vendor Management Program talked about here is out of the scope of a Disaster Recovery Plan. However, if used in tandem with your DR plan, the VM program can be a life-saver. When it comes to restoring your telephone system, or getting your internet connection back up, is there a better way to consider who to call then to look at your list of vendors and all the information there? If built correctly, the Vendor Management repository can be the difference in calling the company that can get your PBX up in 24 hours verses the company that takes 3 days to make it on-site.

Day-to-Day

An area where a VM program is very useful is in your regular work day. How many times have you had to ask accounting to check the invoice for the telephone number of a company you have not worked with in six months? Where do you turn when you are charged with double checking the billing process and contract of a service provider against the actual bills that come through the mail? When you fixed your Telephone Banking server 3 months ago and spent 5 hours on the phone with a very knowledgeable and helpful technician you need somewhere to store their name so down the road you can work with them again. The Vendor Management Program, specifically the map/repository, is a very useful place to store that information. But it is not just useful to yourself; it is useful to everybody. In the Network Administrators absence, should a peer need to call about that same server it would be useful for that person to have a little note in the vendor notes that says something like, “worked with Sam Garlington on the replication issue. Very helpful technician. Use him again”. Any number of day-to-day situations can be dreamed up where-in a central vendor repository can be useful.

Internal/External Audit

Lastly, we cannot forget the audit and review aspect of our lives in banking. This engulfs everything from internal review and audit to third party audits and on to exams. If you have a program in place that solidly reviews your vendors and manages information about them you never have to worry about hunting down the SAS 70 for the PRI provider. Do you need the latest results of a compliance or IT audit for your third party bill payment company? Check the vendor information in the central repository. One of the shining things vendor management can do for you, especially as banks receive more and more scrutiny during these hard times, is it can prove documentation. Document, document, document. Make notes. Review notes. Review vendors. Keep up to date on what they are doing within their companies. Should an auditor ask you what you think when company ‘X’, which happens to host your core data, gets bought out by company ‘Y’ you can say, “I thought it was a sound decision. Our company will stay in good standing with them and we made a notation to that effect on the vendor notes in our Vendor Management Program. But we will review it again when the contract is up.” Let’s not forget, in light of the some current scandals, that due diligence and proof of your work and review process can go a long way when your company is scrutinized.

Cost

I mentioned earlier that this can be done on the cheap. Let’s take a quick look at some options. A quick scan of the software finder over at Forbes.com yields the following results. There are many great tools there. Additionally, you can take the dirt cheap approach and do it all in a spreadsheet. Or hire a database guru from a consulting firm (if you have neither the staff on hand or the do-it-yourself mentality, or the time) and have them build a database and help you populate it. My personal choice is to use Windows SharePoint (specifically the free SharePoint Services 3.0) and turn your Vendor Management repository into a website.  You have tons of options for managing, updating, and allowing quick access to all the pieces and parts discussed in the previous post. There are low cost options out there for companies that do not have big bucks to throw at this kind of program. That seems to be a lot of companies at the present time.

There is plenty more that can be said about Vendor Management Programs and how useful they are. It takes a long time, a big effort, and interest to build a working and useful program. Executives should be willing to put the effort into seeing this through in 2009. By the end of the year your company and the employees there will have a major tool that will help solve problems all over the place. Whether it be when you reconsider a vendor that was bought out and wants to sell you a new product, if you need to quickly get a hold of a repair technician and need to know who is the best and closest to you, or even if you need to grab a quick document about the company for a presentation, you will find a big need can be filled by your Vendor Management Program.  Most importantly: you cannot manage what you cannot measure and vendor relationships are no exception.

At the forefront we need to answer the question “why”. The simple answer is that hard work up front outlining standards for managing your vendor relationships and developing a central storage for vendor information will save you boatloads of time in the future. We will first discuss what a Vendor Management Program should entail, and in my next post we will consider how you can use the program to save you time and heartache down the road.

Scope

It is very important to begin your Vendor Management program by defining the scope of what should be managed. During this process two things should come to the front: a logical mapping of vendors, and a Vendor Management Policy. The map should consist of general information about your vendors; contact information, sales reps, product/service genre, references and resources. Here is the process model I suggest for mapping out vendors and all the various information involved:

• Product or service genre: Here is the fun part. You get to sit down with a really big piece of paper (or fun drawing program) and map out the different categories into which all of your vendors fall. If your bank uses outsourced Item Processing does that fall under Deposits, or Information Technology? The vendor’s mapped location within your chart should logically comport with the product or service provided. You could alternatively map them based on the department oversight responsibilities, although, for reasons stated below, I lean away from that for simplicity sake.

• Contact information: This information should include at the least the company name, sales representative, service representative, address, telephone number, fax number, and e-mail address.

• Internal information: What we are concerned with here is the information pertaining to the relationship with the vendor. Who are the internal contacts for the vendor? Which department is in charge of managing that relationship (because this can be several people within the company spanning several departments I choose not to map vendors based on department oversight)? What are the billing procedures? What are the terms of the contract with the vendor? What documents, provided by the vendor, need to be kept on file (SAS 70, penetration testing results, internal audit results, proof of regulatory compliance etc..)?

• Review: This section should encompass the first 2 steps in our policy model (see below). Primarily, we want to include in our vendor review proof that the company has done the due diligence and that a relationship is acceptable between your company and the vendor. Any formal review information should be documented and attached to the vendor in this section. Additionally, all periodic relationship reviews, to include contractual, should be documented here. Any changes to the relationship and/or contract should be noted and kept on file. This section, as you will see, becomes crucial to the Vendor Management Program and your companies ability to leverage the program to the fullest extent.

• Notes: I prefer to include a general “notes” section for everything I do. This is where I compile little tidbits over time about a project that I’m working on. In the case of our Vendor Management map we are interested in those little things you learn as you use a product or service: how happy are you with the product? Does it fit the need? Is it easy or difficult to manage? Are you happy with the support you receive? Does the billing setup cause problems for you (especially compared to other vendors)? Does the product break a lot? This section is different mainly because it takes time and use to populate. I separate it from the Review section because while this information is useful as you review a vendor it is easier to access if it is organized within itself. Additionally, the notes tend to be reflective mostly of the product or service, and not so much the vendor. The “notes” section can be a lifesaver when you have to explain to management why you want to grow or shrink a relationship with a vendor.

Once you have the map in place and you know who your vendors are and what your relationship with them looks like you can begin managing those relationships more fully while leveraging your repository of information.

Policy

In concurrence with your vendor map a Vendor Management Policy should come to fruition. What should develop during the scope/mapping process is a synoptic document that can govern the companies understanding of your vendors from start to finish. I follow this process model:

1. Discovery & Research: This pertains to the preliminary stages of your relationship with the vendor. What documentation and research is required to be in place before a formal relationship can occur with the vendor? What are minimum standards for a relationship with a vendor?

2. Approval & Acquisition: An important piece of the puzzle regarding company relationships with vendors is the outline of a standard for accepting the formal relationship. Who has the authority to legally bind your company with the vendor? What is the approval process for contractual agreements with a vendor?

3. Implementation: Again, what we are shooting for here is a standard all departments can follow. Set in place processing standards for the implementation of the relationship with the vendor. Ensure the correct departments have the capability to put in place the products and services acquired in a timely fashion.

4. Management: Consideration should always be given to a standard of management of vendor relationships. In this section of the policy it should be expressed who has the senior and junior responsibilities for managing the relationship with the vendor. My suggestion is that it be departmentally based and delegated from there. Also include a brief explanation of what should be managed (i.e. all the different parts of the map given above).

5. Audit & Review: Here we want to lay out a framework for regularly reviewing the Vendor Management Program. Also, standards and responsibilities should be outlined regarding who should audit the vendors, their products, and the relationship.

6. Termination: Lastly, the policy should include language address what documentation is required to allow termination of a relationship with a vendor. Who is responsible for it? What standards are in place outlining these requirements?

What I have attempted to outline above is a framework by which your company can get all its ducks in a row regarding your vendors. Next, we will give consideration to three ways you can put this information to use. It will become evident that it is heavily important to have the documentation and oversight in place in order to leverage the Vendor Management Program.

Here in wonderful East Tennessee we have two major wireless providers that dominate the geographical footprint: Verizon and Sprint. I’m not so much interested in the comparison of the number of stores, customer perceptions based on that, actual customer base, or market share information so much as I am interested in what is in the stores. More importantly, what lessons can banks (especially community banks) learn from comparing them. I am going to kick off my blog with a multi-part series on this comparison.

First we’ll look at the Sprint stores

These stores are small. And by small I mean they are about the size of a community college classroom. They all trend towards the same rectangular interior shape with a little wall decoration, some phones along one or two walls, one work desk with a couple of employees, and maybe a kiosk for payment use. Outside they are defined by the strip-mall-ish buildings in which they are housed. They are usually found near businesses in the other parcels like smoothie shops and video game stores.

Second the Verizon stores

Most of these stores are big; 3 or 4 times the size of Sprint stores. Inside they are littered with phones, wall art, interactive kiosks, TV’s, and seating, greeting stations and visual queues. I have yet to walk into one of these without seeing 4 or more employees. They tend to be broken into two sections: a service area and a sales area recognizable by large signage. Outside the most noticeable thing is that these stores, unless coupled with a retail store like a Circuit City, are they’re own well lit buildings with lots of parking.

Thoughts

I am sure you see some serious contrast already. I want to look at the exterior first because that is a major issue with me. Verizon stores have their own parking and lots of it. Sprint stores share a few spaces with other businesses. Verizon is well lit and always noticeable from the road. Sprint stores not so much – unless you want a smoothie or they have 8 of those big flags (they’re neat, but what happens when one falls in the highway?). Pertinence to our industry? Please provide ample parking, lighting, and walkways when building new branches or expanding old ones. Maybe instead of an addition to a current branch you can expand the parking and add some much needed convenience for customers.

What about the interior you say? Have you noticed yet that the Sprint stores are a lot like typical branches? You can walk inside a Sprint store and queue into the line to get served by an understaffed group of people comparable to transactioniers who only greet you when you stand in front of them or you can step into the Verizon store where you get greeted immediately by a smiling face, queued into an electronic system for staff to search you out, walk around a room that could park two hummers and a 5^th grader’s lemonade stand, and play with more nifty gadgets than you can shake a stick at. Oh, and you can sit down. Banks tend to have the sitting covered, with a couch. Thanks. Can I keep my place in line? No? OK, I’ll stand. Community banks can take another one from Verizon by varying the size and shape of their branches and what they litter them with. What about interactive kiosks for online banking and bill pay? What about a courtesy desk with interactive software modules to help customers understand products? Where are the bank employees that maneuver around the branch seeking out people to help instead of just saying “next” every 38 seconds? It’s time to rethink this aspect of our business models.

So which store do you think wins the staffing game? Sprint sales rep, “Uh, sorry we don’t have a technician here for that phone. You’ll have to go to our store located at Lincoln Street.” Wait. What? Yes folks, Lincoln Street; a place that could be anywhere within 50 square miles (or if you live in my town you have to drive 40 minutes to the closest Sprint tech). And there’s only one, with likely one service technician at that location. I hope you don’t break your Sprint phones. The Verizon story on the other hand has a section of the building dedicated to service techs with at least two in the building and even a separate electronic queue so they can find you while you play with the phone you’re going to have to buy outright since you didn’t get insurance on your Blackberry. So that’s 3 for Verizon and none for Sprint. Lesson here: have enough people on hand that know enough to answer most simple questions about all products provided. Have specialists to help customers with hard questions like FDIC insurance coverage or what a HELOC is. PoD? What’s that? How many more times do our customers have to go into a branch and hear, “Who’s next! Car loan? No we can’t handle that here you’ll have to go to our Lincoln Street branch. NEXT.”

Great, maybe now I can get my Blackberry fixed!!!