Web Technology @ CCBC

Student driven blog for all things web.

My “test lab”

Posted by killerkulture on February 24, 2008

I test websites, mostly e-commerce and non-tranactional sites (less frequently). When I test websites, I’m testing the following (keep in mind that I havent received a requirements doc or functional spec in over 2 years, so most of my testing is based on ecommerce standards, usability best practices, and common sense, and some of the things I test for aren’t traditionally QA tasks) :

  • Functionality
  • Data Validation
  • Usability (ease of use, conformance to industry standards, intuitive flow, navigation, design, etc.)
  • Search Engine Optimization
  • Form field validation
  • Data validation (web to db)
  • Integration between the website and ERP
  • Data migration

I’m really excited about web standards because without them there are too many uncontrollable variables when you consider browser settings, computer configurations, and on top of it all add user personalization.

Since there aren’t enought people and computers in the world to test for every possible combination out there, I first needed to decide which browsers, OS’s, and configurations to test in. There are an infinite number of different combinations of browsers/versions/resolutions out there and thankfully we have Omniture which I use to determine which settings/configurations our site visitors most commonly use (just like politics, you can’t please everyone – so I look at the metrics and determine who the “majorities” are). From these metrics I determine:

  • 3 Most frequently used operating systems – 2 for PCs (WindowsXP and Vista), and 1 most frequently used Mac OS. The ratio of PC to Mac visitors is about 90/10.
  • Most popular PC browsers (and versions) used by our customers. I usually test against the top 3-4 PC browsers and top 2 Mac broswers. For my situation this comes down to IE 6.0, IE 7.0, FF 1.5, FF2.0 for the Windows users and FF 2.o and the latest Safari release for Mac users. Keep your eye on that Windows Vista + IE 7.0 combinaton…I find alot of design flaws there, especially when Flash is involved.
  • 2 groups of the most commonly used monitor resolutions:
    •Equal to 1024 x 768
    •Above 1024 x 768
    •Below 1024 x 768
    …About 60% of our site visitors use “1024 x 768 AND below” and 40% use “Above 1024 x 768”. Since the majority of our site visitors belong to the “1024 x 768 AND below” group, from a design perspective I use 1024 x 768 as the base line to test against. In other words, the site must look and function ideally in 1024 x 768 (which I constantly argue about with our design team who are all Mac users on 30″ monitors. I swear- one day I’ll become rich from inventing a “Monitor cleaner/Nose smudge remover” for these users.)
  • Browser window dimensions: is the reason why I group 1024 x 768 users into the “1024 x 768 and below” bucket instead of grouping them as “1024 x 768 and above”. Based on the Omniture metrics, the majority of site visitors who have their monitor resolutions set to 1024 x 768 do not have their browser windows fully maximized to fill their monitor heighth and width capacity. This can be attributed to browser toolbar add-ons, variable Windows OS taskbar heighth, and users wanting the ability to see multiple windows. (Keep that in mind next time you purposefully design a chrome-less site to exclude the scrollbar.)

So basically I have two computers at my desk:
1. PC
• OS = Windows XP
• Browsers – IE 6.0, FF 2.0
2. Mac (iMac)
• OS 10.x
• Browsers = FF 2.0, Safari (latest ver.)
…my co-workers have other computers with different browsers that I ask them to test on when I’m testing for cross-browser compatibility.

Here are the tools and browser plug-ins that I use, and the purpose for using them:

1. Firefox plug-ins:

  • Live HTTP Headers
    – Website Front end testing: when I’m testing session data and behavior (loggin in/out, loggin out, multi user computers, saved and abandoned shopping carts, availability of secure customer data through use of the browser’s navigation (back button)).
  • Flash Versions
    – Since Flash is so widely used and accepted (arguably), I don’t test to see how the sites behave in flash-less browsers…instead I use a Firefox plug-in application to test the site in different versions of Flash (offered by adobe/macromedia).
  • FireForm (Form Saver)
    – When you need to test anything involving user input (post data incl form field validation, drop down menus, etc) like product searches, order placement, user registration, logging in, etc – form saver is good for quickly pasting data into form fields (but don’t let it make you a lazy tester).
  • Resize Window
    -Allows you to resize your browser window to emulate different monitor resolutions without doing the old “Show desktop, right click, properties, settings, change to new monitor resolution, ok, save, yes (save changes)…” for each resolution you’re testing for.
  • Firebug
    – “Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page.” Since our developer’s are already debugging before it even gets to the testing phase, I usually use Firebug for displaying the details of script errors I find in the different browsers.
  • IE Tab
    – It’s a huge pain to test in all of these different browsers, but thankfully, this add on alleviates some of that by loading IE in a Firefox tab.
  • Recorders (iOpus Macros, Selenium IDE)
    – Repetition, complacency, and boredom are the enemies of testing. If you have repetative tasks that require the same actions over and over again but don’t require an eye for details, sometimes it makes sense to record these front end tests so that you can quickly execute them later – like for regression testing. Big caveat here though – maintaining automated/recorded tests if you rely on them too heavily….so don’t. I use them when I need to verify that user input is getting to the database.
  • SEO, SEOquake
    – Aside from validating tracking pixel behavior, most QA analysts and testers don’t need to touch any of the marketing duties. However, the marketing team in my dept doesn’t really know what SEO stands for, so I’ve made SEO testing part of my checklist….mostly checking to see that title tags have content and that the pages are crawlable/indexable by the search engines. Because SEO really touches on every piece of a website, these plug-ins help with keyword analysis, measuring inbound/outbound links, the Search engine friendliness of your pages, and much more.

2. Xenu

  • Xenu is a free application that you can use to automagically crawl your site for broken links and assets.

3. Mantis

  • I’ve just installed this open source bug tracking application on my web server, and it seems to be pretty decent. After 3 days of struggling to install Bugzilla, I opted for the ease of the Mantis installation.

4. TestLink

  • For the last 5-6 years I’ve been writing test cases in Excel…with no real structured method of managing them (aside from file naming conventions.) While searching for an open source test case manaement application I came across TestLink….it allows you to assign test cases to test suites, builds, releases, requirements, test plans, and projects. Seems pretty good so far.

5. Clear Cache

  • For many tests, you’ll need a fresh browser. Clear cache adds a button to Firefox that enables you to quickly clear your browser’s cache between tests (without having to go to the toolbar>tools>clear private data>etc.

6. Screengrab, SnagIt

  • I’m constantly taking screenshots of website defects. For a while I was just doing Print Screen and photoshop or MS Paint, but Screengrab allows you to take a screenshot, edit, crop, copy, annotate, etc without leaving the browser. Infortunately, Screengram doesn’t grab Flex very well, so for that and for IE I use SnagIt.

3 Responses to “My “test lab””

  1. akeatin1 said

    Thanks for sharing. Some of these tools I have stared to see in my readings etc, but I’ll definitely make some notes to myself for when I start to do more website design.

  2. mashkenes said

    I second the previous comment.: it’s good of you to share your experiences, methods and tools. I, too, will refer back to some of what you cited in your post as I get further into web design.

    I did, though, get hung up a bit on your statement that you “haven’t received a requirements doc or functional spec in over 2 years…” I’m probably making more of it then there is or perhaps getting lost in some of the semantics but… if you’re not being given the precise requirements or specs, how can you do complete functionality testing? And if the site is perhaps not doing what it was functionally intended to do, usability, data validation, form field validation and other testing wouldn’t really seem to matter.

  3. killerkulture said

    Almost every project ends up with me having to dissect, investiage, and reverse engineer the new functionality because everyone’s “too busy” for documentation. I don’t mean “too busy to document” in an Agile Methodology kinda way…I’m in the “documentation is a constraint to productivity, and besides that, what’s a Requirements doc/functional spec anyways?” kind of environment.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: