Web Technology @ CCBC

Student driven blog for all things web.

Archive for the ‘QA’ Category

My “test lab”

Posted by killerkulture on February 24, 2008

I test websites, mostly e-commerce and non-tranactional sites (less frequently). When I test websites, I’m testing the following (keep in mind that I havent received a requirements doc or functional spec in over 2 years, so most of my testing is based on ecommerce standards, usability best practices, and common sense, and some of the things I test for aren’t traditionally QA tasks) :

  • Functionality
  • Data Validation
  • Usability (ease of use, conformance to industry standards, intuitive flow, navigation, design, etc.)
  • Search Engine Optimization
  • Form field validation
  • Data validation (web to db)
  • Integration between the website and ERP
  • Data migration

I’m really excited about web standards because without them there are too many uncontrollable variables when you consider browser settings, computer configurations, and on top of it all add user personalization.

Since there aren’t enought people and computers in the world to test for every possible combination out there, I first needed to decide which browsers, OS’s, and configurations to test in. There are an infinite number of different combinations of browsers/versions/resolutions out there and thankfully we have Omniture which I use to determine which settings/configurations our site visitors most commonly use (just like politics, you can’t please everyone – so I look at the metrics and determine who the “majorities” are). From these metrics I determine:

  • 3 Most frequently used operating systems – 2 for PCs (WindowsXP and Vista), and 1 most frequently used Mac OS. The ratio of PC to Mac visitors is about 90/10.
  • Most popular PC browsers (and versions) used by our customers. I usually test against the top 3-4 PC browsers and top 2 Mac broswers. For my situation this comes down to IE 6.0, IE 7.0, FF 1.5, FF2.0 for the Windows users and FF 2.o and the latest Safari release for Mac users. Keep your eye on that Windows Vista + IE 7.0 combinaton…I find alot of design flaws there, especially when Flash is involved.
  • 2 groups of the most commonly used monitor resolutions:
    •Equal to 1024 x 768
    •Above 1024 x 768
    •Below 1024 x 768
    …About 60% of our site visitors use “1024 x 768 AND below” and 40% use “Above 1024 x 768”. Since the majority of our site visitors belong to the “1024 x 768 AND below” group, from a design perspective I use 1024 x 768 as the base line to test against. In other words, the site must look and function ideally in 1024 x 768 (which I constantly argue about with our design team who are all Mac users on 30″ monitors. I swear- one day I’ll become rich from inventing a “Monitor cleaner/Nose smudge remover” for these users.)
  • Browser window dimensions: is the reason why I group 1024 x 768 users into the “1024 x 768 and below” bucket instead of grouping them as “1024 x 768 and above”. Based on the Omniture metrics, the majority of site visitors who have their monitor resolutions set to 1024 x 768 do not have their browser windows fully maximized to fill their monitor heighth and width capacity. This can be attributed to browser toolbar add-ons, variable Windows OS taskbar heighth, and users wanting the ability to see multiple windows. (Keep that in mind next time you purposefully design a chrome-less site to exclude the scrollbar.)

So basically I have two computers at my desk:
1. PC
• OS = Windows XP
• Browsers – IE 6.0, FF 2.0
2. Mac (iMac)
• OS 10.x
• Browsers = FF 2.0, Safari (latest ver.)
…my co-workers have other computers with different browsers that I ask them to test on when I’m testing for cross-browser compatibility.

Here are the tools and browser plug-ins that I use, and the purpose for using them:

1. Firefox plug-ins:

  • Live HTTP Headers
    – Website Front end testing: when I’m testing session data and behavior (loggin in/out, loggin out, multi user computers, saved and abandoned shopping carts, availability of secure customer data through use of the browser’s navigation (back button)).
  • Flash Versions
    – Since Flash is so widely used and accepted (arguably), I don’t test to see how the sites behave in flash-less browsers…instead I use a Firefox plug-in application to test the site in different versions of Flash (offered by adobe/macromedia).
  • FireForm (Form Saver)
    – When you need to test anything involving user input (post data incl form field validation, drop down menus, etc) like product searches, order placement, user registration, logging in, etc – form saver is good for quickly pasting data into form fields (but don’t let it make you a lazy tester).
  • Resize Window
    -Allows you to resize your browser window to emulate different monitor resolutions without doing the old “Show desktop, right click, properties, settings, change to new monitor resolution, ok, save, yes (save changes)…” for each resolution you’re testing for.
  • Firebug
    – “Firebug integrates with Firefox to put a wealth of development tools at your fingertips while you browse. You can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page.” Since our developer’s are already debugging before it even gets to the testing phase, I usually use Firebug for displaying the details of script errors I find in the different browsers.
  • IE Tab
    – It’s a huge pain to test in all of these different browsers, but thankfully, this add on alleviates some of that by loading IE in a Firefox tab.
  • Recorders (iOpus Macros, Selenium IDE)
    – Repetition, complacency, and boredom are the enemies of testing. If you have repetative tasks that require the same actions over and over again but don’t require an eye for details, sometimes it makes sense to record these front end tests so that you can quickly execute them later – like for regression testing. Big caveat here though – maintaining automated/recorded tests if you rely on them too heavily….so don’t. I use them when I need to verify that user input is getting to the database.
  • SEO, SEOquake
    – Aside from validating tracking pixel behavior, most QA analysts and testers don’t need to touch any of the marketing duties. However, the marketing team in my dept doesn’t really know what SEO stands for, so I’ve made SEO testing part of my checklist….mostly checking to see that title tags have content and that the pages are crawlable/indexable by the search engines. Because SEO really touches on every piece of a website, these plug-ins help with keyword analysis, measuring inbound/outbound links, the Search engine friendliness of your pages, and much more.

2. Xenu

  • Xenu is a free application that you can use to automagically crawl your site for broken links and assets.

3. Mantis

  • I’ve just installed this open source bug tracking application on my web server, and it seems to be pretty decent. After 3 days of struggling to install Bugzilla, I opted for the ease of the Mantis installation.

4. TestLink

  • For the last 5-6 years I’ve been writing test cases in Excel…with no real structured method of managing them (aside from file naming conventions.) While searching for an open source test case manaement application I came across TestLink….it allows you to assign test cases to test suites, builds, releases, requirements, test plans, and projects. Seems pretty good so far.

5. Clear Cache

  • For many tests, you’ll need a fresh browser. Clear cache adds a button to Firefox that enables you to quickly clear your browser’s cache between tests (without having to go to the toolbar>tools>clear private data>etc.

6. Screengrab, SnagIt

  • I’m constantly taking screenshots of website defects. For a while I was just doing Print Screen and photoshop or MS Paint, but Screengrab allows you to take a screenshot, edit, crop, copy, annotate, etc without leaving the browser. Infortunately, Screengram doesn’t grab Flex very well, so for that and for IE I use SnagIt.

Posted in QA | 3 Comments »

Good testers can make great developers (but great developers are rarely good testers )

Posted by killerkulture on February 5, 2008

I’m a Web Application Quality Assurance Analyst (e-commerce mostly)…which means that I’m responsible for authoring test plans, test cases, testing strategies, identifying risk, finding and tracking defects, and analyzing business requirements and functional specs (which almost always ends up becoming a full blown requirements gathering research project through reverse engineering) for websites.

One task that was recently added to my plate was the duty of training junior developers as website testers before allowing them to delve into coding. This was a great experience because not only did it teach the junior developers how the site functions and how the data is stored, shared, and validated between systems, but most importantly, it taught them how to use the site as an inexperience end user/customer. 3 months later, these junior developers are thinking through every scenario (before coding) that users may encounter, and expanding their duties and mindset beyond just coding the “happy path”. They’re actually much more thorough than the senior developers on the team. Now the managers on the team see the junior developers as extremely valuable resources because of the knowledge and experience they’ve gained by learning (through exploratory and structured testing methods) the ins and out of the web site’s front end, all of the different paths that customers can take, all of the different types of data that customers can input, where/how that data is stored, how the data is validated, communicated, and shared with other systems within the company, business rules, order and fulfillment processes, page structure, asset/content management, and how to think like a customer.

I can’t recommend this enough…improving your testing and QA knowledge will help you tremendously with creating better websites. There are a lot of websites on software testing and QA. Some of these include:





Posted in QA | 1 Comment »