Some experimental notes

Hi y'all from Atlanta

2012-03-04T20:44:15Z

Hi everyone,

I have always been a tinker or tester (as my parents would no doubt agree) and favour a scientific outlook (experiments, hypothesis, proof) on most things. It may be only little surprising then that I have always been interested in and now work in information security. Which could be characterized as showing people how to break their most precious things ... to make them stronger.

I have been advancing the idea in various venues that the skills and knowledge of a good tester are key to effectiveness as a security professional (or a good hacker) and that's part of why I've been involved with BBST almost from the beginning and I'm back taking Foundations again. I'm an advocate of BBST for testers and for security people of all sorts ... if not everyone in any technical field ... and I hope to be an instructor in the future.

I can barely keep up with my own rotating hobbies though they seems to oscillate around hacking, languages, and video games. Non-fiction reading these days is either on network intrusion detection or technical review of other courses still... which is not to say there isn't a pile, or that I haven't bought new books this week.

The last fiction I finished that didn't have pictures was the Hunger Games trilogy and in graphic novels I'm working my way through Fables and The Walking Dead. I don't get much time to play Star Wars online with all of this homework and the demands of woman and cat.

I take lousy photos with my mobile phone, often of my cooking experiments. You'll find some of those online, as well as various rambles, including my experiences in other BBST courses, starting from my site http://f.adric.net/index.cgi/home , which is run out of a DVCS because I am just that much of a nerd :D

Hajimemashite! Nice to meet y'all!

(adric)

comments

Risk based domain analysis of one OOo variable, for BBST Test Design Assignment 3

2011-12-03T18:41:01Z

A follow-on to the risk catalogue from previous assignment and DW post. Again here is a messy blog post on methodology and what the cat was doing (yowling) while download LibO to my iMac and as I struggle to put together the formal report specified in the assignment.

LibO lastName entry points

I started off this morning at the VW dealership with some exploratory testing to see if any of the spare few test ideas I have noted are feasible/sensible. I quickly realized a few key things. To keep my test case collection manageable I should probably settle on one or at most two ways to input values of the variable under test. In my initial risk catalog write-up (cf prev post) I identified three major entry points: start-up wizard, options, read from user settings.

( goes on for quite a bit, it does )

Some notes on tools

While blundering around I found many better ways to do what I was up to. Even without a formal harness it's easy to generate and evaluate data against a SUT this way with the built-ins of any scripting language. I settled on Python because after trying some Ruby and Perl I got going the fastest in Python .. though as I found easily enough I was still going much slower than I needed to. I should have googled sooner as that would have saved me some time and keystrokes.

Here are two things I will use in the future for this sort of exploration: Python libraries to interact directly with the platform clipboard and (if available) the Ruby black bag security toolset. There's no coincidence at all that the same tools that work for neutral/academic software testing and QA are also used to attack software and find its weak point and I've studied both.

Ed note: I've submitted the assignment to the class and will post it here after the course is done and grading is completed.

comments

Test Design Lab: technique attributes

2011-11-20T05:07:05Z

If the core attributes for test design in scenario testing are Appropriate complexity, credibility, motivation, validity and value and the desirable characteristic is ease of evaluation , then I'd say:

The core attributes for function testing are coverage, validity, and affordability. Desired characteristics of a function test (plan) are performability and reuseability. Function testing considers whether individual functions (features, groups, of functions, etc) are performing correctly and function test plans are usually based on covering all of the functions adequately with tests. At such a small scale validity is very important for function tests or they have little value. Function testing can be automated which may reduce costs. Automated function testing can be implemented into development procedures (as unit testing often is) which allows for reduced cost through reuse. The test suite can assert that the operation fo the functions tested is unchanged after changes to the SUT by developers.

The core attributes for risk based test design are credibility, information value, motivation, and power and a desirable characteristic of risk test is affordability. Risk based testing seeks to find potential failure modes fo the SUT, produce them, and gather information about their likelihood and consequences. Good risk based test design should find demonstrable bugs which case noteworthy failures in the SUT including failures that may affect other systems. These tests should help convince stakeholders to make needed changes based on the potential risks identified and demonstrated. Risk based testing can vary in it's application between quicktests which are effective for certain common failure types to more complex and expensive failure mode analysis such as that performed by engineers in safety industries.

Core attributes of specification testing are accountability, coverage, and value. Often specification testing is for purposes of determining compliance with formal specification which may have legal weight. In these cases the details fo testing may be published or even be part of the public record through a standards body or court proceeding. In these cases coverage of the entire formal specification by the test plan is completely required and the information gathered is of considerable importance to the organization running the tests. Other kinds of specification testing, such as competitive product analysis, also provide good value to the testing organization.

comments

Applying the HTSM to the Google Presentations manual

2011-11-17T01:12:20Z

From the online manual for Presentation and relevant sections of the main Docs manual we can harvest about 36 declarations of features which are mostly Functions in Product Elements:

Share presentations with your friends and coworkers. Upload and convert existing presentations to Google Docs format.
Download your presentations as a PDF, a PPT, or a .txt file.
Insert images and videos, and format your slides.
Publish and embed your presentations in a website, allowing access to a wide audience.
Draw organizational charts, flowcharts, design diagrams and much more right within a presentation.
Add slide transitions, animations, and themes to create show-stopping presentations.
See exactly what others are working on with colorful presence markers
Edit a presentation with other people simultaneously from different locations
Use revision history to see who made changes or to revert to earlier versions
Say hello, start a conversation or share new ideas using built-in chat
Create Google documents, spreadsheets, other file types, and collections.
Upload (from your computer, if you'd like), manage, and store files and folders.
Share Google Docs, files, and collections.
Preview your docs and files before you open or share them.
View images and videos that you've uploaded to your Documents List.
Search for items by name, type, and visibility setting.
Convert most file types to Google Docs format. 
Add flair and format your documents, with options such as paint format, margins, spacing, and fonts.  
Invite other people to collaborate on a doc with you, giving them edit, comment or view access.
Collaborate online in real time and chat with other collaborators. 
View your documents' revision history and roll back to any version.  
Download Google Docs to your desktop as Word, OpenOffice, RTF, PDF, HTML or zip files. 
Translate a document to a different language. 
Email your documents to other people as attachments.  
Share and edit presentations with your friends and coworkers.   
Import and convert existing presentations in .ppt and .pps file types.   
Download your presentations as a PDF, a PPT, or a .txt file.  
Insert images and videos, and format your slides.   
Allow real-time viewing of presentations, online, from separate remote locations.  
Publish and embed your presentations in a website, allowing access to a wide audience.  
Share and edit drawings with your friends and coworkers. 
Download your presentations as a PNG, JPEG, SVG, or PDF file.   
Insert images, shapes, and lines, and format them to fit your preferences.   
Real-time collaboration with other people, no matter where they are.   
Insert a drawing into a document, spreadsheet, or presentation.

We can infer these details about the environment and delivery of the product from the application and the feature list:

Presentations is an online application and runs in a web browser and on Google servers.
Presentation requires Internet access to Google servers and other Internet resources for full use of features.
Presentations accepts file upload from the web browser in certain file formats and outputs files.
Presentation files may also be embedded into other web sites.
Presentations uses Google's shared user authentication and authorization systems and Google search.
Presentation is free to use without charge for anyone with a Google account.

and so fill in some other details of the Product Elements:

Structure: and Operations: Presentations is delivered as a online service (SaaS) and there is no physical product.
Platform: Google's platform is used for (at least) storage, authn, authz, search, revision control, machine translation, document format conversions, and application delivery.
Data: Presentations reads in and writes out in multiple well-documented formats and also has an internal format.

We can also fill in some information about possible Operational Quality Criteria.

Compatibility: Presentations reads and writes multiple file formats besides it's native data format.
Compatibility: Presentations is sensitive to web browser feature levels.
Installability criteria may not be applicable since Presentation is delivered as a service
Security in Presentations is implemented with features of Google hosting services and may not be directly testable.
Capability: A serious failure or lack of any of the features bragged about in the manual will strongly impact quality

comments

Brain rejecting assignment, not going well.

2011-11-16T23:58:44Z

Unfortunately the next assignment is repelling me forcefully. Even after a few flybys as I try to actually dig into it it's not passing my "this is dumb" filters and is being rejected by a voice in my head yelling about how dumb it is and telling me to run or find something productive to do.

To help us learn how to digest and actually get useful information from specs and other complex documents the lectures and reading explain active reading techniques and emphasize the use of mind-mapping software. The assignment is to use a mind mapping application to make a map of a specification and answer some questions about the results. If you haven't got or aren't familiar with the mind mapping tool you are encouraged to snag it and start in early in the assigned time for this assignment , and I did yesterday with mixed results.

Here's where it gets choppy: I haven't met a mind mapping program that I can actually use effectively, though I have tried a few a few times. Much as no note-taking application is faster or more versatile for me than scribbles on paper (alas I would this were not true, see recent /. discussion for ample discourse. tl;dr use a pen and paper) I have to whiteboard or pen sketch flowcharts, timelines, swimlanes and especially formal maps (at work) before trying to fight them into a computer. So this assignment's technique is unlikely to work well for me however awesome it is. And confirmation bias as it may, I had enough trouble inputting the skeleton into the mapper yesterday that I'm pretty convinced it's a net-loss for efficiency and don't want to use it again, certainly not for inputting data.

But the real problem is that the specification document we are supposed to analyze isn't a spec. It's the bleedin' online manual and is mostly full of marketing and fluff. I haven't seen any numbers in the parts I have tried to skim and if there's a section of fluff about interoperability I haven't been able to find it yet.

I think I could have surmounted (kludged) one of these two problems but with both staring me down I'm locking up. I should be able to analyze this app from the fluffy manual and using it, but I won't have anyhting in 3-4 hours but a headache and a wall-white board full of scribble which it would take me a couple hours to clean up and get into the mapper (Or faster into Visio, Omnigraffle, Inkscape in order of speed and cost).

I've either got to learn to type quickly or the computers need to learn to understand my scribble and/or when I yell at them .. but this shouldn't have anything to do with how to analyze a spec or mock the thing they gave you in the specification folder.

I guess I should try and active read through the spec, taking notes as best I can (still no study skills to speak of) and ignore the map for the remaining 2-3 hours and try and come up with something. It either that or I'll drawn a moderately useful map and need another couple hours to get it into the shiny metal box on my table here. Argh.

ETA: a snip from the assignment to demonstrate the gulf between these techniques and anything that will actually work for me:

Every sentence of a specification should be telling you what the product is (Product Elements), in what way it is good or bad or needs to get better (Quality Criteria), or how it will be built and the context in which it will be built (Project Environment). As you find information about the product, note it under one of the topics or subtopics under these main headings.

This sounds like something best accomplished with printouts, scissors, and maybe a bunch of index cards. Then once you have something maybe you can put it in a computer. Am I really so far out on this? How can anyone actually organize a bunch of random crap on a computer?

ETA: Pics or it didn't ...

comments

Risk analysis of one OOo variable, for BBST Test Design Assignment 2

2011-11-12T22:02:32Z

I worked through the lab assignment's questions and now need to covert this into a list of risks for other to critique. This was so much fun to write I wanted to hang onto it anyway even as an intermediate product. My actual submission for class is below the cut.

1) The variable of lastname is input (optionally) in the wizard that runs on first program execution of OOo applications, modified in the Options dialogue of any OOo application, and used in every facet of the office suite.

2a) Undefined is a valid state for this variable and any code path that uses the variable without checking for undef or doing so incorrectly will introduce errors in its functions. Additionally, too much data in this field would also be dangerous to any of the code that uses it do the likelihood of buffer overruns and unexpected characters or encoding in this field could lead to format string errors or exceptions in library string-handling code. If you can get a non-character or non-string value into this field due to input validation failures then wholesale memory and stack corruption becomes a concern.

2b) Use of lastname as well as the companion variables first name and initials is widespread throughout the SUT applications. Beyond the dialogs which directly manipulate this value (new user wizard, Options) many other functions read this variable and incorporate it into interface displays (document properties) or include it in requests to other modules (printing). The name variables are included in various places in the document data saved to disk automatically and intentionally including the document properties. If change tracking is enabled a tag generated from name variables and dates is displayed next to each change made by a particular user and recored with version information in the document files. Perhaps most excitingly the name variables are posted to Internet servers with registration information allowing for the small possibility that an error related to this variable could affect not only systems that process the document but completely remote systems as well!

2bi) Lastname is used in (at least) many display functions in all parts of the SUT applications, change tracking functions, save/load functions, printing functions, macros, user preferences, data generating functions such as headers and footers, and online registration.

2bii) Values of lastname are displayed in numerous parts of the UI, in change-tracking feature's tags, inside saved documents (and temp/autosaved ones), and may be printed depending on settings for header/footer and cover pages.

2biii) Values of lastname are sent to the operating system as part of stored data about the user and document as well as to remote devices for printing (settings dependent) and to remote Internet servers with optional software registration. I'm unsure about how lastname values may be used in API calls and macros.

2biv) Values of lastname are sent to the operating system as part of stored dat
a about the user (registry UserProfile.xcu) and document (meta.xml).

2bv) Values are read from the registry user profile files if available and may be input into the SUT via the first-run wizard or on demand with Options dialogs.

2c) Changes to the presence or boundedness (?) of last name during program operations could lead to corrupted data in memory, on disk, and displayed to users.

2cviii) Display of user data, document data or metadata could be impacted by incorrect information about the presence (undef), values (could change), or boundedness (wrong data type) of last name.

2cix) Boundary errors on lastname could influence other variable values in document metadata, document content, or application configuration leading to problems with these unrelated variables and functions. Gross misbehaviour on the part of lastname could completely corrupt XML program and document data structures rendering the document or preferences unreadable and thus broadly disrupting document or application functionality.

2cx) There any number of cases where software errors could cause the value of lastname currently in memory and on disk in the user registry or document to become de-synched. This could lead to incorrect data being saved or printed. Some of these cases include local or remote file system errors, unaccounted for 'races' with other OOo (or alien) process running on either the SUT or the file storage device, or just faulty RAM.

2cxi) Lastname is optionally sent with registration information to remote Internet servers operated by the OOo project. A chained failure of input filtering or other unlikely occurrence could cause unexpected format, encoding, or sized lastname to be injected into the remote system and processed. It is not entirely far-fetched that this could lead to serious problems on the remote system(s) that receive and process the data such as a buffer overflow or SQL injection attack.

2cxii) Perhaps the most outlandish and unlikely risk is to the hardware of a printer that receives a document to print with malformed or corrupt lastname information in the cover page, headers, or document body. The could lead the printer to malfunction, develop sentience, or start making toast if enough failures chain together in just the right (wrong?) way.

( Risks for OOo Last Name: a continuum of failure stretching towards disaster )

comments

Hello from Atlanta!

2011-11-07T22:04:18Z

Hi everyone!

I am quite excited (and nervous) to be diving into a new BBST course after how much I learned from the previous ones (and how much work they were).

I'm a security analyst in a small business unit of a really large company and before that I was in IT as a system administrator and what all else.
Cheerfully enough I live within walking distance of the office and try to take advantage of that as often as weather permits.

I use testing techniques, especially those learned in BBST courses, in a lot of non-development software work including vulnerability assessment and configuration management. I have a sneaking suspicion there is a crossover between security analysis and software testing methodologies in my future careers ...

I like testing and automation tools because they support my scientific focus on dealing with computers and users both. I demand reproducibility and hooks for automation in, well, practically everything ... and not just at work. I've been heard to cry out "It's computer science, not computer superstition!" on occasion when reboots(!) are suggested as a solution** to a problem.

Outside of work or software even I read escapist fiction and nerdy non-fiction, watch some telly, bemoan how I'm not keeping up with my foreign language studies, crafting, or martial arts lessons, and then get distracted by video games, cats, or other humans.

Greetings!

** Workaround they may be but not a solution and they destroy any hope of researching that instance of the problem...

comments

BBST BA: final (exam) lap, now with extra obstacles

2010-04-05T17:12:39Z

Ugh. ( Background, narrative, details of the last week of class )

Still, I am troubled by these conflicts and felt I should write up what's going on. I'm not happy to be scraping the course rules and I'm really concerned about this making the final still more difficult than it was going to already be.

comments

Saturday into Sunday

2010-03-28T13:38:19Z

Yesterday didn't go particularly well. I overslept quite a bit, climbing out of bed at the crack of noon (in the olde style). I piddled around at homework and spent more time playing Dawn of War battles and chatting than making progress against annoying assignment Phase 3. Finally, after bedtime had well past I wrote up what I could for the assignment, holding back my remarks on the assignment or my classmate for the course eval. And Friday's calf cramp throbbed mildly all the while.

Today is already better. Up at the luxurious weekend hour of 8:30 I have had tea and checked email. I stretched out a bit and did 15 minutes on the fake bike. A paltry sum to be sure, but an important psychological victory to complete a short "ride" without getting a cramp or otherwise injuring myself :/ And the CSI I put on to watch while "riding" isn't too bad. Second cup of tea is steeping.

I misunderstood the course schedule of assignments in my favour. This whole week I have merely to answer and discuss sample exam questions in the forums and try to grade two other students Phase 3 writings (as well as two more videos and quizzes). The final will be next week, which is a much better scheme for me being able to finish this week before the con. It's still plenty of work but not nearly as bad as I had thought.

I think roomie and her boyfriend are here somewhere ... haven't seen them really since dinner Thursday. I'm waiting until a decent hour to go take a shower, say 11 ?

comments

OOO Impress and Bitmap files

2010-03-27T21:36:09Z

So I'm attempting to investigate a file save bug report for class and not making much progress with that. I did trip over some behaviour that seems odd to me: OOO process BMP to PNG silently and mercilessly when you add one to a presentation, and when you save it. This is odd because it does not do this to gif, png, or jpg. Gif, png, and jpg files came through the process fine, though they were renamed. I just checked a tif and it was also unaffected (renamed but checksums match).

Must be a feature?

( shell transcript )

comments

Productivity? BBST madness

2010-03-20T16:52:54Z

Well, after finally crawling out of bed at 10 I have:

* read over the final exam study guide, (terrifying)
* read over the assignment guidelines (phase 1 is due at midnight tonight latest),
* read over a dozen OOo Impress bugs, (I need to comment on one)
* installed OOo on the Dell mini 12 (might be able to reproduce a crash bug on that platform!)
* bought ~~two~~ three** Social Distortion albums from Amazon MP3
* grazed on crackers, crisps and jerky
* finished a cup of citrusy green and brewing a cup of PGtips now.
* made a presentation of no worth in OOo Impress to get familiar with it.

I still have to:
* pick a bug report, and then both comment usefully on it and write about the report
* make intelligent sounding remarks on answers to previous exercise (others and my own, should have been done earlier in the week)

And this is just the first week of the crazy accelerated 4 week class. WTF was I thinking trying this again while working fulltime?

** So far: Mommy's Little Monster, Social Distortion, Prison Bound

comments