Evaluation
Interactive Children's book: Wizard Of Oz
Exercise 1
The first criteria set out was for the Cd to be fun to use. This will be measured by observing the user, and analyse how they react when using the site. To Quantify this, the site would be considered engaging if the user stays on the site for longer than 2 ½ minutes.
Criteria
How long does the user stay on the CD-ROM?
The user was timed and used the CD for over five minutes
Does the user laugh when using the CD-ROM?
The users facial expressions changed dramatically from one stage to the next. The user laughed and smiled through the introduction.
Does the user smile when using the CD-ROM?
The user intently sat and smiled throughout the different stages of the CD-ROM.
Does the user look confused?
No
Does the user read the whole story or leave through it?
The user completed the story.
Does user try game?
Yes the user coloured in Dorothy
Does the user exit without difficulty?
Yes the user had the option to leave throughout the site, and left when all options were explored.
Exercise 2
Enlightenment: The user will be asked to find different information relateing to the scarecrow throughout the story. The user was tested on this to gauge the level of enlightenment achieved.
Questions
What is the name of the Scarecrow?
The user answered the question correctly (Sylvester)
Where did Dorothy find the Scarecrow?
The user answered the question correctly (Field of blueberries)
Who was the scarecrow wanting to stop from eating the fruit?
The user answered the question correctly (The birds)
What does the scarecrow not have?
The user answered the question correctly (A brain)
Exercise 3
Surprise & Wonderment: The user will navigate through the site and find the colouring section, also help section featuring the good fairy wand. This will test the sites stanard of navigational features.
Criteria How many steps did it take to find the colouring section?
Two clicks from the main page.
How mant steps did it take to find the good fairy wand?
One as it is located on all pages
Were any difficulties experienced by the user?
No
Evaluation & Recommendations
The results basically state that the CD-ROM was a success. Our proposed product fits the brief as the user finds enlightenment from the interaction which takes place. The product is fun to use and the user learns indirectly through its use as seen in the testing results. It utilizes as many of the senses as possible and the product provokes surprise and wonderment for the user. The user is rewarded with entertainment, education and enjoyment.
The options available to the user were carefully planned to best represent what the user may want to do at any given time. For example, in the linear story the user is allowed to go back to the previous screen, and also have a link to the main menu area. Choices are presented to the user at appropriate times. The user is not expected to neither read a lot of text before interacting, nor be forced to select too many options. Every page contains a reward or point of interest.
To bring life to the story, bright colourful images are used, with animation and sound for emphasis. The pictures animate, dependent on predetermined user actions.
Alternate endings or tasks built in to the story add to the level of interaction. The user feels they play a part in the story and adds to the experience and make it more engaging.
All products can be improved and this one is no different. If a voice activated system was added it would increase its usability to enable diabled people to use it. Also different stories could be added to give the user a better variety of choice.
Evaluation Critique
Was the evaluation plan clearly defined?
The evaluation plan set out by Virtual Vision was clearly defined and thorough. Using a questionnaire, both indirect and direct observational tests and a task analysis, the evaluation plan clearly identified the aspects of the program which needed to be examined.
Could you carry out the three evaluation tasks based on your knowledge of the evaluation technique and the additional input from the development team, or did you have to consult the development team further to clarify what they wanted you to do?
The tasks set out for the purpose of evaluation were easy to perform and complete. The questionnaires were clear and easy to follow. The observational tasks were non-inhibitive and involved no additional input from Virtual Vision. The task analysis proved to be uncomplicated and the specific tasks which were to be completed corresponded well with the product to be tested.
Was there a clear relationship between the evaluation tasks and the criteria that were supposed to be the reason for carrying out the evaluation plan?
One of the criteria Virtual Vision aimed to evaluate was ‘ease of use’. The observational and task analysis methods were a good choice to ensure an accurate result. The team were able to view the users who were faced with the product for the first time. Any problems or times when the user was ‘struggling’ would have been easy to detect. The task analysis ‘coffee break test’ adequately evaluated ‘informative-ness’ as we the users had to perform the tasks set out by Virtual Vision to demonstrate the learnability of the product. Virtual Vision were able to also record how long it took for the users to retrieve the appropriate information. The third and last criterion set out by Virtual Vision was the adaptability of the product. The observational tests and questionnaires provided sufficient feedback for the team to evaluate adaptability.
Was it easy or difficult to link the results of the evaluations to the general meaning of the brief? If you think things could have been better, please state how.
Virtual Vision stated that - ‘the proposal fits the brief as it provides an informative, personal experience to the intended user. It shall give the users a feeling of satisfaction and achievement through being able to bring their ideas to life.’ The results of the evaluations are easily linked to the meaning of this brief. The questionnaire showed that the users were given a sense of satisfaction in viewing the different materials which could be applied in the program. The observational tasks showed Virtual Vision the sense of achievement and ability of the users to bring their ideas to life as it was noted that the users tried all of the different options available to decorate each room and house.