Using Web-based Resources in Teaching Mathematics

Introduction

The scope of this study is to assess the impact of using web-based resources in the teaching of mathematics on students’ attitude and learning.

Three factors define the scope of this study:

• To determine if visual representation of mathematical concepts and interactivity enhances learning
• Impact on student attitude and motivation
• To determine feasibility of using of tools that are freely available from the perspective that limited financial resources should not constrain teachers from use of computers in classroom teaching.

8 students of Grade VIII and 7 students of Grade VII at an international school in Bangalore, India were a part of this study. Given the qualitative nature of the study, the small sample size enabled interaction and discussion at depth.

Student profile

Most students had access to computers at home and used it mostly to play games, watch videos and search for information on school projects. Only a few of them used computers for accessing educational websites/ CDs or doing homework assignments. Their past academic performance varied significantly, indicating that the student representation was not homogeneous.

Current practice

The school chosen for the study is a new school, only in its second year of operation. The school has a computer lab with internet access but it was not yet used in teaching any subject. It served more as a resource center for staff and also for students to create their project reports.

The regular mathematics teachers are aware that web-based resources can be used in teaching. However, practical difficulties in structuring lessons and lack of awareness of specific resources that suit the curriculum limit them from actually incorporating web and other computer based resources in teaching of mathematics.

Strategy

The first step was to identify freely available web-based resources relevant to the school curriculum and due to be covered in the first semester of the current academic year (2009-10).

The next step was to teach the students the selected topics over a 3-day period. The motivation levels and attitudes were assessed through observation, feedback collected in writing at the end of every class and finally a group discussion. Learning was assessed through a short pen and paper test covering the topics that were taught using the web-based resources.

Identifying the web-based resources

The web-based resources were identified through internet search and from educational resources such as MERLOT (Multimedia Educational Resource for Learning and Online Teaching). The topics covered (sources listed in Appendix 1):

Grade VIII

· Introduction to quadrilaterals

· Nets

· Linear equations in 1 and 2 variables

· Time- distance & velocity

Grade VII

· Fractions

· Linear equations in one variable

The resources were identified keeping in mind the following criteria:

· Provide an opportunity for interactivity

· Show dynamically the impact of manipulation

· Allow students to investigate without the fear of going wrong

· Did not require any training for use

Research process

The sessions were conducted in the computer laboratory and it was possible to alternate between using a whiteboard and the computers.

A text file containing links to the web-based resources was copied onto each computer. Students were first introduced to the topic using a regular whiteboard and then a brief instruction was given to the students on how to use the web resource.

Given the limited number of computers available, most students had to share them, with one computer to two students.

As this was the first time the computers were being used, it was decided that the scores of the test would not count towards their annual assessment. Feedback was collected in writing at the end of each day. The group discussions were conducted on the third day, after the test.

Assessment of Learning

Grade VIII

Time-distance:

The relationship between time, distance and speed was understood by most students. However, only the academically stronger students managed to produce a correct graph using the information provided in the question.

Linear equations:

While students understood that a linear equation in two variables represents a straight line, identifying points on the line by x and y coordinates and the concept of slope seemed to be difficult. This was their first exposure to the coordinate geometry and the students clearly needed more preparation before introduction of (and more time with) the web based resources.

Quadrilaterals:

During the computer lab session, the students were able to independently identify relationships between the sides and the angles and summarize the properties of various quadrilaterals by manipulating the figures in the web-based resource. However, in the test, they struggled when they were expected to determine the measure of an angle or length of a side in a trapezium.

Nets:

In a quiz conducted just after this session, most students were able to match a polyhedron and the corresponding net. However, in the final test conducted on the third day, the students did not fare as well as anticipated. Students could identify a pyramid but encountered difficulty in identifying a tetrahedron. More students had success with a simple application of Euler’s formula.

Grade VII

Fractions:

As there was some prior exposure to this topic, it helped students in understanding the concepts. Most students were able to answer the related questions in the test.

Linear equations in one variable:

Students did not appreciate the analogy of a balance. While this was not investigated at depth, it is attributed to a great extent to students carrying out mathematical operations mechanically, without trying to understand the reason why a certain term was added to or subtracted from both sides of an equation. Observations, Feedback & Discussion

Access to a computer is fairly easy with most students already having a computer at home. All students were quite familiar with the use of the computer and relevant software.

Student feedback

Grade VIII

“The websites were fun and creative as we ourselves could solve our doubts. Before I found all these shapes to be complicated and boring, but these seem to be very easy now “Student A

“I liked the opening and closing up of the nets and playing with the figures” Student B

“We could really see the object and play around with it…This has been one of the most interesting class I had even though I do not like maths” Student C

“It was good but quite confusing” Student B

“I haven’t understood everything yet but I liked the software a lot” Student C

Grade VII

“We learnt something completely unknown to us and I found it very interesting” Student D

“At first I didn’t understand how to do the algebra but when I understood about it I felt very happy” Student E

“The class was interesting but I did not understand algebraic expression” Student F

The motivation levels were high and there was substantial participation in the class across both the grades. All the students enjoyed learning mathematics using the web based resources. They enjoyed self discovery by manipulation of sliders, interaction with figures representing fractions, quadrilaterals, nets and straight lines. The feedback at the end of everyday indicated that the students clearly enjoyed the sessions and found them interesting. They wanted their future lessons to incorporate the web-based resources. Students were eager to note down the links so that they could practice at home as well.

About half the students felt they now liked learning mathematics more. Some students were keen that the software provides them with feedback when a particular input/ action is wrong. These observations are consistent with those reported elsewhere (Hannafin, Burruss and Little, 2001).

Students reported difficulty in understanding linear equations, evident from the quotes above.

Student pairing

Student pairing does seem to have a powerful role to play. In pairs with students of differing abilities, the weaker ones also demonstrated increased interest and participation. In pairs, where both the students were academically strong, they explored the resources provided to a greater extent and asked more questions. Weak students were not paired together.

The students expressed a need to record on paper what was learnt using the computer as the on- screen manipulations are not accessible in preparation for a written test.

To summarize the differences observed between students, they have been grouped into: “Strong” and “Challenged” based on their past performance in examinations and meaningful classroom participation. These are not meant to reflect their intelligence or any other such measure. “Strong” implies better academic performance and constructive participation in the classroom while “Challenged” implies relatively weaker academic performance.

Strong students	Challenged students
High motivation levels through all sessions and desire to use computers more in their learning	High motivation levels initially but lost interest if unable to cope, leading to seeking distractions- clicking on irrelevant icons, changing screen magnifications, going to other sites etc.
High interest levels in mathematics continue to hold	Significantly increased interest expressed in learning mathematics
Seek independence requesting that the computer program prompt them if a suggested solution is wrong	Expressed need for guidance, simpler visual representation and more preparation in terms of raising the level of prior knowledge.
Prior mathematical techniques used sometimes posed a challenge. e.g. mechanically transferring terms from left hand side to the right hand side by changing the sign in a balanced equation as against adding/ subtracting the same term from both sides	In simplifying a balanced equation (to find the value of a variable), identifying which term needs to be added/ subtracted was not easy
Had used the computer as a tool for learning and doing school projects.	Comfortable in using a computer but had never used it as a tool for learning

Concerns such as those expressed by prospective teachers in Hazzan (2002/2003) become evident. Representing mathematical ideas on a computer did help visualization and stimulate thinking but inadequate preparation on manipulation of algebraic expressions posed difficulties.

Conclusion

• Visual representation and interactivity does enhance student learning and motivation.
• It is possible that the stronger students may gain more in the short term. Given the high motivation levels, over a longer period of time, it is anticipated to be beneficial to all the students. To ensure that this happens, student pairing needs to have a strategic approach. The suggested approach is to group together students of different abilities and shuffle partners periodically, monitoring the groups to ensure that they remain cohesive.
• It is possible to use freely available/ inexpensive web based resources and applications for the teaching of mathematics. However, a limitation of using these freely available resources is that it is not be possible to have a consistent pedagogical approach and on- screen environment, given that sources would be different.
• An instructivist learning environment as advocated by Hannafin et al. (2001) would work well: Accommodating learners’ needs and prior knowledge but limiting the amount of content available. Wherever possible, the on-screen information should be limited to what is needed for a student at a particular level- if fractions to be taught are limited to denominator less than 20, the teacher must have the option to restrict the application accordingly.
• Worksheets need to be devised based on the activities done using the computer so that students have a record for revision and can also share it with parents at home.
• The activities need to well planned and integrated into the classroom routines. Introduction to the topic, preparing students for what to expect is important before the computers are switched on. Students must be asked to switch the monitors off when their attention needs to be diverted to the whiteboard.
• Given that the students in Grade VIII and VII consistently expressed their discomfort with Algebra, it would be advisable to adopt an embodied approach with visualization as recommended by Tall (2009), to ensure that students are comfortable with Algebra at higher levels.

Appendix: List of resources used

Grade VII

Lesson 1: Linear equations

http://nlvm.usu.edu/en/nav/frames_asid_324_g_3_t_2.html?open=instructions&hidepanel=true&from=vlibrary.html

Lesson 2: Fractions

http://nlvm.usu.edu/en/nav/frames_asid_102_g_1_t_1.html?from=topic_t_1.html

http://illuminations.nctm.org/ActivityDetail.aspx?ID=45

http://nlvm.usu.edu/en/nav/frames_asid_159_g_2_t_1.html

Grade VIII

Lesson 1a: Quadrilaterals

http: / /mste.illinois.edu/m2t2/geometry/quads.html

Lesson 1b: Nets

http://www.mathsnet.net/geometry/solid/nets.html

Lesson 2: Linear Equations

http://www.waldomaths.com/Linear2NLW.jsp

http://www.bbc.co.uk/schools/gcsebitesize/maths/algebra/graphsact.shtml

Lesson 3: TIME DISTANCE

http://mathdemos.gcsu.edu/mathdemos/jogger/jogger.html

Appendix 2: References

Hannafin, R. D., Burruss, J. D., & Little, C. (2001). Learning With Dynamic Geometry Programs: Perspectives of Teachers and Learners. The Journal of Educational Research, 94(3), 132-144.

Hazzan, O. (2002/2003). Prospective High School Mathematics Teachers’ Attitudes toward Integrating Computers in Their Future Teaching. Journal of Research on Technology in Education, 35(2), 213-225.

Tall, D. (2003). Using Technology to Support an Embodied Approach to Learning Concepts in Mathematics. In L. M. Carvalho & L. C. Guimarães (Eds.), História e Tecnologia no Ensino da Matemática (Vol. 1, pp. 1-28). Rio de Janeiro, Brasil. Retrieved 31-Aug-2009 from http://www.warwick.ac.uk/staff/David.Tall/pdfs/dot2003a-rio-plenary.pdf

Authentic Assessment of Marketing Projects

Background
For the assessment tool I have outlined, I have considered students in India undertaking a Post Graduate Diploma in Management- which is a two years- full-time course, each year comprising 3 terms. Such course is recognized by AICTE (All India Council for Technical Education)
The key motivation for students in undertaking this course is to acquire a diploma in business, preparing themselves for entrepreneurship or employment opportunities in Indian and multinational companies.

Assessment instrument
Rather than a single project for the entire term, 2 or 3 group projects, each with increasing levels of complexity, is more appropriate for project based learning and assessment.

Group Project 1
A group of students to identify a brand they will represent within a given product category and build a marketing strategy for it. Deliverables include a written report and a presentation for the group, an individual critical reflection and peer assessment. (Weight: 40%)

Group Project 2
A group of students to select a marketing problem from the options provided. Deliverables include a written report and a presentation for the group, an individual critical reflection and peer assessment. (Weight: 60%)

Instructions for Students

Each student will submit two projects as a member of a randomly assigned group.

1. Each project will include

• Submission of written report not exceeding 3000 words excluding appendices such as tables or charts.
• A 25 minute presentation to peers and the examiner. Presentation should be about 20-25 slides to ensure completion in the time allotted.
• The report and the presentation must clearly identify the students who contributed to that project on the cover page
• Presentation will be followed by a Q&A

Submissions are to be made using a web-based system/ email

2. After the completion of the project and presentation, each student must submit one page critical reflection that must:

• Relate theoretical concepts covered in the classroom to applications on the project
• Interpret the experiences associated with working in the group: how challenges were met and problems were resolved
• How any of their assumptions have changed

3. Students must also submit self and peer assessment using a rubric that is made available to them, identifying their own contribution to the project and that of their team members.

4. As there is substantial weightage for the group projects, success is a function of teamwork and cooperation.

5. Timeline: An extension of not more than 2 days will be allowed to the groups to submit the project reports.

6. Access to software: Students are free to use software of their choice for spreadsheets, word processing and presentation so long as the files are compatible with Microsoft Office. If software is required for data analysis, students may access SPSS software installed on the computers in the college laboratory.

7. Internet access from college premises is permitted on all days including Sundays and public holidays from 7:00 am to 9:00 pm.

8. Students are not expected to use any external resources to conduct fieldwork or provide incentives to participants in any research. In case their project demands telephone interviews, visits to business locations or retail stores, students are expected to conduct it personally and at their own expense.

9. Students are free to raise questions pertaining to their projects in the classroom or via a discussion forum to ensure transparency and commonly asked questions can be answered efficiently.

10. Plagiarism of any kind will be strictly dealt with. Disciplinary action may be taken at anytime even after the results have been reported or the degree has been awarded.

Instructions to the examiner

• Projects must be assigned to groups of 4/5 students each.
• Projects must be authentic, reflecting issues that practitioners of marketing face in the current environment
• The projects must commence after the key marketing concepts are discussed in the classroom.
• Each member of the team must be involved in either making the presentation or fielding the questions from the examiner. The examiner will randomly assign 2 group members to do the presentation while the rest will respond to the questions at the end of the presentation.
• All members of the group will receive the same number of marks for the presentation and the report. However, the overall grading will be influenced by peer assessment and critical reflection.
• Other students attending the presentation may ask questions but these have to be routed through and moderated by the examiner who must ensure that questions asked are appropriate and relevant.
• Examiner must look for consistency in the peer assessments done by group members. Any doubts on individual contribution should be clarified prior to assessment.
• Samples of projects and critical reflection are to be made available to the students.
• The assessment rubrics are to be shared with the students at the beginning of the term.
• If a report is suspected not to be original work and/ or references to content that is reproduced from elsewhere are missing, it must be checked for plagiarism. Marks may be deducted if a match of 25-40% is identified. If it exceeds 40%, the group receives no marks for the report and the incident is reported in the student records.

Assessment Feedback
Students must be given feedback on

• Theoretical concepts, analytical techniques not taken into account- if any
• Depth of research conducted/ needs identified to understand the problem
• Feasibility of implementation of suggested solutions
• Improvements that can be made to the report writing, presentation skills and managing the Q&A sessions
• Collaborative efforts of the group

Project Assessment Rubric

http://www.scribd.com/doc/17108622/Assessment-of-Marketing-Projects-Rubric-Milind-Sattur

Educational justification for authentic assessment

There is not much focus on formative assessment because of lack of time and continuity as the various subjects are taught by specialists and the contact may not extend beyond a trimester.

To be adequate, comprehensive and authentic, assessment should consist of tasks which are representative of the ‘knowledge, skills and strategies needed for the activity or domain being tested’ (Fredericksen & Collins 1989) cited in Pitman (1999). The holistic, competency based assessment is designed keeping in mind the learning objectives and expected learning outcomes.

Learning Objectives
The focus is on acquiring knowledge about the different components of marketing such as product offering and life cycle, communication, pricing and distribution in the context of current business environment in India with emphasis on case studies and projects.

Expected learning outcomes
On completion of this course, students will have an understanding of the marketing management process. Students should be able to personally construct and evaluate a marketing plan created by others, ascertain reliability of data sources used for consumer understanding and market intelligence, justify the plan suggested and define criteria for measurement of success of the proposed marketing strategy.

Ensuring authenticity
As we are dealing with students who have already completed 3 or 4 years of University level education and possibly have some work experience, the focus is on providing learning outcomes that are relevant to employability and entrepreneurship. Brogan (2006) highlights how project based learning in groups is appropriate from the employability perspective.

To ensure a high degree of authenticity, project based learning (PBL) and assessment is recommended for a substantial weight of assessments as it

• Allows students to create a product and demonstrate learning
• Reflects real work situation where
• They would be expected to do both-collaborate and demonstrate individual competence
• Present their work to others to seek support or approval
• Is relevant in an adult learning
• Focuses on improving problem solving capabilities

Authentic assessment has a positive impact on student motivation to learn and students are able to see its relevance to their future professional lives. (Gulikers, Bastiaens & Kirschner, 2004).

Most teaching staff at such institutions being from the marketing industry, are well-equipped to construct and scaffold real-life projects, provide real life examples in classroom discussions on how marketing plans were created and implemented, what were the hurdles- in terms of information and resources and how those were resolved and the end results. Effectively- projects become central to both learning and assessment and students are able to connect between theory and practice. On some occasions, there are opportunities to even implement the projects in real-life at least in part, if not in entirety.

The group formation is also randomized as professional life does not always offer you opportunities to work with like-minded people. This suggestion of randomizing groups comes from professionals who experienced that inability to work with people of different attitudes and values can have a negative impact on the success of a project.

The two projects are sequenced such that the first one is well-structured and is limited to one product category. While this limits the choice, students will be able to appreciate how they view the brand they represent and how their competition views them. The second project is somewhat ill-structured and also allows some flexibility in terms of choice so that students can choose a project that fits best with their interests or employment objectives.

Grading with rubrics within a marketing course helps students clearly identify the different aspects of project work and is what is expected of them. (Amantea, 2004)

Critical reflection

According to Catterall, Maclaran, and Stevens (2002), in group projects, students need to reflect upon the impact of interpersonal factors, differing agendas of team members, compromises, leadership battles, hurdles encountered and resolved. They also need to identify themselves in a wider economic, political, cultural and social environment.

The rationale for inclusion of critical reflection to the project work is four-fold:

• Transfer of learning from the classroom to the real world
• Bring students to realization that decision making and events occur in a socio-political context and that individual perspectives can be very different
• Making students aware of why they perceive, think, feel or act in the way they do
• Assess their own contribution to the group projects

Validity

Face validity
The project based learning and assessment has face validity because the objective is to assess students on their ability to apply concepts covered in the classroom on real-life marketing problems. Further, in their professional lives, the management students would be expected to collaborate and work with others. The assessment is structured using a clearly defined project brief, timeline and assessment rubric, yet giving some autonomy to students.

Content validity
The projects are defined by experienced marketing professionals, ensuring that the content is relevant and current. They cover the expected learning outcomes as students will be tested on their ability to create and evaluate marketing strategy and their ability to communicate the same via a report and presentation, as they would, in a working environment.

Construct validity
In an authentic assessment construct validity is very important. That is, the assessment task must assess problems encountered in the real world using criteria that would also be encountered in real-life (Gulikers, Bastiaens & Kirschner, 2004).

The rubric for project assessment helps assess the students not only on the final product but also the process– the research needs identified, data gathered, understanding of the consumer and the competitive environment demonstrated.

Predictive validity
As the projects reflect real business problems, the assessment does reflect future performance in a working environment (Moskal and Leydens, 2000).

"Students taught with a more progressive, open, project-based model developed more flexible and useful forms of knowledge and were able to use this knowledge in a range of settings." (Boaler, 1998a) cited in Thomas (2000)

The project rubric also covers graduate attributes such as presentation skills and managing questions.

Reliability
The projects are assessed using a rubric that takes into account the report, presentation and the critical reflection. Under each of these areas are specific criteria clearly defined to make assessment as reliable as possible and minimize subjectivity. While subjectivity cannot be fully eliminated, moderation is recommended for the project reports to ensure inter-examiner consistency.

The rubric may also be pre-tested for consistency with 3-5 examiners who are given the same project reports and rubric before the actual assessment is commenced.

The student/ group performance across the two projects must be compared for consistency and to see if differences can be explained. In an ideal situation, we should expect an improvement in performance from the first to the second project.

Fairness
Adequate choice is given for the second project so that groups can identify a context that is of interest to them. The rubric is shared with the students at the beginning of the term.

Allowing students to make use of open source/ free software and access to resources such as internet and the library ensures fairness.

By assigning a minimum score for critical reflection and peer assessment, we can ensure that a student who has contributed more to the project or demonstrates learning/ application of theory to practice fares better on the assessment. It is important to allow for individual’s accountability to the group (Newhouse-Maiden and de Jong, 2004)

The projects are checked for plagiarism minimizing instances of cheating.

Reporting and Feedback

Meaningful feedback to the students at the end of every assessment is vital as the goal is not just completion of the project but to ensure that relevant learning happens along the way. The project rubric allows for identification of specific issues / subject areas to provide feedback. E.g.” Student demonstrates a deep understanding of the competitive environment and can recommend appropriate marketing strategy. He/ she needs to improve on managing questions from audience. He/ She must focus on improving understanding of pricing strategy”

The weight for different components of assessment is increased gradually providing the learner an opportunity to reflect and better their performance.

The assessment will also be meaningful to potential employers who may choose to review the projects.

Sample Project briefs

Marketing Strategy Project Brief: Project 1 (40%)
Choose a brand in the detergent category from the list below

• Ariel
• Tide
• Surf Excel
• Nirma
• Henko
• Mr.White
• Bang
• Vanish
• Wheel
• Rin

No two groups can choose the same brand.

• Demonstrate an understanding of the brand and it’s competitive environment
• What do you think are the key issues for the brand today, at what stage is it in the product life cycle
• Develop a marketing strategy for 2010, highlighting the changes that you would do to the marketing mix- if any (Product offering, Distribution, Pricing and Advertising & Promotion)

The group must submit the project report to within 15 working days followed by a presentation in the subsequent week. After the completion of the project and presentation, each student must submit one page critical reflection that covers:

• A brief description of the key tasks he/ she performed on the project
• Relate theoretical concepts covered in the classroom/ study material to applications on the project
• Interpret the experiences associated with working in the group: how challenges were met and problems were resolved
• Personal goals for improvement – if any

Sample Project Brief: Project 2 (60%)
Select one of the options below.

Option 1
Identify a product or a service category that currently does not exist in India. Develop a strategy to launch it in India.

• Describe the product or service category
• Demonstrate the relevance of the product or the service to the Indian market
• Identify and describe the target segment
• Strategy for launch of the product including advertising and promotion
• Sales strategy
• Forecast of sales/ subscriptions for the next 3 years and provide rationale for the forecast
• Identify metrics that will determine a successful launch

Option 2
You are employed with a leading telecom operator and seek to increase revenue by offering value added services targeted to rural India.

• Describe the service offering
• Describe the target segment
• Demonstrate the relevance of the product or the service to the rural Indian market
• Strategy for launch of the product including advertising and promotion
• Strategy to build the subscriber base
• Forecast of subscriptions for the next 3 years and provide rationale for the forecast
• Identify metrics that will determine a successful launch

Option 3
You are responsible for marketing a premium airline brand. With the economic downturn, your company decides to launch a budget airline with fares that are 40% lower than that of the existing brand. You are now responsible for marketing both the brands but have to ensure that cannibalization of the premium airline brand is to the minimum. How would you differentiate between the two in terms of:

• Target segments
• Service offerings
• Pricing
• Advertising and promotion strategy
• Identify metrics that will determine the success of the marketing plan.

The group must submit the project report to within 20 working days followed by a presentation in the subsequent week. After the completion of the project and presentation, each student must submit one page critical reflection that covers:

• A brief description of the key tasks he/ she performed on the project
• Relate theoretical concepts covered in the classroom/ study material to applications on the project
• Interpret the experiences associated with working in the group: how challenges were met and problems were resolved
• Personal goals for improvement – if any

References

Amantea, C. A. (2004). Using Rubrics To Create And Evaluate Student Projects In A Marketing Course. Journal of College Teaching & Learning, 1(4), 23-28.

Brogan, M. (2006). What you do first is get them into groups": project-based learning and teaching of employability skills. Fine Print, 29(2), 11-16.

Catterall, M., Maclaran, P., & Stevens, L. (2002). Critical reflection in the marketing curriculum. Journal of Marketing Education, 24(3), 184-192.

Forehand, M. (2005). Bloom's taxonomy: Original and revised. In M. Orey (Ed.), Emerging perspectives on learning, teaching and technology. Retrieved 27 May 2009, from http://projects.coe.uga.edu/epltt/

Gulikers, J. T. M., Bastiaens, T. J., & Kirschner, P. A. (2004, June 23-25). Perceptions of authentic assessment- Five dimensions of authenticity. Paper presented at the Second biannual joint Northumbria/EARLI SIG assessment, Bergen.

Kotler, P. (1999). Marketing Management – The Millennium Edition (10th Edition ed.): Prentice Hall of India Private Limited.

Newhouse-Maiden, L., & Jong, T. d. (2004). Assessment for learning: Some insights from collaboratively constructing a rubric with post graduate education students Paper presented at the 13th Annual Teaching Learning Forum, 9-10 Februrary 2004, Perth: Murdoch University.

Pitman, J. A., O’Brien, J. E., & McCollow, J. E. (1999, May 1999). High-Quality Assessment: We are what we believe and do. Paper presented at the IAEA Conference, Bled, Slovenia.

Thomas, J. W. (2000). A review of research on Project-Based Learning. Retrieved May 10, 2009, from http://www.bie.org/index.php/site/RE/pbl_research/29

Thomas, R. E. (1997). Problem-based learning: measurable outcomes. Medical Education, 31, 320-329.

Williams, J. B., & Wong, A. (2009). The efficacy of final examinations: A comparative study of closed-book, invigilated exams and open-book, open-web exams. British Journal of Educational Technology, 40(2), 227–236.

Wolf, P. (n.d.). Transformational learning and critical reflection. Retrieved 17 May, 2009, from http://www.wlu.ca/documents/20396/Critical_reflection_-_handout.pdf