Transforming Exams
A scalable examination platform for BYOD invigilated assessment
A scalable examination platform for BYOD invigilated assessment
OLT Innovation and Development grant awarded: ID15-4747 'Transforming exams across Australia: processes and platform for e-exams in high stakes, supervised environments' 2016-2018.
Further details will be posted as presentations are confirmed.
Note: Free during the funded project period to Dec 2018 - a charge may be incurred after this date.
Expressions of interest in running an e-Exam trial for
a) e-exams based on word processor documents that run without a network. A selection of 'e-tools of the trade' software applications (e.g. spreadsheet, programming, multimedia, maths apps) are available. See the flyer outlining document centric exam trials with a full roadmap on the reverse.
b) e-exams that extend the capability using a new 'robust' Moodle approach that can survive network outages through to completion. Moodle based e-Exams can also include 'e-tools of the trade' applications. See the flyer outlining robust Moodle based exam trials.
Please get in contact if you are interested in running an e-Exam trial at your institution or organisation.
Publications and presentations relating to Australian Government Department of Education (DET), Innovation and Development project phase of work (Project ID15-4747).
Final project report
Documents are available from the DET Learning and Teaching repository.
Overview
e-Exam pedagogical implementation roadmap [PDF] - A one page strategy outline for scaling up pedagogically rich e-Exams at an Institution.
e-Exam student user walk through [PDF] - A look at how a student undertakes an online exam. This uses a client/server approach with Safe Exam Browser onboard the client e-Exam USB connecting to a robust Moodle server. Process available as of 2018.
Refereed
Hillier M., Kumar N. & Wijenayake N. (2020) e-Examinations: The Impact of Technology Problems on Student Experience. In: Brinda T., Passey D., Keane T. (eds) Empowering Teaching for Digital Equity and Agency. OCCE 2020. IFIP Advances in Information and Communication Technology, vol 595. Springer, Cham. DOI 10.1007/978-3-030-59847-1_4
Hillier M. & Lyon N. (2019) Student Experiences with a Bring Your Own Laptop e-Exam System in Pre-university College. In: Passey D., Bottino R., Lewin C., Sanchez E. (eds) Empowering Learners for Life in the Digital Age. OCCE 2018. IFIP Advances in Information and Communication Technology, vol 524, pp. 253-263. Springer, Cham. DOI: 10.1007/978-3-030-23513-0_25
Hillier M. & Lyon N. (2019) Writing e-Exams in Pre-University College. In: Passey D., Bottino R., Lewin C., Sanchez E. (eds) Empowering Learners for Life in the Digital Age. OCCE 2018. IFIP Advances in Information and Communication Technology, vol 524, pp. 264-274. Springer, Cham. DOI: 10.1007/978-3-030-23513-0_26
Fluck, A. E. (2019). An international review of eExam technologies and impact. Computers & Education, 132, 1–15. DOI: 10.1016/j.compedu.2018.12.008
Hillier, M., Grant, S., & Coleman, M. (2018) Towards authentic e-Exams at scale: robust networked Moodle (full refereed conference paper), Australasian Society for Computers in Learning in Tertiary Education conference, 25-28 Nov, Geelong, Australia, pp. 133-143 [download full paper]
Hillier, M. & Grant, S. (2018) Do-it-yourself e-Exams (full refereed conference paper), Australasian Society for Computers in Learning in Tertiary Education conference, 25-28 Nov, Geelong, Australia pp. 123-132 [download full paper]
Cowling, M., Hillier, M. & Birt, J. (2018) Integrating mixed reality spatial learning analytics into secure electronic exams (refereed conference paper), Australasian Society for Computers in Learning in Tertiary Education conference, 25-28 Nov, Geelong, Australia, pp. 332-336 [download concise paper]
Hillier, M. & Fluck, A. (2018) "Robust networked e-Exams with Moodle" (abstract), Presented at the Open Conference on Computers in Education (IFIP TC3), Linz, Austria, 25-28 June. [download pre-review version of abstract]
Hillier, M. & Lyon, N. (2018) "Student experiences with a bring your own laptop e-Exam system in pre-university college" (refereed conference paper), Presented at the Open Conference on Computers in Education (IFIP TC3), Linz, Austria, 25-28 June. [download pre-review version of full paper] [download PPT]
Hillier, M. & Lyon, N. (2018) "Writing e-Exams in pre-university college" (refereed conference paper), Presented at the Open Conference on Computers in Education (IFIP TC3), Linz, Austria, 25-28 June. [download pre-review version of full paper] [download PPT]
Hillier, M. (2018) "Bridging the digital divide with off-line e-learning", Distance Education, 39(1), pp. 110-121. DOI: 10.1080/01587919.2017.1418627
Pagram, J., Cooper, M., Jin, H. & Campbell, A. (2018), Tales from the Exam Room: Trialing an E-Exam System for Computer Education and Design and Technology Students. Education Sciences, 8(4), article 188 DOI: 10.3390/educsci8040188.
Fluck, A., Adebayo, O. S. & Abdulhamid, S. M. (2017) Secure e-examination systems compared: Case studies from two countries. Journal of Information Technology Education: Innovations in Practice, 16, 107-125 DOI: 10.28945/3705
Hillier, M. & Fluck, A. (2017). Transforming Exams - How IT works for BYOD e-Exams. in H. Partridge, K. Davis, & J. Thomas. (Eds.), Me, Us, IT! Proceedings ASCILITE 2017: 34th International Conference on Innovation, Practice and Research in the Use of Educational Technologies in Tertiary Education (pp. 100-105). Toowoomba, 4-6 December. [download full text]
Fluck, A. & Hillier, M. (2017) "eExams: Strength in diversity", Presented at the World Conference for Computers in Education (IFIP TC3), Dublin, Ireland 3-6 July. [download pre-review version of full text]
Fluck, A., Pálsson, H., Coleman, M., Hillier, M., Schneider, D., Frankl, G., & Uolia, K. (2017). "eExam symposium: design decisions and implementation experience". Presented at the World Conference for Computers in Education (IFIP TC3), Dublin, Ireland 3-6 July. [download pre-review version of full text]
Hillier, M. (2017) "Bridging the digital divide with an off-line e-learning and e-assessment platform", Presented at 'Expanding Horizons in Open & Distance Learning', Open and Distance Learning Association of Australia Bi-annual conference, 5-7 February, Melbourne, Australia [download full text][download slides PDF].[中文:全文下载] "为离线电子学习和电子评估平台的数字鸿沟搭建桥梁".
Fluck, A. & Hillier, M. (2016) "Innovative assessment with eExams", Australian Council for Computers in Education Conference, Brisbane, 29 Sep to 2 Oct. [download full text]
Hillier, M. (2015) "To type or handwrite: student's experience across six e-Exam trials", ASCILITE Conference, Perth, 29 Nov - 2 Dec. [Full paper], [Slides with 2015 results included].[Español: Descarga de texto completo] "Teclear o escribir a mano: la experiencia de los estudiantes a través de seis ensayos acerca del e-Exam". [中文:全文下载] "打字或手写:学生通过六个实验计算机化考试的体验".
Hillier, M. & Fluck, A. (2015) "A pedagogical end game for exams: a look 10 years into the future of high stakes assessment", ASCILITE Conference, Perth, 29 Nov - 2 Dec. [short paper], [Slides]. [Español: Descarga de texto completo] "Cambio de juego pedagógico para los exámenes: una mirada diez años hacia el futuro de las evaluaciones de alto impacto". [中文:全文下载] "如何考试成为教学的终极角逐：展望今后十年高利害关系评估测试发展". [日本語：全文ダウンロード] "テストの教育的見地から見た最終目標:ハイステークス・アセスメントの10年後の将来".
Gregory, S., Jacka, L., Hillier, M. & Grant S. (2015) "Using virtual worlds in rural and regional educational institutions" Australian and International Journal of Rural Education, Vol. 25 (2), pp. 73-90. (This paper explores the use of the e-Exam System as the basis for an offline e-learning delivery platform).
Hillier, M. (2015) "Filling the learning analytics gap for high-stakes exams", HERDSA Conference, Melbourne, 6-9 July 2015. [Abstract and Poster].
Fluck, A. & Hillier, M. (2015) "eExams for curriculum transformation", HERDSA Conference, Melbourne, 6-9 July 2015. [Abstract for Roundtable].
Hillier, M. (2015) "e-Exams with student owned devices: Student voices", International Mobile Learning Festival 2015, Hong Kong, 22-23 May 2015 [download full paper] (refereed abstract). [中文:全文下载] "学生自带设备参加电脑考试——学生的想法".
Hillier, M. & Fluck, A. (2015) "Transforming Exams into e-Exams", Proceedings of the International Conference on Assessment for Learning in Higher Education, 14-15 May, Hong Kong. [refereed abstract] [view slides]
Hillier, M. (2014) "The Very Idea of e-Exams: Student (Pre)conceptions" [Full paper], Proceedings of the 31st Australasian Society for Computers in Learning in Tertiary Education, 24-26 November, Dunedin New Zealand. [pre-press] [view slides]
Hillier, M. (2014) "e-Exams at University of Queensland: Trial Results 2014" [Poster], Proceedings of the 31st Australasian Society for Computers in Learning in Tertiary Education, 24-26 November, Dunedin New Zealand. [Poster A4]
Gregory, S., Jacka, L., Hillier, M. & Grant S. (2014) "Using virtual worlds in rural and regional educational institutions" in 3 Decades On: Reflections Upon Rural Education - Achievements, Challenges and Future Directions, Proceedings of the 30th Society for the Provision of Education in Rural Australia Conference, 1-3 October, Armidale, NSW, Australia.
Fluck, A. & Hillier, M. (2014) "eExams Transforming Curriculum", 26th Australian Computers in Education Conference, 30 September - 3 October, Adelaide. [session synopsis]
Hillier, M. & Fluck, A. (2014) "e-Exam System v5: A Secure, Scalable, Ethical Approach to High Stakes e-Assessment" [Poster], Higher Education Research and Development Society of Australasia (HERDSA) Conference 7-10 July, Hong Kong. [read abstract] [view poster]
Hillier, M. & Tran, L. (2014) "Student Perceptions of e-Exams: Hopes and Fears" [Special Presentation], Higher Education Research and Development Society of Australasia (HERDSA) Conference 7-10 July, Hong Kong. [read abstract] [view slides]
Hillier, M. & Fluck, A. (2013) "Arguing again for e-Exams for high examinations" Proceedings of the 30th Australasian Society for Computers in Learning in Tertiary Education, 1-4 December, Macquarie University Sydney, Australia. [view slides]
Hosted Events & Edited Proceedings
Hillier, M. (ed) (2018) "e-Exams Symposium". Proceedings, 24 Nov, Melbourne, Australia. Session recordings of all sessions
Hillier, M. (ed) (2014) "Final Answer: The Question of Summative eAssessment. Proceedings of eAssessment Scotland 2014 Online Conference", 8-19 September, Dundee, Scotland, UK & Brisbane, Australia. [Session synopses and recordings of all 8 sessions]
Presentations & Meetings
Tran, P. & Hillier, M. (2019) Making a case for online exams: Efficiency , integrity and insight, EdTech Posium, 28-39 Oct, ANU, Canberra, Australia [Download presentation slides PDF].
Hillier, M. (2019) A holistic digital assessment strategy: anywhere, anytime, anyhow! 21 Oct, Swinburne University, Melbourne, Australia.
Hillier, M., Kumar, N., & Wijenayake, N. (2019) Online Exams with Safe Exam Browser, Inspired Learning Summit, 18 Oct, UNSW Sydney, Australia [Download poster PNG].
Hillier, M. & Fluck A. (2019) International Conference on e-Learning and Innovative Pedagogies Conference, 2-3 May, Hobart, Australia.
Hillier, M. (2018), Robust, authentic e-Exams in STEM (presentation), FSET Learning Innovation Showcase, 6 Dec, Swinburne University, Melbourne, Australia.
Hillier, M. (2018), Striving for authentic assessment in supervised e-Exams (presentation), Tertiary Education and Quality Standards Agency Conference & Higher Education Quality Forum, 30 Nov, Melbourne, Australia [download slides PDF].
Hillier, M., Coleman, M. & Grant, S. (2018) Design and evaluation of robust online e-Exams with Moodle, EdTechPosium, 29-30 Oct, Canberra, Australia
Hillier, M., McGrath, D., Hubbell, A., Simbag, V., Cowling, M. & Howah, K. e-Exams: Queensland experiences and future directions (Poster) (2018) Queensland University Educators Showcase, 28 Sep, University of Sunshine Coast, Australia [download poster]
Geer, R., White, B., Hillier, M. & Fluck, A. (2018) Authentic supervised e-Exams: National project, SA experience (refereed abstract), Higher Education Research Group of Adelaide conference, 25 Sep, p.13 [download conference abstracts PDF].
Hillier, M., Coleman, M. & Grant, S. (2018) Look Ma, No Network! Robust e-Exams with Moodle, Moodle MOOT Australia, 24-26 Sep, Brisbane, Australia
Hillier, M. (2018), Robust online authentic e-Exams, Examinations Network Conference, 29-30 Aug, Adelaide, Australia
Hillier, M. & Grant, S. (2018), Using spreadsheets for e-Exam delivery (refereed abstract). Presented at Higher Education Research and Development Society of Australasia (HERDSA) conference, 2-5 July, Adelaide, Australia. [download abstract]
Hillier, M., Fluck, A., Cowling, M., Howah, K., Bower, M., Pagram, J., Meacheam, D., Geer, R., White, B., & Grant, S. (2018), "Diversifying modalities for supervised e-Exams" (refereed abstract to poster). Presented at Higher Education Research and Development Society of Australasia (HERDSA) conference, 2-5 July, Adelaide, Australia. [download abstract]
Hillier, M. & Fluck, A. (2018), "Doing authentic supervised e-Exams" (refereed abstract to poster). Presented at Higher Education Research and Development Society of Australasia (HERDSA) conference, 2-5 July, Adelaide, Australia. [download abstract]
Fluck, A. & Hillier, M. (2018) "e-Exams for Maths and Computing" (private briefing/workshop), School of Computing and Mathematics Colloquium, Charles Sturt University, Canberra, 1 Feb.
Hillier, M. (2017) "A conversation about... options for authentic assessment in computerised exams" (invited presentation), Queensland University of Technology Gardens Point campus, Brisbane, 8 December.
Hillier, M. (2017) "Authentic assessment in computerised exams" (invited presentation), Central Queensland University, Brisbane, 8 December.
Hillier, M., Fluck, A. & Coleman, M. (2017) "Transforming Exams – hands on with the technology" (post-conference workshop - focusing on assessment design and DIY logistics), ASCILITE conference, Toowoomba, 7 December.
Hillier, M. et.al (2017) "Transforming Exams – stories from across Australia" (panel session) [slide PDF] ASCILITE conference, Toowoomba, 4-6 December.
Hillier, M. (2017) "Striving for authentic assessment in the exam room and beyond", (keynote presentation) EdTechPosium, Canberra, 30-31 October.
Hillier, M. & Fluck A. (2017) "Running Authentic e-Exams" (invited presentation/closed conference), Examinations Network conference, Hobart, 6-7 September.
Hillier, M., Fluck A & Coleman M. (2017) "Running Authentic e-Exams Workshop" (focusing on e-exam administration and logistics), Hobart, [9am to 12.30], 8 September.
Hillier, M. & Fluck, A. (2017) "The assessment triad and e-exams" (plenary presentation), Assessment integrity and the role of examinations, Melbourne, 22 Aug 2017 [abstract] [Presentation]
Fluck, A. (ed) (2017) "eExam symposium - design decisions and implementation experience" (symposium), World Conference for Computers in Education (IFIP TC3), Dublin, Ireland 2-6 July.
Hillier, M. & Coleman, M. (2017) "Transforming exams across Australia: Processes and platform for e-exams in high stakes, supervised environments" (presentation) in eExam Symposium, World Conference for Computers in Education (IFIP TC3), Dublin, Ireland July.
Hillier, M. (2017) "Wrangling e-Exams: authenticity, integrity, and scalability" (invited presentation), Assessment in Higher Education conference, Manchester, UK, 29 June.
Hillier, M. (2017) "Towards authentic e-Exams in Australian universities" (invited presentation), The Open University, Milton Keynes, UK, 26 June.
Hillier, M. (2017) "Authentic assessment on Exams using computers" (invited presentation), Warwick University, UK, 27 June.
Hillier, M. (2017) "Towards authentic computerised exams national project progress" (invited seminar), University of Queensland, 10 April.
Hillier, M. (2017) "e-Assessment for Digital Learning" (guest online lecture), University of South Australia, Adelaide, 11 May.
Hillier, M., Fluck, A. & Coleman, M. (2017) "Transforming Exams Across Australia - project partner workshop", Monash University, 17 February.
Hillier, M. (2016) "Authentic Assessment with e-Exams" (invited seminar), University of Adelaide, Adelaide, 1 Dec.
Hillier, M. & Fluck, A. (2016) "Exam Analytics: how authentic can we get" (Workshop) Australian Learning Analytics Summer Institute (ALASI) for Australasian Society for Computers in Learning in Tertiary Education (ASCILITE) Conference, Adelaide, 27 Nov.
Hillier, M. & Coleman, M. (2016) "OLT e-Exams: Authentic Assessment" (briefing/demo), Swinburne University of Technology, Hawthorn campus, 10 Oct.
Hillier, M. (2016) "Pushing the boundaries of authentic high-stakes assessment for the 21st century" (talk/briefing), University of New South Wales Sydney, 22 Sep. [abstract]
Hillier, M. & Fluck A. (2016) "Ramping up the authenticity of high-stakes supervised assessment for the 21st century" (workshop), Macquarie University, Sydney, 23 Sep. [abstract][slides PDF]
Hillier, M. & Fluck A. (2016) "An off-line e-assessment platform for Computer Science education" (poster), ACM International Computing Education Research (ICER) Conference, Melbourne, 9 to 11 Sep [Download A4 poster][Download lightening talk slides].
Hillier, M. (2016) "Transforming Exams Across Australia - Exams of 2030" (poster), Australian Government Office for Learning and Teaching Conference, Melbourne, 28 to 29 April [Download A4 poster].
Hillier, M. (2016) "e-Exam Trials at Monash" briefing, Faculty of Business & Economics Education Committee, Monash Collins Street campus, 24 March.
Hillier, M. (2016) "Exploring e-Exams Technology" (workshop), UWA Assessment Conference, University of Western Australia, Perth, 17 March [2:30pm-3:30pm].
Hillier, M. (2016) "Transforming Assessment and Exams with Technology" (invited keynote), UWA Assessment Conference, University of Western Australia, Perth, 17 March [11:15am-12pm].
Hillier, M. (2016) "Developing exams for the twenty first century" presentation, Australian Business Deans Council, Geelong 1 Feb.
Hillier, M. (2015) "e-Exams and assessment" briefing, University of Western Australia, Perth 3 Dec.
Hillier, M. (2015) "e-Exams: Developing exams for the twenty first century" presentation, Edith Cowan University, Perth 3 Dec.
Hillier, M. (2015) "Welcome to the Future (of Assessment for 21C Skills)", Faculty of Business & Economics, Innovation Incubator Residential Workshop, Cape Schanck Resort, 10 Nov.
Hillier, M. (2015) "e-Assessment and e-Exams" presentation, Auckland University of Technology, 7 September.
Hillier, M. (2015) "Transforming Exams Across Australia - National OLT project brief", Exams Network Conference, University of Auckland, 3 Sept.
Hillier, M. (2015) "About e-assessment and e-exams", Open Polytechnic of New Zealand, Lower Hutt, 31 Aug.
Hillier, M. (2015) "Filling the learning analytics gap for high-stakes exams", HERDSA Conference, Melbourne, 6-9 July 2015.
Fluck, A. and Hillier, M. (2015) "eExams for curriculum transformation" (Roundtable discussion), HERDSA Conference, Melbourne, 6-9 July 2015.
Hillier, M. (2015) "Transforming Assessment and Exams", Institute for Education, Tsinghua University, Beijing, 26 and 28 May 2015.
Hillier, M. (2015) "About e-Exams", Invited Seminar, Chinese University of Hong Kong, 20 May [view slides].
Hillier, M. (2015) "Transforming Exams with E-Exams", Course co-ordinators workshop, [via video conference], James Cook University, Cairns, 27 Jan 2015.
Hillier, M. (2014) "Transforming Exams: bring-your-own laptop to high-stakes e-exams", Staff seminar, University of Auckland, Auckland, NZ, 1PM 4 December 2014.
Hillier, M. (2014) "Transforming Exams: BYO laptops for e-Exams at University of Queensland", Staff seminar, University of Canterbury, Christchurch, NZ, 11AM 1 December 2014.
Hillier, M. (2014) "e-Exams", Private briefing, Christchurch Polytechnic Institute of Technology, Christchurch, NZ, 1PM 28 November 2014.
Hillier, M. (2014) "BYOD e-Exam Trials at UQ", UQ Teaching and Learning week, St Lucia campus, University of Queensland, 1.30PM 3 November.
Hillier, M. (2014) "Transforming Exams: Results from the 2014 BYOD e-Exam Trials at University of Queensland" [slides], Invited Presentation, Educational Innovation Through Technology conference, Brisbane, Australia, 28 Oct.
Fluck, A. (2014) "BYOD on-campus eExams at University of Tasmania (UTAS)" [invited presentation], Joint eAssessment Scotland and Transforming Assessment Online Conference, 8 September. [session synopsis and recording]
Hillier, M (2014) "UQ: BYOD for e-Exam Trial outcomes" [slides], Presentation (via online video conference), Exams Network Conference, University of New England, Armadale, Australia, 20 Aug.
Hillier, M. (2014) "e-Exams: UQ trial findings", Private briefing, Centre for Enhancement of Teaching and Learning, University of Hong Kong, Hong Kong, 4 July.
Hillier, M. (2014) "Transforming Assessment with e-Assessment", Seminar, Centre for Development of Teaching and Learning, National University of Singapore, 15 July.
Hillier, M. (2014) "e-Exams", Private briefing, Singapore Institute of Technology, Singapore, 16 July.
Hillier, M. (2014) "e-Exams", Private briefing, Ngee Ann Polytechnic, Singapore, 16 July.
Hillier, M. (2014) "UQ e-exams trial preliminary findings" [Slides], ASCILITE Live! webinar Series, National University of Singapore, 15 May
Hillier, M. (2014) "Transforming Assessment with e-assessment for e-exams" [Slides], Global Links Speaker Series, Bond University, Gold Coast, 4 April
Hillier, M. (2014) "e-Exams", Teaching Innovations Conference, University of Queensland, Gatton Campus, 29 Jan
Hillier, M. (2013) "Arguing for e-Exams" [Slides] Transforming Assessment in the Digital Era, Melbourne.
Hillier, M. (2013) "Exploring e-Exams for high stakes assessment" [Slides] Examinations Network Conference, Brisbane.
Presented at UQ T&L Week (Flipped Classroom Expo) - Monday 28 Oct 2013 10-11am, Learning Innovation Building 17 Level 2
Presented at UQ T&L Week (Science-EAIT T&L Showcase) Wednesday 30 Oct 2013, 10.30-12noon. 316/316A, Level 3, Advanced Engineering Building (49)
Other Documents
Hillier, M. (2014) "e-Exams: UQ Trial Outcomes S1 2014" A general information presentation with system overview and first phase trial findings.
Hillier, M. (2013) "Argument Map – of a 'wicked' problem!" diagram.
Case studies of e-Exams from Australian Universities during 2017-2018.
a) e-Exam Roadshow workshops held in most Australian cities between August and November 2018. [Now completed]
b) e-Exam Symposium (mini conference) Saturday 24 November 2018, in Melbourne, Australia. View proceedings and session recordings.
Fluck, A. (2013) 'e-Exams Transforming Curriculum' Transforming Assessment webinar, 13 March (a presentation relating to the predecessor project at UTAS)
e-Assessment Zotero library items tagged with 'e-exams'.
Related resources
Transforming Assessment - our sister site with e-Assessment examples.
Written Content (CC BY-NC-SA 3.0 AU). e-Exam logo all rights reserved. Software GPL | Site template based on 'justifiable' by FreeCSSTemplates.org.
