Executive summary
ICDL National Operators and ICDL Foundation carried out digital literacy studies in five countries in Europe (Austria, Denmark, Finland, Germany, and Switzerland) and two countries outside Europe (India and Singapore). The studies consisted of two key parts: self-assessment, and practical evaluation of digital skills. The findings revealed that people tend to overestimate their abilities and that significant digital skills gaps exist in all of the analysed countries. Moreover, young people have digital skills gaps that are just as wide as in the rest of society.
Research demonstrated that self-assessment is a poor measure of digital skills. Thus, any self-assessment tool, such as the Europass CV digital competence grid, should always be complemented by a diploma or a certificate as evidence of the claimed competence level. Certification defines skills and knowledge that individuals need, validates training quality, and motivates participants to successfully complete the training. These findings should be taken into account when developing digital skills policies on international, national and local levels.
Introduction
Our day-to-day lives are becoming more and more dependent on digital technologies. Life without a computer, a tablet or a mobile phone has become unimaginable to many, and more people than ever have access to the internet. In Europe, 84% of households have internet access at home1, whereas in Singapore, this number reaches 88%2. ITU data shows that more than 80% of the youth population (15-24-year-old) worldwide is online and that out of the 830 million young people who are online, 320 million (39%) are in China and India3. These impressive numbers can create an illusion that, by having access to digital technologies, people, and particularly young people, automatically learn how to use them. People tend to assume that if they own a digital device and know how to use certain applications, then they already have all the necessary skills for personal and professional life.
Previously, a number of National Operators of ICDL carried out digital literacy studies in Europe to find out the real levels of digital skills in their countries4. Austria, Denmark, Finland, Germany and Switzerland were investigated. The data showed that gaps exist between self-perceived and actual levels of digital skills and that even young people, who are sometimes falsely assumed to be ‘digital natives’, usually underperform in practical tests5. In 2017, a replication of the study in India and Singapore revealed that the digital skills gaps identified in Europe are also prominent in these countries.
This paper provides an overview of the main findings of the six studies, discusses their implications for digital skills development on national and international levels, and argues that any self-assessment tool should always be complemented by a diploma or a certificate as evidence of the competence level.
Digital literacy studies by ICDL National Operators
Two different target groups were used in the digital skills studies. In Austria and Switzerland, a representative group of participants aged between 15 and 64 was analysed. In Denmark, Finland, Germany, India and Singapore, the selected target group was young people, primarily university students.
The studies were based on a simple research logic. First, respondents were requested to self-assess their digital skills in the areas relating to five ECDL modules: Computer Essentials6, Online Essentials7, Word Processing, Spreadsheets and Presentation. Then, they were invited to answer some practical questions and to complete assignments in a simulated work environment. Results from these studies allowed a comparison between self-assessed and actual digital skills levels.
The findings were consistent throughout all the analysed countries:
- First, self-assessment is a poor predictor of actual performance, and people tend to over-estimate their digital skills.
- Second, digital skills gaps exist in all the surveyed countries in Europe and outside Europe.
- Third, digital skills gaps are persistent among young people as well as their older counterparts.
- Fourth, people who have previously acquired digital skills certification perform better than those without such a certification.
Self-assessment is a poor measure of digital skills
Results from all of the analysed countries indicate that people cannot adequately assess their digital skills. Respondents incorrectly evaluate their competences, most often by overestimating. For example, in Austria, 94% of survey participants assessed their general computer skills as ‘average’ to ‘very good’. However, in the practical test, only 39% of them scored that high (see Figure 1). Very similar results were reached in Switzerland. For example, 85% of survey respondents indicated that they were ‘good’ or ‘very good’ in using the internet and email, whereas, in reality, only 34% of them answered the practical questions correctly.
The fallacy of the ‘digital native’
The term, ‘digital native’
The term ‘Digital Native’ was suggested by Marc Prensky in 20018. The author defined ‘digital natives’ as young people who grew up surrounded by, and using computers, cell phones and other tools of the digital age. His theory implied that young people are all “native speakers” of the digital language of computers and the internet. This powerful metaphor was quickly taken-up by media, politicians and parents. Several new terms were introduced such as ‘Millennials’, ‘Generation Z’ and ‘iGeneration’ to describe young people who inherently possess the skills for safe and effective use of digital technologies.
Young people lack digital skills
Various research shows that young people do not inherently possess digital skills. The International Computer and Information Literacy Study (ICILS), the ‘EU Kids Online’ survey, as well as studies in Australia, Italy, and Canada, demonstrate that exposure to technology cannot be equated to an ability to use it9. A recent study published in the journal, ‘Teaching and Teacher Education’, also found no empirical evidence to back up the claims that exposure to technology affects the way that young people use it10. In fact, if youngsters do not have access to digital education, they do not acquire the crucial competences needed for their personal and professional lives.
“I don’t think you can expect children to know that (digital literacy) instinctively any more than you can expect them to understand Shakespeare or Proust instinctively. It’s something that is taught; it’s a skill.”11
Matthew d’Ancona, research fellow at Queen Mary University of London
Critical, safe, and effective use of digital technologies is gaining more importance as there are increasingly more fake news stories12 and information that is hard to validate online. Anonymity and physical distance has enabled internet users to create fake identities and use them to manipulate children. The NMC Horizon Report shows that young people lack the ability to critically evaluate information online and judge the validity of content shared on social media13. Hence, familiarity with technology does not necessarily equate to a genuine and critical understanding of digital environments.
Young people overestimate their digital skills
Digital literacy studies carried out by ICDL National Operators provide additional proof that young people are not as digitally savvy as it is often assumed. Denmark, Finland, Germany, India and Singapore focussed their surveys on young people – university students or higher-tier secondary education students. In all of these countries, young people overestimated their digital skills. For example, in Singapore, 88.5% of the study participants rated their skills as ‘fair’ to ‘excellent’, but their average score was only 55% (see Figure 2). Similarly, in India, 84.6% of respondents self-evaluated as ‘fair’ to ‘excellent’ in word processing, but their performance was less than 50%.
Young people particularly lack productivity skills
In all five countries that chose young people as their target group, the widest skills gaps existed in productivity applications such as spreadsheets, word processing and presentation software (see Table 1). These discrepancies could be explained by the difference between digital ‘lifestyle’ and digital ‘workplace’ skills. Young people easily develop skills that are associated with their hobbies: social media, games, online music and videos14. Moreover, they consume digital media passively: “social media is used as a passive source of information reception and not as a tool for actively creating content, interacting with others, and sharing resources”15. In this way, the productivity skills that young people need for their studies and future work remain poor.
| Country | Target group | Spreadsheets | Presentation | Documents | |
|---|---|---|---|---|---|
| Denmark | First-year university students | Self-assessed | 89% | 84% | 76% |
| Actual | 57% | 58% | 79% | ||
| Finland | First-year university students | Self-assessed | 74% | 87% | 83% |
| Actual | 37% | 60% | 63% | ||
| Germany | First-year university students and final-year higher-tier secondary education students | Self-assessed | 79% | 81% | 84% |
| Actual | 38% | 66% | 60% | ||
| India | University students | Self-assessed | 64% | 86% | 85% |
| Actual | 33% | 60% | 49% | ||
| Singapore | University and polytechnic students | Self-assessed | 60% | 66% | 68% |
| Actual | 35% | 57% | 45% |
Table 1. Self-assessment vs actual skills in productivity applications for young people in Denmark, Finland, Germany and Singapore.
Certification is an objective measure of skills levels
The tendency to over-estimate one’s digital skills is not surprising. Digital skills are associated with modern, active and successful people. Similarly, exclusion from any part of the digital world due to incompetence may be perceived as being old-fashioned and odd. It is natural for most people to aspire to be a part of the former group, where being digitally savvy is a must.
People often assume that they automatically develop the right skills simply by using digital applications. High self-assessment is also driven by a willingness to perform better or by trying to impress potential employers. The fact that self-assessment is often biased, has also been recognised in multiple academic articles16. To avoid self-evaluation bias, an objective measure of actual skills is necessary.
Certification is the right tool to measure the actual level of digital skills. It defines skills and knowledge that individuals need, guides and validates training, and provides proof of the skills acquired. As a consequence, if self-assessment tools are used, they need to be complemented by a diploma or a certificate as evidence of the competence level claimed. One of the best-known examples in Europe in this area is the Europass CV template. Its ‘Digital competence’ section is based on a self-assessment grid, but it also asks users to add a certificate. Similarly, any other document that includes self-assessed skills should always be supported by a diploma or a certificate.
Another significant benefit of certification is that it validates training quality. The fact that a person has simply sat through a number of hours of training does not necessarily mean that they acquired the expected skills and knowledge. Only a test at the end of the training can reveal what the participant has learned. In addition, people are motivated to successfully finish the training when they expect to acquire a certificate at the end of it. Indeed, research data from Austria, Germany, Switzerland and Singapore shows that ICT certificate holders score significantly better in practical tests than respondents without similar certification.
Conclusion
The research carried out in the seven countries demonstrates that self-assessed skills should never be taken at face value. Whether you are a training provider evaluating achievement of the training goals, an HR manager scanning CVs for a new position, a university admissions officer selecting students for the next school year, or a job seeker wishing to demonstrate your skills to a potential employer – you should only rely on an objective skills measure. Thus, any form of self-assessment should be always accompanied by a credible certificate or a diploma as a proof of the skills declared.
Research from Austria, Denmark, Finland, Germany, Switzerland, India and Singapore revealed that all of these countries suffer from significant digital skills shortages. Digital skill gaps in these countries could be worse than expected as official statistical data is often based on self-assessment, rather than objective measures.
Moreover, the data refutes the widespread belief that young people are ‘digital natives’. In all of the analysed countries, young people underperformed in a number of tasks, primarily those relating to productivity skills such as working with word processing documents, presentations and spreadsheets. This proves that, without digital education, young people cannot unlock the full potential of digital technologies as learners, employees, entrepreneurs and citizens of the digital world.
Certification is an effective tool for addressing these challenges. It provides objective proof of the skills acquired. It defines skills and knowledge that individuals need, validates training quality, motivates participants to successfully complete the training and achieve better results. Certification also demonstrates a person’s competence to potential or current employers.
Awareness about the value of digital skills has been rising internationally. For the first time in Europe, the ‘Upskilling Pathways: New Opportunities for Adults’17 initiative officially recognised that digital literacy is of the same level of importance as reading, writing and numeracy. In Singapore, a new digital readiness programme is being set up under the Ministry of Communications and Information to coordinate initiatives on digital inclusion, cyber safety, information and media literacy18. Activities such as these need to be supported in a consistent and structured manner – and certification programmes such as ICDL can help to achieve this goal.
References and footnotes
1: ITU, “ICT Facts and Figures 2017”, https://www.itu.int/en/ITU-/Statistics/Documents/facts/ICTFactsFigures2017.pdf
2: World Economic Forum, “The Global Information Technology Report 2016”, http://www3.weforum.org/docs/GITR2016/WEF_GITR_Full_Report.pdf
3: ITU, “ICT Facts and Figures 2017”, https://www.itu.int/en/ITU-/Statistics/Documents/facts/ICTFactsFigures2017.pdf
4: ECDL Foundation “Perception and Reality: Measuring Digital Skills in Europe”, 2016, https://www.icdleurope.org/policy-and-publications/perception-reality-measuring-digital-skills-gaps-in-europe-india-and-singapore/
5: ibid
6: The syllabus of the Computer Essentials Module consists of the following categories: Computer and Devices, Desktop, Icon, Settings, Outputs (working with text and printing), File Management, Networks, Security and Well-Being, see https://www.icdleurope.org/workforce/computer-essentials/
7: The syllabus of the Online Essentials Module consists of the following categories: Web Browsing Concepts, Web Browsing, Web- Based Information, Communication Concepts and Using Email, see https://www.icdleurope.org/workforce/online-essentials/
8: Marc Prensky “Digital Natives, Digital Immigrants”, 2001, http://www.marcprensky.com/writing/Prensky%20-%20Digital%20Natives,%20Digital%20Immigrants%20-%20Part1.pdf
9: Results of these studies are summarised in ECDL Foundation’s paper, “The Fallacy of the ‘Digital Native’: Why Young People Need to Develop their Digital Skills”, 2014, https://www.icdleurope.org/policy-and-publications/the-fallacy-of-the-digital-native/
10: P. Kirschner, P. De Bruyckere, “The Myths of the Digital Native and the Multitasker”, 2017, https://doi.org/10.1016/j.tate.2017.06.001
11: OECD blog, “Digital literacy will probably be the only kind of literacy there is”, 2017, https://oecdedutoday.com/digital-literacy-will-probably-be-the-only-kind-of-literacy-there-is/
12: For example, recent US presidential elections. See H. Allcott, M. Gentzkow, “Social Media and Fake News in the 2016 Election”, 2017, https://web.stanford.edu/~gentzkow/research/fakenews.pdf
13: NMC Horizon Report: 2017 Higher Education Edition, https://www.nmc.org/publication/nmc-horizon-report-2017-higher-education-edition/
14: Selwyn, Neil, “The Digital Native – Myth and Reality”, 2009, http://www.emeraldinsight.com/doi/abs/10.1108/00012530910973776
15: P. Kirschner, P. De Bruyckere, “The Myths of the Digital Native and the Multitasker”, 2017, http://www.sciencedirect.com/science/article/pii/S0742051X16306692
16: See, for example, G. Evangelinos, D. Holley “Developing a Digital Competence Self-Assessment Toolkit for Nursing Students”, 2014, http://arro.anglia.ac.uk/333373/1/Evangelinos%20Holley%202014.pdf and H. Beetham, “IT Induction – is it still necessary and how it should be done”, 2013, Online Mailing List: LDHEN@JISCMAIL
17: Council Recommendation of 19 December 2016 on Upskilling Pathways: New Opportunities for Adults, (2016/C 484/01), http://eur-lex.europa.eu/legal-content/EN/TXT/?qid=1503570759226&uri=CELEX:32016H1224(01)
18: Ministry of Communications and Information. Preparing our people for digital future, https://www.mci.gov.sg/wps2017
Creating the future: computing in education
ICDL and qualification frameworks worldwide
DigComp and ICDL
ICDL in your country