Showing posts with label assessment - diagnostic. Show all posts
Showing posts with label assessment - diagnostic. Show all posts

Wednesday, February 07, 2024

Recent articles: Outcomes; Disinformation; Print/digital preferences; Social media evaluation; Searching as strategic exploration

I need to catch up on articles from the Journal of Academic Librarianship (a priced publication)! The last 2 issues of Volume 49 (2023), include the following:
- Miller, K. Kaufmann, K. & Trnka, R.E. (2023). Measuring outcomes: Information literacy instruction assessment in a cross-institutional study. The Journal of Academic Librarianship, 49(6), article 102796. https://doi.org/10.1016/j.acalib.2023.102796.
- Chomintra, M. (2023) Assessing the use of critical literacies in mis/disinformation literacy instruction. The Journal of Academic Librarianship, 49(5), article 102762. https://doi.org/10.1016/j.acalib.2023.102762. Rather curiously this article posits in the abstract that "The approaches to mis/disinformation in libraries and information studies have largely been grounded in two forms of literacy education; media literacy and digital literacy" but it is interesting in taking selected lesson plans uploaded to the ACRL sandbox and project CORA and doing a text analysis, including for Equity diversity inclusion and social justice terms.
- Chyne, R.C., Khongtim, J. & Wann, T. (2023). Evaluation of social media information among college students: An information literacy approach using CCOW. The Journal of Academic Librarianship. 49(5), article 102771. https://doi.org/10.1016/j.acalib.2023.102771. CCOW = Credentials, Claims, Objectives, Worldview. "The findings highlighted significant differences in the evaluation of social media information following the implementation of the CCOW information literacy approach."
- Dahlqvist, C.(2023). Cognitive and motivational qualities of task instruction: Cognitive appraisals and achievement emotions of Swedish primary teacher students. The Journal of Academic Librarianship, 49(6), article 102797. https://doi.org/10.1016/j.acalib.2023.102797.
- Johnson, B. & Reed, E. (2023) The impact of cultural capital on searching as strategic exploration in incoming first-year students. The Journal of Academic Librarianship,49(6), article 102807, https://doi.org/10.1016/j.acalib.2023.102807.
- Yu, C. (2023). Integrating Information Literacy and Academic Writing: Developing a Self-Assessment Scale of Information-Based Academic Writing. The Journal of Academic Librarianship,49(6), article 102804. https://doi.org/10.1016/j.acalib.2023.102804.
- Shehata, A. & Eldakar,M. (2023). Scholarly communication practices and attitudes of Egyptian junior researchers: An exploratory study. The Journal of Academic Librarianship, 49(6), article 102811, https://doi.org/10.1016/j.acalib.2023.102811.
- Matthews, J. & Johnson, A. (2023). Preferring print: The planned behavior and preferences of first-generation college students in the academic library. The Journal of Academic Librarianship, 49(6), article 102813. https://doi.org/10.1016/j.acalib.2023.102813.

Image by Sheila Webber using Midjourney AI with prompt librarian heroine saving books in a forest --niji 5 cute

Friday, November 05, 2021

New articles: Measuring informed learning; Undergrads and assignments; Services to users with disabilities; Search phrases

The latest issue of open access journal College & Research Libraries (Vol 82, No 7) includes:
- Developing the Informed Learning Scale: Measuring Information Literacy in Higher Education. Michael Flierl, Clarence Maybee, and Emily Bonem.
- Exploring the Development of Undergraduate Students’ Information Literacy through Their Experiences with Research Assignments. Amanda L. Folk.
- Citation and Referencing Support at an Academic Library: Exploring Student and Faculty Perspectives on Authority and Effectiveness. Lydia Dawe, Jackie Stevens, Bob Hoffman, and Morgann Quilty.
- What Information Are We Providing to Users with Disabilities? An Analysis of ARL Libraries’ Accessibility Webpages. Amelia Brunskill, Catherine Lantz, and Kavita Mundle. 
- Reference and Instructional Services to Postsecondary Education Students with Intellectual Disabilities. Mirah J. Dow, Bobbie Sartin, and Brady D. Lund.
- Phrasing in Reproducible Search Methodology: The Consequences of Straight and Curly Quotation Marks. Katie Barrick and Amy Riegelman. "The study found that 42.5 percent of platforms ignored curly quotation marks and interpreted the test term as a phrase, 30 percent of the bibliographic platforms acknowledged curly quotations and completed the phrase search, and one platform flagged curly quotation marks as an unsupported character."
Go to https://crl.acrl.org/index.php/crl/issue/view/1608/showToc
Photo by Sheila Webber: autumn leaves, November 2021

Tuesday, February 04, 2020

New articles: Reddit; Point-of-Need teaching; Librarians as developers; PIL scale; Misinformation

The latest issue of the open access journal Communications in Information Literacy has been published (Volume 13, Issue 2, 2019). The articles are:
- Reddit as an Analogy for Scholarly Publishing and the Constructed, Contextual Nature of Authority by Anna M. White
- Conceptual and Procedural Knowledge: A Framework for Analyzing Point-of-Need Information Literacy Instruction by Amy VanScoy
- Academic Librarians’ Experiences as Faculty Developers: A Phenomenographic Study by Michael Flierl, Clarence Maybee, and Rachel Fundator (the four categories they discovered were: Connector – connects instructors to pedagogic or technology experts; Facilitator – guides instructors through course design; Colleague – nurtures mutually beneficial relationship with instructors; Developer – develops instructors to transform their approach to teaching)
- Initial Development of the Perception of Information Literacy Scale (PILS) by Matthew Doyle, Britt Foster, and Mariya A. Yukhymenko-Lescroart (they say they are responding to "a lack of valid and reliable Framework-based scales for assessing students’ knowledge practices and dispositions")
- From Syndication to Misinformation: How Undergraduate Students Engage with and Evaluate Digital News by Cara Evanson and James Sponsel
Go to https://pdxscholar.library.pdx.edu/comminfolit/vol13/iss2/
Photo by Sheila Webber: St Georges (lecture theatre and home of peregrines), February 2020

Thursday, August 16, 2018

Threshold Achievement Test for Information Literacy (TATIL)

The Threshold Achievement Test for Information Literacy (TATIL) has been released. It has four "modules" (tests) that "can be completed in 50 minutes or less" using internet-connected computers. Each "module" is "inspired by one or more of the frames of the ACRL IL Framework." The modules are Evaluating Process & Authority (“Information Creation as a Process” and “Authority is Constructed and Contextual”); Strategic Searching (“Searching as Strategic Exploration”); Research & Scholarship (“Research as Inquiry” and “Scholarship as a Conversation”); The Value of Information (“Information Has Value”). There is a brochure with a list of the outcomes, performance indicators, and dispositions that each test aims to assess. The tests are priced: you pay per-student, and there are various options for results' reporting.
TATIL was developed with the aid of an advisory board of librarians etc., and they have been working on it for several years. https://thresholdachievement.com/
Photo by Sheila Webber: a London garden, July 2018

Tuesday, July 03, 2018

New articles: misconceptions; IL testing; artists' IL

There is a new issue of the open access journal Communications in Information Literacy; Volume 12, Issue 1 (2018). The articles are:
- Predictable Information Literacy Misconceptions of First-Year College Students by Lisa Janicke Hinchliffe, Allison Rand, and Jillian Collier
- SAILS, Take 2: An Exploration of the “Build Your Own Test” Standardized IL Testing Option for Canadian Institutions by Rumi Y. Graham, Nichole Eva, and Sandra Cowan
- Crossing the Studio Art Threshold: Information Literacy and Creative Populations by Sarah Carter, Heather Koopmans, and Alice Whiteside
Go to https://pdxscholar.library.pdx.edu/comminfolit/vol12/iss1/
Photo by Sheila Webber: travellers in Stavanger, May 2018

Wednesday, May 04, 2011

ACRL Proceedings

Some of the papers from the (US) Association of College and Research Libraries (ACRL) annual conference, held on March 30-April 2 2011, are available full text. The papers, covering many aspects of academic library work, include a number relevant to information literacy e.g.:
- Do Screencasts Really Work? Assessing Student Learning through Instructional Screencasts; Jo Angela Oehrli, Julie Piacentine, Amanda Peters, and Benjamin Nanamaker
- Do You See What I See?: Comparing Student and Librarian Perceptions of Learning Outcomes; Faith Steele and Scott Mandernack
- Uncovering the IL Disconnect: Examining Expectations among Librarians, Faculty and Students; Sheila Cunningham, Allison Carr, and Stephanie Sterling Brasley
- Show Me the Data! Partnering with Instructors to Teach Data Literacy; Karen Hogenboom, Carissa M. Holler Phillips, and Merinda Hensley
- Put the Pencil Down: Using Student Podcasts to Assess Learning in a For-Credit Reearch Course; Lauren Yannotta and Brian Lym
- Listening to Students: A User-Centered Assessment of Incoming Graduate Students’ Research Skills; Julie Petr and Amalia Monroe-Gulick
- From Embedded to Integrated: Digital Information Literacy and New Teaching Models for Academic Librarians; Marisa Walstrum, Larissa Garcia, and Rob Morisson
The index page for the papers is http://www.ala.org/ala/mgrps/divs/acrl/events/national/2011/papers/index.cfm
Photo by Sheila Webber: Tulips, April 2011

Thursday, April 21, 2011

LILAC report: Pecha Kucha sessions #lilac11

The LILAC (Information Literacy) conference 18-20 April 2011 in London, UK, finished yesterday and I will be catching up with posts over the next few days. The final day of the conference was held at the LSE. We were encouraged to go up to the building's top floor at lunchtime to enjoy the views, and I took several photos, including this one.
I'll report here on Wednesday's Pecha Kucha session, where people have to do very short presentations (with 20 slides, each up for 20 seconds i.e. 6 min 40 seconds). There were six presentations (in two groups of three, with brief question time at the end of each group). The included talks on an evaluation/assessment tool, a refreshing approach with "problem clinics", distance learning, and a skills resource. These are quick summaries of what I thought they were about.

1. Can we do it? Yes we can! Information literacy perceptions among Croatian school librarians. Sonja A Å piranec and Mihaela Banek Zorica. This described an initiative from two academics and school librarians in Croatia. The aim was to define information literacy strategy which was sustainable. To develop this, eight workshops were held, with over 200 school librarians participating. They recognised to achieve their goal, it was important to involve all stakeholders, such as head teachers, and a large workshop was held with these stakeholders. Three aims were identified: educational aspects (practical examples and gaining insights); Building up communities (including all stakeholders on an ongoing basis); and Defining bottom-up strategies from the practitioners’ viewpoint. The Ministry of Education recognised this was a useful approach, and so more workshops and progress are planned.

2. Earth to distance learning librarian was presented by Lucy Collins and Mari Ann Hilliar. They talked about supporting distance learning students at Cardiff University. With the increase in student numbers, a personalised service became less feasible, so a distance learning toolkit was developed. It was noticed that people who had a face to face induction were doing better with this toolkit: so how to substitute for this with distance-only students? They looked at free software options. They decided to use dimdim. For a while this was great, but unfortunately it changed hands, so eventually they moved on to Big Blue button, in the version operated by JISC. They found it doesn’t have audience participation/ interaction features they would like. But does have some positive features, including being able to customise your screen. They are targeting specific cohorts together, fitting in with module contact time, with a maximum of 12 people in any one session. They are aware that they need to adopt a strategy which encourages questioning etc. They want screen sharing options (for more learner-centred work) and ability to record sessions.

3. Nancy Goebel talked at gathering evidence on information literacy assessment and evaluation. She was describing the open source software WASSAIL, which they developed. The WASSAIL home page is here: http://www.library.ualberta.ca/augustana/infolit/wassail/ It facilitates questionnaire data acquisition and processing, including helping to process pre and post tests and individual session evaluations. In WASSAIL you can add questions, create templates, input data and produce reports. You can have multiple choice and scaled answers, and also open ended questions. You create the questions, group them into templates, create a web form (specifying expiry dates etc. or some special features). After the responses come in, you can generate reports. For pre and post test, you can generate a “gains analysis” chart (showing change from pre to post test). This tool won the ACRL Instructional achievement award in 2010.

4. Rowena Macrae-Gibson talked about Upgrading to Upgrade at City University. They saw a need for a resource that brought together information on lifelong learning skills, recognising that students preferred having short snappy materials, using different media, and may start at a basic level. It includes study tips, mindmapping, lectures, employability etc. Upgrade was produced by a team. They overcame problems of lack of money and local website functionality by embedding/ linking materials in materials in the cloud. They use Wordle to describe content visually. The resource complements face to face teaching, and material can also be used in workshops or lectures. They see it opening up doors with the official university website. Upgrade is at http://www.city.ac.uk/upgrade/

5. Susan Boyle of University College Dublin talked about Problem Clinic: Information Literacy triage. She described a global confusion crisis, symptoms include being google eyed, blurred vision and so forth. The clinics for students are active learning sessions which include practicals, attacking afflictions like googlitis and Wikipedia dependence. So – you take histories, note symptoms, diagnose and then recommend therapy. Firstly you need to be an active listener, note the priority symptoms, then treat with a demo, tutorial or game, depending on the symptoms described. For an example of “games” she has got a matched cards game (matching a strategy and what you use it for). Then there is a stepping stone game, putting steps in process (e.g. getting an article) in the right order. A variety of treatments was recommended, as well as engaging the student brain creatively at the start. At the end she quoted some problem-clinic testimonials from students. This talk was a good example of using the “IL as …” approach consistently, effectively and amusingly.

6. Finally Andrew Walsh (University of Huddersfield) talked about using active learning techniques in information literacy instruction. He emphasised that technology was not required for active learning (which he defined as learning by doing), although it could be a useful tool. In terms of instant response, he mentioned using “Poll anywhere” (free online version), but also using cards, getting people to stand up or sit down, having people run to different spots in the room. He also mentioned non linear presentations (have images for each section, get the audience to pick an image to determine what comes next), and instant podcasting.

Monday, February 21, 2011

Poll everywhere

There has been a good discussion on the American information literacy list ili-l in the past few days, about the polling tool Poll Everywhere; the advantages and disadvantages of using it for information literacy. There's a cut down free service, and a priced service for larger use. The ili-l list archive is at http://lists.ala.org/sympa/arc/ili-l/ and the Poll Everywhere site is at http://www.polleverywhere.com/
Photo by Sheila Webber: St Georges lecture theatre/ student flats (former church), Sheffield, February 2011

Wednesday, February 02, 2011

iCritical Thinking test

I don't think I have blogged the fact that the name of the test produced, sold and administered by the (US) Educational testing Service (ETS) is now called iCritical Thinking. This was formerly called iskills, and before that was called the ICT Literacy test (I think the content has also changed a bit each time). It incorporates aspects of information literacy. To quote their website "An outcomes-based assessment, the iCritical Thinking Certification measures applied ICT literacy skills through a range of real-world tasks." "features real-time, scenario-based tasks that measure an individual's ability to navigate, critically evaluate and understand the wealth of information available through digital technology" "is endorsed by the Global Digital Literacy Council (GDLC) and aligned with the nationally recognized Association of Colleges & Research Libraries (ACRL) standards" http://www.ets.org/icriticalthinking/about/
Photo by Christopher Webber: me near Conwy, September 2011

Friday, August 27, 2010

Health literacy indicators

DSS Research specialises in market research for the healthcare industry in the USA; therefore it aims (I assume) to show how it can help them maximise profits, which can be done by keeping those who take out health insurance healthier, so people don't make so many claims (or am I being cynical here?) Anyway, this "Health Care Engagement Index" did seem interesting and relevant to those aiming to improve health information literacy. Also, for those interested in information literacy indicators, the inclusion of "behaviours" and "attitudes" is interesting (i.e. it isn't just a matter of testing skill and knowledge).
"To help our clients identify the level of involvement of their customers and how best to encourage those consumers to adopt healthy behaviors, DSS Research applied its years of experience in the health care industry to create a definitive measure of consumer engagement. The DSS Health Care Engagement Index™ (HCEI™) uses health literacy, knowledge, attitudes and behaviors to measure each consumer’s involvement in their personal health care and the health care system overall."
The 4 constructs they look at are: health care literacy, health care knowledge, health care attitudes and health care behaviours
http://www.dssresearch.com/insurance/products_services/engagement/use_hcei_for_free.asp
There are some results (identifying four degrees of engagement: disengaged, reactive, involved, and engaged) at
http://www.dssresearch.com/insurance/products_services/engagement/engagement_results.asp
Photo by Sheila Webber: Memorial benches, Happy Valley, Llandudno, Wales, August 2010. I have never seen so many memorial benches in any other town: a lot of people loved holidaying in Llandudno.

Tuesday, January 12, 2010

Reference services review

The latest issue of Reference services review includes the following (at time of writing the issue was only in e-preview, so I can't provide page numbers)
- Klentzin, J. et al. (2010) "Pecha Kucha: Using "Lightening Talk" in University Instruction." Reference services review, 38 (1).
- Ismail, L. (2010) "What Net Generation students really want: determining library help-seeking preferences of undergraduates." Reference services review, 38 (1). (This study showed that young people did not necessarily prefer "net gen" options for help).
- Lym, B. et al (2010) "Assessing the assessment: how institutions administered, interpreted, and used SAILS" Reference services review, 38 (1).
("Results from this study suggest that SAILS can be most effective if there is statistical/institutional research support for data analysis, if the sampling method for selecting test takers is more rigorous, if SAILS is used in conjunction with other instruments, and if the SAILS data is correlated with other institutional data.")
Photo by Sheila Webber: Snowman in Regent Court, January 2010

Saturday, April 21, 2007

American i-skills

Thanks to the InformationLiteracy Land of Confusion (which has recently taken a new web address, http://www.information-literacy.net/ by the way, as Michael Lorenzen mentioned on a comment to the blog a week or so ago) for alerting me to the fact that the Educational Testing Service in the USA has changed the name of its ICT Literacy test to the iskills test. There is information about the test here. The press release on the ETS website has a quote about why they chose it " 'We selected iSkills because it is catchy, contemporary and relevant,' says Mary Ann Zaborowski, ETS’s Director of Product Management for the iSkills™ assessment. 'The new name is more spirited and better reflects the nature of this very different type of assessment . . . it’s interactive and really cutting edge.' " Hmmmm ... notice that little ™? I'm not a trademark expert, but I wonder if they can sustain that, given that JISC launched the i-skills initiative a couple of years ago (see http://www.jisc.ac.uk/publications/
publications/pub_sissdocs.aspx
) Perhaps the "-" makes all the difference, but I think that might count as Prior Art in intellectual property terms.
Photo by Sheila Webber: Cherry tree, part of Sheffield University's Firth Court in the background, April 2007.

Friday, March 16, 2007

IL article

An American news article provides varying views on information literacy and summarises some different approaches, including the Educational Testing Service test.
Foster, A. (2007) "Information Navigation 101." Chronicle of Higher Education, 9 March.
http://chronicle.com/free/v53/i27/27a03801.htm
Photo by Sheila Webber: view over Malvern, March 2007.

Monday, January 15, 2007

Health information literacy

Photo by Sheila Webber: Schonbrunn Palace, Vienna

Thanks to Michael Lorenzen for bringing this paper to my attention: it is available full text on the web. Ivanitskaya, L, O'Boyle, I and Casey, A. (2006) "Health Information Literacy and Competencies of Information Age Students: Results From the Interactive Online Research Readiness Self-Assessment (RRSA) ." Journal of medical internet research, 8(2). http://www.jmir.org/2006/2/e6/

"The Research Readiness Self-Assessment (RRSA) is based on the [ACRL] Information Literacy Competency Standards for Higher Education, and it measures proficiency in obtaining health information, evaluating the quality of health information, and understanding plagiarism. [...] This study aimed to measure the proficiency of college-age health information consumers in finding and evaluating electronic health information; to assess their ability to discriminate between peer-reviewed scholarly resources and opinion pieces or sales pitches; and to examine the extent to which they are aware of their level of health information competency." 308 students participated. As well as being asked some multiple choice questions and doing some short exercsies, the students were asked "How do you rate your research skills?" . "Most respondents (84%) believed that their research skills were good, very good, or excellent. Students’ self-perceptions of skill tended to increase with increasing level of education. Self-reported skills were weakly correlated with actual skill level, operationalized as the overall RRSA score..."

Photo by Sheila Webber: ground of Schonbrunn Palace, Vienna, December 2007.

Wednesday, November 22, 2006

ICT Literacy

The Educational Testing Service has published a presentation in which it summarises some of the results from students taking their ICT Literacy test (which covers aspects of information literacy). Data is taken from the results of over 6,000 test-takers in 2006. They emphasise the extent to which students failed to achieve full scores: presumably partly to demonstrate the value and rigour of their (priced) test, but the presentation could also provide useful data for people who wish to argue that students are coming into university with insufficient information literacy.
ETS. (2006) 2006 ICT Literacy Assessment preliminary findings. http://www.ets.org/Media/Products/ICT_Literacy/
pdf/2006_Preliminary_Findings.pdf
see also http://www.ets.org/ictliteracy/
Photo by Sheila Webber: Weston Park, Sheffield, Nov 2006.