Sheffield Hallam University is currently exploring the potential of technology-enabled feedback to improve student learning. Funded from the Higher Education Academy’s e-learning research call 2008/9, Technology, Feedback, Action! is exploring the impact of learning technology upon students’ engagement with their feedback. Building on recent innovations at Sheffield Hallam including the use of the Blackboard Grade Centre for online publication of feedback and marks, the adaptive release of marks through a bespoke Assignment Handler and electronic Feedback Wizard linking feedback to learning outcomes, the study aims to consider which elements add most value, e.g. the extent to which the timely delivery of feedback supports effective forward planning, whether withholding marks does encourage deeper reflection upon the written feedback given and the impact of electronic publishing on quantity and quality of feedback provided.
As part of this project we have recently published a review of current literature regarding the application of technology to deliver and support the use of feedback: http://evidencenet.pbwiki.com/Technology%2C-Feedback%2C-Action!-Literature-Review. We hope that you find the information in the literature review useful and invite you to contribute to and comment on the content of the literature review wiki.
More information about this project (along with the other 2008/9 e-learning research projects) can be found via the Higher Education Academy EvidenceNet at: http://evidencenet.pbwiki.com/e-Learning+Research+Projects+200809.
If you would like further information regarding the study please contact Stuart Hepplestone at Sheffield Hallam University (0114 225 4744 or S.J.Hepplestone@shu.ac.uk).
Wednesday 20 May 2009
Tuesday 13 May 2008
Wednesday 23 April 2008
Coding data the tagcloud way
Health warning for research methods purists: THERE IS VERY LITTLE SCIENCE IN THIS POST (though there is a healthy dose of pragmatic common sense)
Since we introduced a personalised online space for students 3 years ago, we have been undertaking an annual survey to ensure that the student voice is central in informing future developments. With approximately 2000 responses each year and a mix of 20 quantitative and qualitative questions the exercise is highly valued. I have to admit that I personally look forward to getting my hands on the data and getting a feel for what students value. I am also a closet (or not so closet some might say) data freak, but it is a double edged sword - sleep is often sacrificed for the greater good of coding the data and it can take over chunks of my life.
Sooooo, on the bus on the way home to get stuck into coding more data, a strange thought crossed my mind. If I input the responses to a tagcloud generator would it do some useful coding for me? Obviously this is not the same a reading the full comment, and full qualitative quotes are the most useful in how we disseminate that data, on the other hand, could it give me a quick overview of the response to a particular question. So I resolved nothing ventured nothing gained.
I decided to use the free-text service in tweetclouds (mostly cos tweetclouds, for me, is aestheically pleasing and it lists them in alphabetical order - useful for next bits...ie performing search on the words in Excel that catch my interest to check context)
I chose my favourite question on the survey that asked:
How would you describe shuspace to a prospective student?
and this is what I got:
(click on image to see larger size)
But look! look! - cloud created in 20.6341 seconds (my coding exercise will take considerably longer) and I now know that "information" occurs 421 times, "useful" and "easy" feature highly (though without checking, of course, I can't testify to whether they are prefaced with "not very" but a quick check into the data can tell me that) Similies are also easier to identify as are things that might seem surprising.
In the interests of science I will code the question and see what that tells me, but if I compare and, as I suspect, the trends are similar then the 20 second coding clouds may well entice me further ;-)
Since we introduced a personalised online space for students 3 years ago, we have been undertaking an annual survey to ensure that the student voice is central in informing future developments. With approximately 2000 responses each year and a mix of 20 quantitative and qualitative questions the exercise is highly valued. I have to admit that I personally look forward to getting my hands on the data and getting a feel for what students value. I am also a closet (or not so closet some might say) data freak, but it is a double edged sword - sleep is often sacrificed for the greater good of coding the data and it can take over chunks of my life.
Sooooo, on the bus on the way home to get stuck into coding more data, a strange thought crossed my mind. If I input the responses to a tagcloud generator would it do some useful coding for me? Obviously this is not the same a reading the full comment, and full qualitative quotes are the most useful in how we disseminate that data, on the other hand, could it give me a quick overview of the response to a particular question. So I resolved nothing ventured nothing gained.
I decided to use the free-text service in tweetclouds (mostly cos tweetclouds, for me, is aestheically pleasing and it lists them in alphabetical order - useful for next bits...ie performing search on the words in Excel that catch my interest to check context)
I chose my favourite question on the survey that asked:
How would you describe shuspace to a prospective student?
and this is what I got:
(click on image to see larger size)
But look! look! - cloud created in 20.6341 seconds (my coding exercise will take considerably longer) and I now know that "information" occurs 421 times, "useful" and "easy" feature highly (though without checking, of course, I can't testify to whether they are prefaced with "not very" but a quick check into the data can tell me that) Similies are also easier to identify as are things that might seem surprising.
In the interests of science I will code the question and see what that tells me, but if I compare and, as I suspect, the trends are similar then the 20 second coding clouds may well entice me further ;-)
Saturday 12 April 2008
Engaging senior management with e-learning
It is an absolute truism that the best curriculum innovations emerge when academic staff are enthused and driven to change their current practice. However, this is rarely (if ever?) enough in its own right. Back in the mists of time (2000ish) we asked staff what were the main barriers to change, in this case, to make greater use of technology in LTA design. They gave us five "wishes": time (of course), confidence in a robust and reliable infrastructure, institution provide appropriate support ie put money where mouth is, provide technology that they can "own" (putting the door back on the classroom and putting the emphasis on their relationship with the students) and, very importantly validation that this is a valid thing to spend time on ie clear executive sponsorship. There are so many conflicting demands on staff time and we needed to make it clear that this wasn't something you were expected to do in your own time as your hobby but was a legitimate..(maybe some day soon even essential) part of academic life. One of the things I really like about my own institution is the high level commitment to supporting innovation.
Fast forward seven years and we are experiencing a lot of success in e-learning and the senior managers are still supportive but asking questions like "what's it all about?", "where are the benefits", "is it value for money?" "what will the future look like?" It's easy to see the dilemma - governors ask questions about costs...and benefits, students really like it and expect it, but are also wanting more contact time, staff are all progressing at a different pace. The solution...no, not soultion...our approach was to run a "future of e-learning engagement" for the senior management group - a traditional discussion paper, a 6 week Bb course and a highly participative workshop. There is a LOT more information about this at http://sheffieldhallamuniexec.pbwiki.com/
Was it worth it - definitely - it enabled us to get a few things back on the radar at the senior level as well as a commitment to support growth rather than cut back. The important message (not rocket science for people reading this, I'm sure) was that to get the best out of the technology it isn't the technology that needs the investment but the people... the birth of our digital fluency initative. Other follow-ons have included the importance of expectations (students and staff), the tower and the cloud debates and how we might better join up e-supported LTA and how technology supports the broader student experience. Solutions? no. Challenges? yes....but at least something to get our teeth into ;-)
Fast forward seven years and we are experiencing a lot of success in e-learning and the senior managers are still supportive but asking questions like "what's it all about?", "where are the benefits", "is it value for money?" "what will the future look like?" It's easy to see the dilemma - governors ask questions about costs...and benefits, students really like it and expect it, but are also wanting more contact time, staff are all progressing at a different pace. The solution...no, not soultion...our approach was to run a "future of e-learning engagement" for the senior management group - a traditional discussion paper, a 6 week Bb course and a highly participative workshop. There is a LOT more information about this at http://sheffieldhallamuniexec.pbwiki.com/
Was it worth it - definitely - it enabled us to get a few things back on the radar at the senior level as well as a commitment to support growth rather than cut back. The important message (not rocket science for people reading this, I'm sure) was that to get the best out of the technology it isn't the technology that needs the investment but the people... the birth of our digital fluency initative. Other follow-ons have included the importance of expectations (students and staff), the tower and the cloud debates and how we might better join up e-supported LTA and how technology supports the broader student experience. Solutions? no. Challenges? yes....but at least something to get our teeth into ;-)
Friday 4 April 2008
Using twitter as a research data collection tool
One of the Academic Innovation team at Sheffield Hallam (and expert in learning spaces - recently described as the "voice of reality" - scarily for her), Liz Aspden is experimenting with using twitter as a tool for data collection. Liz's note below explains, please email her if you want to find out more.
Liz writes:
Hi all
We're about to start a data generation exercise looking at students' use of learning spaces, which I thought might be of interest to people on this list. It forms part of a project looking at informal learning preferences, activities, and the spaces students use to support these, which is designed to help us inform future space planning at the institution.
For this phase of the project, we've recruited 15 students to send us regular (average 3 per day) updates via Twitter telling us a little bit about where they're learning - well, as much as they can fit into 140 characters! A few of us will be monitoring and facilitating the study - possibly asking additional questions, providing commentary along the way, or generally just lurking and being distracted from our day to day work. Of course, this being the first time we've tried collecting data via Twitter, it could all go horribly wrong - but if you'd like to watch, lurk, or participate anyway as it unfolds (or unravels...) you're more than welcome.
We're hoping that the study will begin around mid-April so if you are interested, or if you have any queries about what we're up to, please drop me a line and I'll send you further details nearer the time.
Best wishes
Liz
Liz Aspden
Senior Lecturer - Curriculum Innovation (Learning Environments)
Learning & Teaching Institute, Sheffield Hallam University, Sheffield
S1 1WB
e.j.aspden@shu.ac.uk
Liz writes:
Hi all
We're about to start a data generation exercise looking at students' use of learning spaces, which I thought might be of interest to people on this list. It forms part of a project looking at informal learning preferences, activities, and the spaces students use to support these, which is designed to help us inform future space planning at the institution.
For this phase of the project, we've recruited 15 students to send us regular (average 3 per day) updates via Twitter telling us a little bit about where they're learning - well, as much as they can fit into 140 characters! A few of us will be monitoring and facilitating the study - possibly asking additional questions, providing commentary along the way, or generally just lurking and being distracted from our day to day work. Of course, this being the first time we've tried collecting data via Twitter, it could all go horribly wrong - but if you'd like to watch, lurk, or participate anyway as it unfolds (or unravels...) you're more than welcome.
We're hoping that the study will begin around mid-April so if you are interested, or if you have any queries about what we're up to, please drop me a line and I'll send you further details nearer the time.
Best wishes
Liz
Liz Aspden
Senior Lecturer - Curriculum Innovation (Learning Environments)
Learning & Teaching Institute, Sheffield Hallam University, Sheffield
S1 1WB
e.j.aspden@shu.ac.uk
Friday 21 March 2008
Hackneyed phrases and new and emerging hackneyed phrases
Recently I blogged a response to George Siemens' weariness of the concept of "Pedagogy first" and it made me ponder what I phrases feel weary of when I hear them. I find that these are particularly prolific in conference presentations, educational developer/learning technologist meetings and (increasingly) job interviews. Following George's example I'm going to endeavour that this doesn't become a rant but considers the "what lies beneath" of the hackneyed phrases.
1. "I'm looking forward to the day we can drop the 'e' and just talk about learning" usually followed by a mexican wave of nods around the room (and a sigh from me). To me this is a "so what" sort of comment, of course I recognise the sub-text of not treating e-learning as a novelty or needing special attention, but I personally think it is interesting enough to generate a rich seam of conversation in its own right - just before it starts to become ordinary something new and exciting come along to refresh thinking. This is not restricted to e-learning, I've also heard "let's stop talking about assessment/teaching/student support/transition and just talk about learning." Of course everything is a subset of learning but do we really want to just describe everything with one word and start every conversation with totality? Will that really advance our cause? Is it a good measure of impact? What's wrong with using language to shine a spotlight?
2. "technology is just another tool in the academic's toolkit that they may or maynot choose to use and I'm OK with that" this is usually an ed dev/learn tech comment and makes me want to ask a whole host of follow up questions - is technology one tool? in this day and age is there a legitimate opt out? in what circumstances might they choose not to use anything at all? and is it really OK to be OK with it? - where are the parallels? - would a librarian declare the same level of comfort if an academic opted out of using any information resources? I realise that I am going against conventional wisdom here, but I don't subscribe to the toolkit perspective. I think about living, learning, working in a digital age (in a world immersed in technology), I think about authenticity to discipline and/or professions, I think about the rich learning opportunities technology can offer and then I wonder in what context is it [yawn yawn] "just another tool"?
3. "technology is fine, but it isn't appropriate within my subject/discipline" obviously an academic comment (well, you know, ed dev/learn tech don't have the monopoly on hackney). I don't really have much to add to this one - its ??? is absolutely explicit. Is there really a subject or discipline in which learning cannot be enriched by thoughtful application of technology or that doesn't have a real world technology component worth embedding? If there is I haven't come across it yet, but if others have, please share...or (I like a challenge) suggest one and let's see if we can't prove/disprove this once and for all.
New and emerging hackneyed.... well "new and emerging" is probably a good candidate in itself, others might include "e-portfolios are the true personalised learning environments", "wikipedia is damaging students' ability to research", "we need to take learning to where the students are, Facebook, MySpace etc". The recent ELI Spring Focus Session with a focus on authenticity through "learning by doing or learning by thinking like.." offered a range of candidates for the coming soon "hackneys" (I even found myself trotting out a refreshed version of 1 above for which I am eternally sorry - "stop talking about them as students but as engineers, psychologists, nurses etc" - oh dear, oh dear) but for now, for the new and emerging ones, I'm not going to fret they are afterall not established yet and as long as they still offer opportunities for conversations and encourage everyone (whatever their role) to re-consider their practice to be more student-focused then I'll stick behind them for a while.
Your comments are most welcome on any of the above together with, if you wish, suggestions for phrases to put in the "Hackneyed Hall of Yawn"
1. "I'm looking forward to the day we can drop the 'e' and just talk about learning" usually followed by a mexican wave of nods around the room (and a sigh from me). To me this is a "so what" sort of comment, of course I recognise the sub-text of not treating e-learning as a novelty or needing special attention, but I personally think it is interesting enough to generate a rich seam of conversation in its own right - just before it starts to become ordinary something new and exciting come along to refresh thinking. This is not restricted to e-learning, I've also heard "let's stop talking about assessment/teaching/student support/transition and just talk about learning." Of course everything is a subset of learning but do we really want to just describe everything with one word and start every conversation with totality? Will that really advance our cause? Is it a good measure of impact? What's wrong with using language to shine a spotlight?
2. "technology is just another tool in the academic's toolkit that they may or maynot choose to use and I'm OK with that" this is usually an ed dev/learn tech comment and makes me want to ask a whole host of follow up questions - is technology one tool? in this day and age is there a legitimate opt out? in what circumstances might they choose not to use anything at all? and is it really OK to be OK with it? - where are the parallels? - would a librarian declare the same level of comfort if an academic opted out of using any information resources? I realise that I am going against conventional wisdom here, but I don't subscribe to the toolkit perspective. I think about living, learning, working in a digital age (in a world immersed in technology), I think about authenticity to discipline and/or professions, I think about the rich learning opportunities technology can offer and then I wonder in what context is it [yawn yawn] "just another tool"?
3. "technology is fine, but it isn't appropriate within my subject/discipline" obviously an academic comment (well, you know, ed dev/learn tech don't have the monopoly on hackney). I don't really have much to add to this one - its ??? is absolutely explicit. Is there really a subject or discipline in which learning cannot be enriched by thoughtful application of technology or that doesn't have a real world technology component worth embedding? If there is I haven't come across it yet, but if others have, please share...or (I like a challenge) suggest one and let's see if we can't prove/disprove this once and for all.
New and emerging hackneyed.... well "new and emerging" is probably a good candidate in itself, others might include "e-portfolios are the true personalised learning environments", "wikipedia is damaging students' ability to research", "we need to take learning to where the students are, Facebook, MySpace etc". The recent ELI Spring Focus Session with a focus on authenticity through "learning by doing or learning by thinking like.." offered a range of candidates for the coming soon "hackneys" (I even found myself trotting out a refreshed version of 1 above for which I am eternally sorry - "stop talking about them as students but as engineers, psychologists, nurses etc" - oh dear, oh dear) but for now, for the new and emerging ones, I'm not going to fret they are afterall not established yet and as long as they still offer opportunities for conversations and encourage everyone (whatever their role) to re-consider their practice to be more student-focused then I'll stick behind them for a while.
Your comments are most welcome on any of the above together with, if you wish, suggestions for phrases to put in the "Hackneyed Hall of Yawn"
Wednesday 19 March 2008
ELI SFS: Authentic Critical Reflection: Critique_It in Second Life
Michael Connors (Uof Wisconsin-Maddison)
Digital Printmaking - student work exhibited in Second Life - aim to get critique from other students (and anyone else) across the world, virtual presentation before actually doing the final printing. Looks interesting, although the scanned logbook pages feels a little contrived (is this the easiest way to see the pages?) Keen to see the critique process - critique process have seven stages and is captured through a wiki so can use track back.
http://slurl.com/secondlife/NMC%20Arts/167/69/30 (you'll need NMC membership to get to this)
Qus asked about students skill level/support for using SL - response that students don't find it that difficult and take to it quickly use NMC orientation island to get started.
Qus about whether it adds value beyond an exhibition website and wiki or a flickr site and discussion area
Few art Qus about size of originals, lighting, layout etc etc
Future plans for students to present portfolios and negotiate with galleries within SL - able to practice presenting themselves, explore different identities, get to work with "real world" (ie outside education) partners.
Digital Printmaking - student work exhibited in Second Life - aim to get critique from other students (and anyone else) across the world, virtual presentation before actually doing the final printing. Looks interesting, although the scanned logbook pages feels a little contrived (is this the easiest way to see the pages?) Keen to see the critique process - critique process have seven stages and is captured through a wiki so can use track back.
http://slurl.com/secondlife/NMC%20Arts/167/69/30 (you'll need NMC membership to get to this)
Qus asked about students skill level/support for using SL - response that students don't find it that difficult and take to it quickly use NMC orientation island to get started.
Qus about whether it adds value beyond an exhibition website and wiki or a flickr site and discussion area
Few art Qus about size of originals, lighting, layout etc etc
Future plans for students to present portfolios and negotiate with galleries within SL - able to practice presenting themselves, explore different identities, get to work with "real world" (ie outside education) partners.
ELI SFS: Integrating Community History, Technology, and Service Learning: The Digital Durham Project
Trudi Abel (Duke)
fyi Community History is a big theme in the History subject group at SHU.
Students pulling together research project re community history to share with 8th grade students (similar to UoS English Lit dept work on kiddult fiction - worked particularly well because lots of the eng students were considering careers in teaching)
not much going on in this session - its good enough, but once you've got the concept (sentence above) you've got the concept... and the rest is, well you know, history.
the website is great, of course, http://digitaldurham.duke.edu/
fyi Community History is a big theme in the History subject group at SHU.
Students pulling together research project re community history to share with 8th grade students (similar to UoS English Lit dept work on kiddult fiction - worked particularly well because lots of the eng students were considering careers in teaching)
not much going on in this session - its good enough, but once you've got the concept (sentence above) you've got the concept... and the rest is, well you know, history.
the website is great, of course, http://digitaldurham.duke.edu/
ELI SFS: Authentic Learning in History and Social Sciences: How "Real" Can We Make the Classroom Experience?
Scot French (U of Virginia)
Digital History project - the "old fashioned" way of presenting the student research projects was for students to develop websites (combination of digitised primary sources and student essays)
+ves - intro to rich archival holdings, collaboration, peer review, student work "published" for use by others - frequently used by others and for local researchers
-ves - access and preservation, little (or nil) interaction with professional historians
New version - partnership with cultural institutions, think like their real world partners who are "doing history" and learn new tools - strong point (as yesterday) of how the partners value the students' contributions. "Thinking like teachers" - as expected, we need to get more of this "thinking like..." into our SHU terminology (alongside learning per sq ft)
Useful resource link from the chat stream http://scholarsbank.uoregon.edu/dspace/handle/1794/309
Sample student video http://www.primaryaccess.org/show.php?id=11285
Sample resource built re Jefferson's visit to UK in 1786 - collaboration between tutors, real world experts and student projects artefacts - check out Jefferson's mail (one student's project) - run the time line - click on an envelope - go on, you know you want to...
http://primaryaccess.org/hub/browser.php?base=jt
+ves: more sustained engagement with real world partners, high expectations, positive feedback from partners...new prospective partners
-ves: mixed results from students (some students didn't use their imagination, less reflection), partners reluctant to use sgc on their official sites (what a shame), no formal participation by partners in the assessment
Really good session - only small class size (approx 10) but could be transferable if group projects
Carie Windham post to chat "A good discussion about student motivation -- as a student, I think I fell into that, "Just give me the facts" mentality but really loved those courses that forced me to engage. Might have been stressful at the time but, in retrospect, that's when I really felt "alive" "
Digital History project - the "old fashioned" way of presenting the student research projects was for students to develop websites (combination of digitised primary sources and student essays)
+ves - intro to rich archival holdings, collaboration, peer review, student work "published" for use by others - frequently used by others and for local researchers
-ves - access and preservation, little (or nil) interaction with professional historians
New version - partnership with cultural institutions, think like their real world partners who are "doing history" and learn new tools - strong point (as yesterday) of how the partners value the students' contributions. "Thinking like teachers" - as expected, we need to get more of this "thinking like..." into our SHU terminology (alongside learning per sq ft)
Useful resource link from the chat stream http://scholarsbank.uoregon.edu/dspace/handle/1794/309
Sample student video http://www.primaryaccess.org/show.php?id=11285
Sample resource built re Jefferson's visit to UK in 1786 - collaboration between tutors, real world experts and student projects artefacts - check out Jefferson's mail (one student's project) - run the time line - click on an envelope - go on, you know you want to...
http://primaryaccess.org/hub/browser.php?base=jt
+ves: more sustained engagement with real world partners, high expectations, positive feedback from partners...new prospective partners
-ves: mixed results from students (some students didn't use their imagination, less reflection), partners reluctant to use sgc on their official sites (what a shame), no formal participation by partners in the assessment
Really good session - only small class size (approx 10) but could be transferable if group projects
Carie Windham post to chat "A good discussion about student motivation -- as a student, I think I fell into that, "Just give me the facts" mentality but really loved those courses that forced me to engage. Might have been stressful at the time but, in retrospect, that's when I really felt "alive" "
Subscribe to:
Posts (Atom)
