Feeds:: Posts; Comments

AIED 2011 Day Four – Final Round

July 5, 2011 by zytemis

The day started off with an interesting talk about cognitive load and e-learning with a focus on biologically primary and secondary knowledge. The first of those, primary knowledge, is like learning to crawl, walk, eat when you are hungry etc. Things that you have learned and humanity has learned through evolution. Secondary knowledge is knowledge you have learned through society like the rules of baseball or how to play chess, reading, arithmetic etc.

Talks:

Jack Mostow:

Great talk by Jack Mostow on using EEG machines and data mining to gain information about getting some type of state that a learner is in. The work is currently in the preliminary stages, as it was written for a proof of concept for a grant but the results look promising. The ideas are similar to the EAPSI proposal I wrote last year but was denied. Using the EEG data they gathered they were able to predict if a reading word was either easy or difficult for students, both in the silent reading case as well as the reading aloud case, which is really exciting and interesting. Later this summer I will be visiting Tohoku University, to tour the lab of Kawashima sensei and it would be incredible if I could be involved in this type of work with either group and better yet with both.

John Stamper:

John had a great talk about using data to generate the list of knowledge components for a problem rather than having an expert create them, themselves. In this particular case their machine learning techniques was able to adjust the student model and result in better accuracy. Although machines aren’t generating the complete list of knowledge components, things are looking promising, and I have confidence along with John that data and computer science will win out over an expert’s opinion about what knowledge components exist in a problem. Admittedly, although a computer may be able to generate better KC’s, an expert will still be necessary in order to label those components.

Ryan Baker as Neil Heffernan:

Ryan Baker presented a paper for Neil Heffernan about using clustering of students in order to more accuately predict end of course test scores. The authors attempted to use ITS log data to predict standard test score predictions, and presented that work last year, allowing educators to teach, via an ITS, and assess their abilities at the same time, having your cake and eating it too. Turns out, that is easier said than done because the difference between the two models was not significant.

So the next step is to consider, maybe there are clusters of students within in a single class. So they broke a class down into three clusters, made different models for each group and tested the models…still not significant. Next they used an iterative K-means clustering to add more centroids until there is a cluster with no data points, and the algorithm found 8 clusters. Still building 8 different models, and comparing results in no significant difference.

Lastly, however they decided to use an ensemble, take all 8 models and average the different models to create a single model and compare that to the control model. In this case they have a significant difference in the models ability to predict the test scores, exciting! Philip Pavlik made a potentially very powerful comment and suggestion, which is use a weighted average for calculating the ensemble model which should return a more accurate prediction. In the end, it does seem that we can have our cake and eat it too. I will be excited to see the results they will be able to present next year after having a year of development time to improve their prediction model. One down side to using the ensemble approach is that, it does require a large data set and for this work they had over 600 data points, so although powerful, not necessarily available for all cases, nevertheless really cool stuff.