Menu Close
Menu

Future Directions

Questions for Future Research

Below are a select few research questions extending from the CrowdEEG project and related work that may be of interest to other research teams:

  • Is it possible to identify critical EEG interpretation tasks that are most probable to set up a “cascade of disagreement”, and if detected successfully, may allow for more cost-effective deployment of expert resources by focusing on resolving disagreement surrounding such tasks?[6]
  • Which methods of expert communication are best for resolving disagreement in the context of EEG interpretation? In addition, what is the minimum “bandwidth”, or level of communication necessary for successful resolution or disambiguation?[6]
  • How can we increase the efficacy of machine learning algorithms in the context of sleep stage classification and seizure detection? Where are such algorithms most likely to make errors? How do we program these algorithms to recognize when more information (e.g., about a sleep stage) is required, and to query an expert for help?
  • How can we increase the confidence of expert clinicians in automated or human-computational systems of EEG analysis?
  • What is the optimal training regimen for non-expert crowd workers learning the same EEG interpretation tasks as highly trained experts? What is the minimum amount of information these non-experts require for successful completion of such tasks such that their aggregate performance can come close to or match that of a highly trained expert?

Resources

Our Annotator: https://app.crowdeeg.ca/

The Bhutan Epilepsy Project: http://www.mhinnovation.net/innovations/bhutan-epilepsy-project-smartphone-eeg-diagnose-seizure-disorders

University of Waterloo Computer Science https://cs.uwaterloo.ca/

Waterloo HCI: http://hci.uwaterloo.ca/

[6] Schaekermann, M., Law, E., Larson, K., & Lim, A. (2018). Expert Disagreement in Sequential Labeling: A Case Study on Adjudication in Medical Time Series Analysis. In 1st Workshop on Subjectivity, Ambiguity and Disagreement (SAD) in Crowdsourcing, at HCOMP 2018. Zurich, Switzerland.