|
See also: Personal
Reviews
SPSP Psychology Software
Summaries
Occasionally, the Society for Personality and
Social Psychology list serve produces an informal survey of psychology
experiment software. These results are extremely informal and biased
in terms of self-selection. With that said, we post them here anyways
because MediaLab and DirectRT usually come out looking pretty good.
Where possible, we've added links to the websites of the other
software programs.
January 6th, 2007
July 7th, 2005
November 9th, 2004
January 8, 2001
DirectRT v2006 Review, Experimental
Psychology
From Stahl, C. (2006). Software for generating psychological
experiments. Experimental Psychology, 53,
218-232:
"Among the reviewed packages, DirectRT achieves the most
complexity reduction. A simple experiment is most easily implemented with
DirectRT's simple list structure syntax, essentially reducing the task to
specifying the stimulus item, its location, and presentation time in a
text editor. The other packages require generating longer and more complex
definitions (Inquisit), or first becoming acquainted with a multiwindow
visual user interface (E-Prime). Therefore, DirectRT would be the package
of choice where assistance with standard paradigms and a quick start with
experimenting is the major objective..." read article
From the SPSP Email
Listserve, January 2nd, 2007:
Greetings: I've been conducting experiments using
pencil/paper or programs designed specifically for a particular problem.
But I would like to start using an all-purpose program (such as MediaLab) to build social
psych experiments. The problem is that I know very little about what's out
there. My primary concerns are 1) ease of use (I don't want to spend a lot
of time "programming") and 2) generality (I'd like to be able to use the
software for a wide range of experiments). Any thoughts on the pros and
cons of existing programs would be very much appreciated! Thanks in
advance.
Brent Simpson
University of South Carolina
Summary
Posted January 7th, 2007:
Hi Everyone: I received a LOT of helpful responses
to my questions about software for building experiments. MediaLab came up most
often but many responders made strong cases for other programs. I've
pasted all the responses I received below, and attached two relevant
documents. Thanks again to everyone who responded.
Brent
- There is a product called SuperLab. It
can do questionnaires, Stroops, subliminal studies. I use it and have
found it to be relatively straightforward.
- You might be interested in the attached manuscript (the pdf file) that
appeared in Experimental Psychology in 2006.
- Some reactions to your question: As a general rule, I have found
that user-friendliness (or ease of use) and generality are negatively
correlated. In other words, whereas the more advanced (and more
difficult) packages or languages will allow you to do virtually
anything, more user friendly programs will eventually always have
certain limitations. This is a potential trade-off you will need to
consider. Having said that, most 'basic' packages are capable of doing
80 to 90% of classic computerised tasks. I know there are many packages
out there. But I have personally mostly used the following programs: (1)
Inquisit (Millisecond
software). I would describe this program as mid-level user-friendly. It
is relatively easy to use, but it does require you to write the syntax
by hand (which can be tricky). Inquisit has its limitations, but is
great to quickly write straightforward programs, especially when
millisecond accuracy is required (e.g. lexical decision, ...). (2)
Visual Basic (Microsoft). This is not a
experiment generator, but a programming language which makes it very
powerful but also rather low in user-friendliness (and therefore not
really suitable for your purpose).
- I have recently done a review of the available packages: Stahl, C.
(2006). Software for generating psychological experiments.
Experimental Psychology, 53, 218-232. You can download
it here: http://www.psychologie.uni-freiburg.de/Members/stahl/publications/expsoft/download
- Can't tell you about other software, but I have been happy with MediaLab. Versatile and easy to use.
- I have been using the Empirisoft suite (Medialab/DirectRT) for several years now, and I can say
that together they have been able to fulfill all of my research needs.
The tutorials are straightforward and quick, and teach you everything
you need to know in a short period of time. Once you have been through
the tutorials, you will be able to program a large variety of
experiments in a short period of time.
- MediaLab is very good and easy to use. For
participant sign ups Sona Systems offers excellent software.
- I love MediaLab. It's very easy to learn and use.
Regarding generality, it has been getting better but there are still
some drawbacks. If nothing has changed recently, then you can only have
one response box (or one questionnaire item) on each screen. This may or
may not be a limitation depending on what you want to do. Also, it's not
as accurate as some may like if they want to measure reaction time and
it probably wouldn't be useful for the subliminal presentation of
stimuli. If you don't have those kinds of needs, then I highly recommend
the program
- The attached document (the MS Word doc) summarizes some reviews on
experiment software. MediaLab/DirectRT is by far the most common and easy to
use program.
- [MediaLab has] excellent support. If you e-mail
them with questions about how to do something, they are always very
helpful and reply promptly. That's been a huge plus for me.
- In response to Brent's question, I've used Medialab and Superlab and found them both to be
user-friendly, with the exception of my stated dilemma, and require
little programming. Medialab has helpful
drop-down menus and the syntax is a simple spreadsheet like Excel. I've
only run reaction time experiments and presented slide shows of stimuli
(easy integration with PowerPoint), but I liked Medialab the best and found it easier to learn
quickly.
- I have been using RiddleMeThis for over a year now to great success.
Check it out for free at loewald.com/RiddleMeThis/
- CDC and e4xchange Inc built a survey program that incorporates
factorial software so you can build experimental vignettes. It's very
easy to use and allows amenities such as dropped cells in an otherwise
fully factorial design (e.g., female + prostate should not be crossed in
an experiment aimed at physicians). I was the project officer for CDC;
Sam Addala, sam@e4xchange.com, is the PI. I should note it's pretty
comprehensive (hosting, data storage), but not free. Feel free to look
over www.e4xchange.com as reconnaissance! It ought
to be just about operational.
- The last two times this question was posted to the list, MediaLab
came out the top favorite.. So just to add my vote -- I use MediaLab and DirectRT in my lab. They are easy, powerful,
and flexible. And I have found the technical support from Blair Jarvis
to be excellent. The company name is Empirisoft.
- Over the past years, I've been working with ERTS, E-Prime, MediaLab, DirectRT,
and Inquisit. Of these software packages, Inquisit stands out as the
most versatile and easy to use. It allows you to program simple
questionnaires AND sophisticated experimental tasks all within the same,
relatively easy to learn programming language (spend 1 or 2 days with
the tutorials, and you're good to go). An additional perk (although one
that I don't use) is its ability to present experiments and
questionnaires via the web. Besides, Sean Draine, the owner of
Millisecond Software and mastermind behind Inquisit, is incredibly
responsive when it comes to troubleshooting and incorporating features
desired by users into new versions of Inquisit. This responsiveness is
something I've come to appreciate over the years. For more info, check
out the Inquisit
website at: www.millisecond.com
- You might check out the SPSP list archive, as this issue has come up
before. In any case, check out http://psychexps.olemiss.edu/ which has a
bunch of experiments run with the Authorware program. This is an
open-source site (you can download the programs that run the experiments
and tweak them to your own purposes). Plus the advantage is that you can
run the programs on the web, which I don't think Medialab and other
programs can do. Another advantage is that you can run your programs on
multiple machines or on an intranet at no extra cost (you buy one copy
of the program for about $600). Authorware does not meet your first
criterion but it does the second. I think your 2 criterion are
negatively correlated, so you can't satisfy both. The learning curve for
Authorware is
steep, but the psychexps folks have an interactive DVD that will teach
it to you, and they used to offer workshops.
- For the ease of use I'd go with Superlab. I've been using it for
years and it is user friendly and can handle a wide range of
experiments. However, if you're thinking about adding fMRI or ERP to
your measures arsenal I'd recommend Presentation. See below for links
describing other software and comparisons between different software
options:
www.socialpsychology.org/software.htm#psychology
psych.hanover.edu/Krantz/software.html
www.empirisoft.com/support/showthread.php?t=17 (a bit biased comparison)
www.bc.edu/offices/help/meta-elements/doc/articles/html/RES-experimental.shtml
xlab.berkeley.edu/news/software.htm
www.psychologysoftwaredistribution.com/Catalogue/catalogue.html
You should also search the SPSP listserve archive, as this question keeps
popping up every now and then.
- [this one is from us!] In case it helps, here is a link to some previous SPSP summaries on
the issue: www.empirisoft.com/pressreleases.aspx. I'd be
curious to know what kind of responses you get this time around! Here
are some additional testimonials not from the SPSP list: www.empirisoft.com/support/showthread.php?t=46
Another package you might want to check out is Inquisit. Although a little trickier to
learn than MediaLab and/or DirectRT, it's also a very
powerful program and worth having in your software collection. My
personal take on the differences between all the major programs can be
found here: http://www.empirisoft.com/support/showthread.php?t=17
- Sadly, there's almost inevitably a trade-off between ease of use of experiment-creating programs and their
flexibility and power. I use an application called Authorware from Macromedia. It
enables you to create standalone .exe programs, or web programs, that do
just about everything. For example, the programs permits text, graphics,
sound, and video to be flexibly handled, accepts variety of user inputs,
records RTs with high resolution, and permits complete control over
things like randomization and counterbalancing. For this, you pay the
price of having to spend a number of weeks getting to grips with it, and
developing a number of standard pieces of code. You can adapt
pre-existing pieces of code however. You can see examples at http://psychexps.olemiss.edu/
- MediaLab and DirectRT will be
perfect for you. I've been using both since 1999 and have nothing but
praise - extremely easy to use, relatively powerful in terms of what you
can do with them.
- Brent, this may be a minority view, but FWIW. I ran several
experiments (including those for my dissertation) with FileMaker Pro, a database manager.
Part of why I'd steer you in that direction is the incredible
versatility of the application. What you learn to use it as a
data-collection tool will then help you use it to solve a wide range of
data-management issues. A somewhat recent improvement is the level of
precision for timestamp functions. It captures 2 decimal places on
seconds. I don't know if that's adequate to all tasks (I think some
reaction-time processes may require more precision). FileMaker Pro
Advanced also allows you to develop a free-standing solution that
can be installed on lab computers, so you can create as many copies as
you need. If you'd like I can give you some more details about the
different kinds of things I've done w/ FMP.
Brent Simpson
University of South Carolina
From the SPSP Email Listserve, July 7, 2005:
This is a long overdue summary of responses to a request I made some time ago concerning research software.
We have also included our own assessment of the products we chose (in good part as a result of these responses).
My original request was:
"Several years ago
there was a request for suggestions concerning software for building
social psychology experiments and collecting data on computers. At that
time, people described programs like Superlab
and Authorware. We are in the process of outfitting a new lab. Given the pace of change in these things, we wondered if people have new or different suggestions for programs of this type."
We had
approximately 20 responses with some people commenting on more than one
program:
Clearly, the most commonly
recommended program was MediaLab (by
Empirisoft) with 13 people commenting positively. MediaLab was highly
recommended for computer-based questionnaires since it allows for
collecting data in a variety of formats (scale response, multiple
response, thought listings, etc.). MediaLab, in general,
was described as being very versatile, user-friendly, and powerful. In
addition, a number of people praised the tech support (Blair Jarvis at
www.empirisoft.com) received for the few problems/bugs encountered while
working with the program. For reaction time measures, DirectRT (also by
Empirisoft and integrates nicely with Medialab) and E-Prime
were recommended with a clear nod to DirectRT. E-Prime was described as
requiring more effort to become completely familiar. For those wanting
more control than MediaLab and DirectRT
can offer, MatLab was recommended (although this program was described as
taking longer to learn and requires more programming knowledge).
RiddleMeThis, Visual Basic, MouseLab, and Inquisit were other recommended
programs. Our own computer technical support person preferred E-Prime
(especially the most recent release), but admitted that learning the
program was much more time consuming and E-Prime was not as user-friendly
as MediaLab. We went
with MediaLab &
DirectRT, and have since used MediaLab for a number
of studies in our lab. We have found MediaLab to be easy to
use and pretty flexible. We have also encountered some issues. For
example, only a single questionnaire item can be presented at a time when
in some instances it would be preferable to present multiple items. In
addition, formatting the questionnaire can be somewhat difficult. In order
to make a single word or phase stand out (e.g., with color, italics…), for
example, an alternate program is needed (e.g., Word, Powerpoint). All that
being said, most problems can be solved if one is creative, and our
research group has become pretty proficient in a relatively short time
frame. We would add our names to the list of those recommending MediaLab
.
Dr. Stephen Wright,
Simon Fraser University
From the SPSP Email Listserve, November 9th, 2004
Thanks for all the responses regarding RT software:
DirectRT, www.empirisoft.com, $475 w/21 day free trial
Inquisit, www.millisecond.com, $395 w/60 day free trial
DMDX, www.u.arizona.edu/~jforster/dmdx.htm, freeware
Affect, www.psy.kuleuven.ac.be/leerpsy/affect, freeware
By far the most recommended software was DirectRT, especially for it's ease of use. Inquisit was a close second, followed by two freeware programs. DMDX was developed by Jonathan Foster. Affect was developed by Hermans et al.
(2002) [Hermans, D., Clarysse, J., Baeyens, F., & Spruyt, A. (2002). I hope this is helpful.
Best,
Tom Denson
University of Southern California
From the SPSP Email Listserve, January 8, 2001:
Dear Colleagues,
It has been a couple of weeks that I have asked you
for experiences with lab software. Your responses were most informative.
Many thanks for your help! Here is a short and impressionistic summary
that is, of course, highly biased by the self-selection of the
respondents: Over all, of the some twenty replies that we have received,
there seems to be a strong preference for MediaLab and the
corresponding RT tool named DirectRT. Only strong suggestions in favour of
MediaLab were
reported, literally every respondent who was using MediaLab seemed to be
pleased with it. A different picture has emerged for other software:
Unfortunately, very few respondents seemed to have first-hand experience
with E-Prime. There still seem to be some problems in adapting old MEL
programs. Supposedly, the programming code has not changed that much and
still seems to be less intuitive than the easy-to-use, clickable tools of
MediaLab.
Nevertheless, the RT measuring tool of E-Prime seems to be comfortable and
exact - something we do not know for Inquisit and Superlab, programs that seem to be
less widely distributed. In sum, both E-Prime and MediaLab
seem to have good
RT-tools that work properly and yield accurate results in the "not so easy to
handle" windows environment.
Fritz Strack
Universitaet Wuerzburg, Germany
|