Research and Publishing. 1: Why Research, Why Publish?

Psychology is a very young science and there is much still to be learnt, in fact most of psychology is not well understood yet. And progress towards better understanding is made through research. This is not simply an academic affair, better understanding of psychology will ultimately improve lives, as trauma, depression etc. become better understood, and better treated.

By research I mean collection of new information, this is not simply the research that one might do for an essay, searching Google, but collecting new information from the real world with surveys, experiments etc. This is why research skills are so important to psychologists. To be a good psychologist one must, at the very least, be able to understand research. Ideally, good psychologists would also be skilled at doing it. Without such research skills it is impossible to be truly evidence based, and if psychologists are not evidence based then they are at serious risks of falling into quackery and pseudoscience. This, unfortunately, characterizes much of current Ecuadorian psychology.

It is therefore imperative for Ecuadorian psychologists to become research active. This is not easy, as many undergraduate psychology degrees in Ecuador lack the capacity to offer strong training in research skills, and there are few postgraduate options. This is unfortunate, as in other counties, such as the UK, psychology graduates are usually the most research-skilled graduates of all. But the situation here in Ecuador will not improve unless we push for it. And the fact is that there are some psychologists here with strong research skills. At the Quito Brain and Behavior Lab we have lots of collaborations with professors both within and without Universidad San Francisco de Quito. We also have lots of research assistants and interns, mainly doing unpaid work. But by collaborating and assisting they are all learning about research and developing skills. The message is that you don’t necessarily need formal training to get into research.

Research when it is performed, must then be published. Research that is not published to a global audience is not of much use. And publication should be in journal articles. I’ll discuss the issues of where, e.g. which journals, in the next blog post. But the for the moment it is important to understand that journal articles are the basic method of transmitting high-level research. Research published  by the London School of Economics suggested that in their survey of social scientists, about 63% or all publications are journal articles, and only about 17% are books. The other 20% is made of various other outputs such as ‘Working and Discussion’ papers. For scientists, including psychologists, I suspect the percentage for journal articles would be even higher. Psychology students should be reading journal articles, and professional psychologists should be writing journal articles.

Academic journals are the most important source of information for academics, including psychologists.

Journals are the basic source because they are peer-reviewed. That means they go through a tough process, being reviewed by anonymous experts from around the world. The majority of articles that are submitted to journals are rejected. And those that are accepted are usually only accepted after being revised based on the anonymous experts’ criticisms, sometimes with several rounds of revisions. It is the strict checking process that means journal articles are more reliable than any other sources of academic information. Newspapers and magazines are written for entertainment not information, and books in general are less trustworthy than many people realize. It is now recognized for example that many, many psychology textbooks contain grossly incorrect descriptions of basic psychological studies and phenomena.

Journals are therefore the basic vehicle of academic research. And researchers are judged mainly on their journal articles. I did my PhD at the Institute of Neurology in London. At the time I was there, the head of the Institute was Professor David Marsden. He was a remarkable scientist-practitioner. and it is said that from the date he graduated as a doctor to his death at age 60, he published 740 journal articles, 208 book chapters, 76 reviews and 100 research letters. An average of one publication every 12 days. This is a truly exceptional research output. Some academics never publish anything in their entire careers. It is notable too that David Marsden was also a clinician. He provides a fine example that clinical and academic are not polar. It may be that clinicians working solely in clinical practice don’t need to research, but those clinicians that also work at a University are also consequently academics. And academics should be involved in research. The point is that academics, whether clinical or not, are judged on their research output, mainly concerning their journal articles.

I have a book on Behavioral Neurology that was presented to David Marsden when winning the American Academy of Neurology Norman Geshwind Award. This is the inscription in the book that is also signed by many leading neuroscientists. An inspirational scientist-practitioner, David Marsden died suddenly just six months after receiving this award. 

So how do we judge research output? People used to talk about impact factors of journals, as a proxy measure for the quality of publications. But really the academic world has moved on from impact factors anyway. Nowadays individual journal articles are all digitally linked together on the internet via their reference lists. It’s easy to see how many times an article has been cited by other journal articles or books. Several different databases calculate this data and display it publicly. The most obvious example is Google Scholar. So rather than look at the impact factor of a journal where a piece of research was published, we can actually see how useful the individual article in question has been. And there is a lot of variation. Some journal articles never, ever, get cited by anybody. Some fly and are cited hundreds of times each year.

Google Scholar shows how many cites every article or book has. We can see in this example that the article by K. Anders Ericsson and Herb Simon was very successful, having been cited 5,688 times since it was published in 1980, which is more useful than knowing the impact factor of the journal. 

So, the real measure of an article’s success, and hence the author’s success, is how many times it has been cited. This is much better than simply looking at the number of articles published, or the impact factors of the journals, or the related metric of whether it’s Q1, Q2 etc. Now we can get a decent estimate of the quality of any individual academic’s research output.

We can look at how many times an academic has been cited. That is a good indication of their success in research. An interesting and commonly used metric is the h-index. This is a single number that captures both the number of articles somebody has published, and how well cited they have been. It is calculated as the highest number of articles that have been cited that same number of times. For example, at the time of writing, I have about 45 published journal articles. Some are highly cited and some are not, overall, I’ve been cited 1,661 times, but I have 18 articles that have all been cited at least 18 times each. My h-index is therefore 18.

This h-index is now a common way to evaluate academics. I’ve been to international conferences in which when the next guest speaker is introduced, their h-index score is publicly announced. By this metric, the most success psychologist in the world has been Herb Simon, who, at the time of writing this, has been cited 323,706 times. He has an h-index of 172. Puts my h-index of 18 into perspective. The research described above from the London School of Economics suggested that in social sciences in general the h-indices of university professors are quite low, ranging from an average of about 2.2 for Law professors to 7.6 for Economics professors. No data was given for Psychology Professors but I’d guess that it may be higher, given the significant research culture in Psychology.

Herb Simon was a truly remarkable academic. A leader in psychology, economics and artificial intelligence, he received a Nobel Prize in 1978 and is the most cited psychologist ever based on his h-index. I attended  the Cognitive Science Society meeting in Edinburgh six months after his death.  Several delegates there were in tears when talking about him. 

So why research and publish? Because it makes psychology better and makes us better psychologists. And it's what academics do. It's that simple.

In the next blog post I'll deal with the issue of where to publish.

'Street Children' of Latin America

One of my research interests is on how socioeconomic factors interact with cognitive and brain development. I’ve done studies with homeless adults in the UK and Japan, and have a study planned here in Ecuador. However, in Quito, the most conspicuous aspect of socioeconomic deprivation from a psychological perspective is the ‘street children’. This is in fact a common feature of several Latin American cities.

A street child that I photographed in Asunción, the capital city of Paraguay. Children selling things to motorists is a common feature of life in many Latin American countries

Street children are a very poorly defined group, and perhaps the term shouldn’t be used at all. It really just means very poor young people who spend a lot of time unsupervised in urban environments. They are not necessarily homeless, and in fact most may have homes to go to and may even be attending school. In Quito the reason for being in the streets is usually to earn money, selling candies on buses and in bars, or shining shoes. I published some musings on this issue of definition a couple of years ago on Favelas@LSE (Pluck, 2015a).

Shoe-shine boys in Quito's Centro Historico. Many 'street children' are really working children. 

Nevertheless, the term sticks, as ‘street children’ is useful shorthand. The research I did was with students of Universidad San Francisco de Quito who helped as research assistants. We interviewed 37 former street children attending an educational program in south Quito. The first part of this was an evaluation of traumatic experiences. That revealed a very high rate of exposure to violence, and consequent post-traumatic stress disorder (PTSD). In fact, about 60% of the children we spoke to met criteria for PTSD (Pluck, Banda-Cruz, Andrade-Guimaraes, Ricaurte-Diaz, & Borja-Alvarez, 2015).  For that part of the research I was interviewed by Universidad San Francisco de Quito and they put a video clip about the research on YouTube.

A video about my street children research made by Universidad San Francisco de Quito

Next we collected data on a group of 26 ‘control’ children who had never been street connected. Both the street children and control children were assessed with several tests of cognitive function. These included a test of fluid intelligence, a test of visuospatial ability, and two tests of executive functioning, the Towers Test and Design Fluency. The reason that we were interested in these functions was that some of the anthropological literature has suggested that street children may fair remarkable well, and may even develop in some ways better than children who stay at home (in probable poverty and possible abuse). On the other hand, the medical literature on substance abuse, psychological trauma, exposure to violence etc. suggests that being street connected is particularly toxic to child development. A literature review that I did a couple years earlier tended to agree with that, in several studies from around the world, samples of street children were shown to perform poorly compared to non-street-connected children (Pluck, 2013).

Another photo of a child street vendor in north Quito, this was featured on the front cover of the Psychologist magazine. Inside was a review article titled 'The Street Children of Latin America' (Pluck, 2015b)

Our own data from Quito tended to agree with the medical view, the street children that we encountered scored quite badly on all of the cognitive tests, significantly worse than the control group. However, we did find some evidence that executive functions might be relatively preserved (Pluck, Banda-Cruz, Andrade-Guimaraes, & Trueba, 2018). This partly agrees with the anthropological perceptive, that the special challenges of being in the street environment as a child may drive the development of some abilities.

We also collected data on Theory of Mind ability in the same sample of street children. Those tests were measures of mentalizing, i.e. the ability to understand the mental contents of other people. We hypothesized that this could be enhanced in children who have to live in a competitive adult world. However, that data has proven quite a challenge to analyse. We are now collecting extra control data and hopefully should be able to add something new to the research literature soon.

Pluck, G. (2013). Cognitive abilities of ‘street children’: A systematic review. Chuo Journal of Policy Sciences and Cultural Studies, 21, 121-133. PDF

Pluck, G., Banda-Cruz, D. R., Andrade-Guimaraes, M. V., Ricaurte-Diaz, S., & Borja-Alvarez, T. (2015). Post-traumatic stress disorder andintellectual function of socioeconomically deprived ‘street children’ in Quito,Ecuador. International Journal of Mental Health and Addiction, 13(2), 215-224. PDF

Pluck, G. (2015a). Challenges and strengths, thinking about´street children´. Favelas@LSE. PDF

Pluck, G. (2015b). The 'street children' of Latin America. The Psychologist, 28(1), 20-23. PDF

Pluck, G., Banda-Cruz, D. R., Andrade-Guimaraes, M. V., & Trueba, A. F. (2018). Socioeconomicdeprivation and the development of neuropsychological functions: A study with“street children” in Ecuador. Child Neuropsychology, 24(4), 510-523. PDF

At the Society for Neuroscience Conference

This week I attended the conference of the Society for Neuroscience in Washington. This is a huge conference, probably the largest neuro- related conference in the world, with around 30,000 attendees every year. The society itself has a membership of about 40,000. Such a large conference and professional organization shows the importance of studies of the brain in modern science. Although it is notable that of the approximately 40,000 members only two are based in Ecuador. We were both there this week.

Many psychologists from around the world were presenting at the conference. Neuroscience and psychological science are two sides of the same coin. You can't really understand the brain without understanding psychology, and vice versa. That's why I was there, not presenting, just for professional development. As a researcher I need to learn of the latest advances as well as widen my understanding in general. As a teacher of psychology, I also need to be constantly updating my knowledge. Ecuador unfortunately has few opportunities for professional development locally, and so it is necessary to attend international events such as this conference in the USA. Keeping up to date with research is particularly important in psychology related to the brain, as it is such a fast-moving field.

The knowledge base of neuropsychology for example has advanced considerably since I was a student in the 90's. I had two very good neuropsychologist teachers, Glyn Humphreys and Jane Riddoch, and I learnt a lot of the latest understanding of the brain from them. But so much has happened since then, the old information is not wrong, but there is so much more detail now. For example, the function of the orbitofrontal cortex has long-been poorly understood. One feature of damage to it, as I learnt as a student, is utilization behavior, the tendency of patients to use objects that are presented to them, even if that use is inappropriate.

This week I watched a presentation by Timothy Behrens (of UCL and Oxford University in the UK) on coding of problem spaces in the orbitofrontal cortex. With magnetoencephalography he has been able to record the different activations in the brain used to solve problems at different stages. And incredibly, they were able to show that when people later rest, their brains run through the same sequence of activations, forward and backwards, over a period of only about 50 milliseconds. This replay phenomenon is likely part of the learning consolidation process. The research is described here just as an example of how rapidly we are increasing our understanding of the psychological processes within the brain.

 

The main lecture hall. Multiple large screens so that thousands of people can see the speaker.

This is the plus side of attending a conference such as Society for Neuroscience. On the other hand, such huge conferences can be overwhelming. With so many presentations it is difficult to choose who to listen to, and as one attendee among 30,000 in a huge conference center there is inevitably a lot of queuing and walking between distant rooms.

For my next conference, I'm planning something smaller and low-key. Jamaica in November 2018 for the Caribbean Psychology Conference.

Neuroeducation

It’s something of an academic fashion at the moment to prefix ‘neuro’ onto other disciplines- neuromarketing, neuromedia studies, neuroeconomics etc. I’m rather dubious about some of these, however one of them does make sense- Neuroeducation. Key topics in neuroscience include learning and memory, language processing, motivation etc., all topics that are relevant to pedagogy. Indeed, a few years ago I took a postgraduate course on Learning and Teaching and was struck by how education as an academic topic was dominated by opinions and hot air, rarely by evidence. So the idea that education specialists are now looking to a true empirical science for guidance is very welcome.

We are doing our part to at the Quito Brain and Behavior Lab to develop this new field and have started our own neuroeducation research program. This is partly in collaboration with the Universidad San Francisco de Quito (USFQ) Masters in Education degree, which focuses on neuroscience and psychology in relation to education. Most of the students on that program are doing their thesis work on neuroeducation topics in the lab. Some of this is on the effects of exercise on symptoms of ADHD in children but also some other projects on socioeconomic effects on the neuropsychological abilities of adolescents.

However our first neuroeducation project started a couple of years ago. The research attempted to find neuropsychological predictors of academic success of undergraduate students. The motivation was the generally very weak relationship between measures of IQ and student performance. Correlations with Grade Point Average (GPA) data tend to be rather meager, with an average of only about r=.2. We hypothesized that tests of ‘frontal lobe’ function, developed in clinical neuropsychology to measure the disruption of behavior following frontal lesions might be better predictors of real life behavior than traditional intelligence tests.

However, there is a problem with that because most tests of frontal lobe function are highly correlated with intelligence tests, and in fact current thinking suggests that a good part of the neurological basis of intelligence is the prefrontal cortex. Nevertheless, not all ‘frontal lobe’ tests are linked to intelligence. Roca et al. in a study of patients with frontal lobe lesions showed that although general intelligence was affected in the patients compared to healthy controls, there were five different ‘frontal lobe’ tests that were impaired independently of the reduced general intelligence (1):

1. The Proverb Test (abstraction)
2. The Faux Pas Test (theory of mind)
3. Hotel Task (multi-tasking)
4. Hayling Test (verbal response suppression)
5. Stop-signal Task (psychomotor inhibition)

We used those five tests, because if they are measuring high level cognition independently of general intelligence, they are ideal candidates to provide an alternative explanation to IQ for why some students excel at university and others don’t. Interestingly the five tests, shown below, seem to measure five quite different general abilities.

The five frontal lobe tests thought to be not linked to general intelligence (1) 

 We administered the five tests plus a standard measure of intelligence, the WAIS-IV, to 64 USFQ students and then looked at which tests best predicated GPA scores. We found that IQ, as expected, was not a great predictor, in fact it predicted only about 12% of the variance in GPA. However, two of the tests that we administered, the Stop-signal Task and the Hayling Test increased the predicative power to about 24%. The important thing is that these two tests are independently associated with GPA, they are explaining some of the variance in GPA not explained by IQ scores. Interestingly, both of these tests are of response inhibition. It seems that as well as being smart, you need to have good behavioral control to get the good grades.

A further interesting observation is that these two tests are both linked to the right lateral prefrontal cortex (2, 3). In contrast, recent neuropsychological studies have suggested that general intelligence is mainly located in the left hemisphere (4). So do we have two parallel and independent contributions to academic success, one intelligence-based in the left hemisphere, and one about behavior control in the right? That’s what our results suggest. Below we can see this idea graphically, where I’ve marked the left-hemisphere areas linked to general intelligence, and the right hemisphere areas linked to response inhibition.

Lateralised regions for response inhibition in the left hemisphere (2,3) and general intelligence in the right hemisphere (4)

That research was published just last week in the journal Trends in Neuroscience and Education (5). We are now further developing the idea with a study in the workplace, using the same tests. We are interested in whether response inhibition will again emerge as a predictor of workplace success. Or will perhaps one of the other ‘frontal lobe’ tests, such as the Hotel Task, a test of multi-tasking, be better predictors?

1. Roca, M., et al.. (2009). Executive function and fluid intelligence after frontal lobe lesions. Brain, 133(1), 234-247.
2. Aron, A. R., et al.. (2003). Stop-signal inhibition disrupted by damage to right inferior frontal gyrus in humans. Nature Neuroscience, 6(2), 115-116.
3. Robinson, G. A, et al.. (2015). Verbal suppression and strategy use: a role for the right lateral prefrontal cortex?. Brain, 138(4), 1084-1096.
4. Barbey, A. K., et al.. (2012). An integrative architecture for general intelligence and executive function revealed by lesion mapping. Brain, 135(4), 1154-1164.
5. Pluck, G. et al.. (2016). Separate contributions of general intelligence and right prefrontal neurocognitive functions to academic achievement at university level. Trends in Neuroscience and Education, 5, 178–185.

New Master's Degree in Psychology

The management here at Universidad San Francisco de Quito have given the go-ahead for us to develop a new master’s degree for training clinical psychologists. It will, without a doubt, be the best clinical psychology training course in Ecuador. The core trainers all have doctoral degrees in clinical psychology from highly respected institutes in the USA. In addition to this we have several full-time psychology professors with research PhDs from top universities in the USA and UK. None of the other universities can muster such academic weight, with most of their professors having only masters’, or lower, degrees. And often those master’s degrees were earned online.

The program will be directed by Dr Richard West. It will follow an evidence-based model with the psychotherapy component focused on Cognitive Behavioral Therapy (CBT). This makes perfect sense, CBT has much more evidence to support its clinical efficacy than any other psychotherapy.  

Such a course is sorely needed in Ecuador. Due to a lack of appropriate regulation, people with only undergraduate level training can call themselves ‘psychologist’ or even ‘clinical psychologist’. Consequently, there is much quackery passed off as professional practice. The situation is changing though and soon a postgraduate qualification will be needed to use the title ‘psychologist’- hence the new master’s program.

It won’t be ready to go for a while, it still needs to be approved by the national authorities, but it will likely have its first intake in 2016. In addition, a second master’s degree is being planned that will take a more eclectic approach and focus more on sub-clinical clients and counselling skills. That will be directed by Dr Teresa Borja. 

The State of Psychology in Ecuador

Psychology here in Ecuador is a strange thing. Firstly, there is currently very little legal protection over the professional title, anybody with a degree in psychology is a ‘psychologist’. I could legally set up private practice here as a Clinical Neuropsychologist if I wanted to, despite having no clinical training. Indeed there are lots of ‘Clinical Psychologists’ here who have no clinical training. Their titles being based on postgraduate academic qualifications from abroad. Ironically, those who do have clinical training usually only have an undergraduate degree in ‘Clinical Psychology’. So most trained clinical psychologists have no postgraduate qualifications.

There is also a tendency to faddishness. Sub-academic pseudo-psychology is peddled by numerous life coaches, brain trainers, aroma therapists and other cranks. When professional psychology is considered an undergraduate degree, then this is understandable. Many students are just not sufficiently trained to spot pseudoscience. Not surprisingly, what passes as mental health care sometimes borders on superstition and charlatanism. For example, I’ve heard of some ‘clinical psychologists’ using tarot cards as part of their therapy, others horoscopes. Many use hypnotic regressions to ‘past lives’.

How can a country have gotten to a state where graduates (aka Clinical Psychologists) think that tarot and horoscopes are real psychology? One reason is that supernatural beliefs are very common here anyway, so things like tarot are respected by the public and tend to go unchallenged. Another issue is the lack of expertise in Ecuador, there are simply not enough people in the country with doctorate qualifications to staff the universities. Thus, many professors have only masters degrees, and of varying quality at that. Worse, for many years there were few checks on the validity of qualifications, and so there are professors here with PhDs that were bought over the internet. These frauds are the likely source of much of the full-blown mumbo jumbo that passes as psychology. When ‘real’ psychology is taught, it’s frequently based on pop-science paperbacks and TED talks, with little distinction made between entertainment and academia.

Fortunately, the current government is taking steps to improve standards in higher education. For example they are recruiting more experts from abroad and sending Ecuadorian students to top universities in the USA and Europe for postgraduate training (there are very few postgraduate opportunities within Ecuador). The students get their fees paid and generous grants if they agree to return to Ecuador afterwards. The government is also checking all professors’ qualifications and adding the details to an open-access database, so the public can check who really has professional training. In addition, legal steps are being taken to require a postgraduate qualifications for professional psychological practice. With time these steps should improve the current dire situation nationally.

Independent of the government provision, at Universidad San Francisco de Quito we now have the best trained psychology faculty in the country, including several professors with (real) doctorates from respected universities in North America. Some of these are Ecuadorian, some foreigners. We have brought skills in research and clinical work that have generally not been available in Ecuadorian psychology departments before. The potential therefore exists to greatly improve psychology training, and over time, the quality of mental health care provided by the country's next generations of psychologists. 

Quito Brain and Behavior Laboratory

Yesterday I gave a presentation at the inaugural Brain Meeting of the Quito Brain and Behavior lab. The presentation was on neurocognitive function and socioeconomic status. Not everybody’s cup of tea, but it attracted about 20 academics from various fields. That was what we wanted, there are lots of people here who want to do some brain based research, and the monthly Brain Meetings are a way to get them together. Hopefully, this will encourage research and collaborations. The meetings will be held at 4pm every first Tuesday of the month during the teaching semesters.

The advertising poster for the inaugural Brain Meeting of the new lab. 

The Brain Meetings are organized by the Quito Brain and Behavior lab, which is being set up at Universidad San Francisco de Quito by me and Dr. Ana F. Trueba, a psychophysiologist. With her interest and expertise, and my background in neuropsychology and psychopathology, we are forming an international class research group on brain sciences. We both have a lot of experience in running projects, and publishing data in international journals, so forming the lab is helping us to extend this. In addition we have a paid research assistant and several graduate and undergraduate volunteers who are gaining research experience in the lab. Recently an expert in genetics and cognitive neuroscience also joined the group. 

We currently have two projects running, one on predication of academic performance with neuropsychological tests, and the other a validation study on various tests of premorbid function. We are also developing a new battery of cognitive tests that could be used in various clinical studies, for example to measure cognitive decline in dementia. The development of these tests is important because there are currently few assessment tools that are validated for use in Ecuador. By producing our own, we can facilitate future research. We also have studies planned on the role of cortisol and inflammation markers in sub-clinical psychopathology, and some experimental studies of the effects of the hormone and neuromodulator oxytocin. As more collaborators join us, the number and range of projects will increase.

The lab houses a reasonable neuropsychological test library, in fact possibly the biggest in Ecuador. This includes several commercial tests (such as Wechsler tests, Delis Kaplan Executive Function System) as well as many more experimental tests. In addition, we already have some physiological recording equipment, and the university is in the process of buying eye-tracking systems, which will be used in neuromarketing research. So there is plenty of potential. The next Brain Meeting will be held on March 3rd at 4pm at Universidad San Francisco de Quito.